If you are into programming and dealt with large scale projects I’m sure you had your time thinking about how great it would have been to have an application that you can look up your previous functions, layouts and definitions. While developing and maintaining a large scale on going project, it is crucial to have access and chance to compare your current version with the previous versions with ease. Cross referencing is what you are totally looking for. It came out mainly from GNU/Linux kernel development process where there are constant version releases and tons of lines of code.
Linux cross referencing (LXR) let programmers to browse the different versions of the code, look up functions by their names, see where you used that exact same variable or function definition or even just search random words that you hope to find in the code. It gives the programmer and opportunity to quickly skim the previous code, find the needed references and continue his development process. Luckily, cross referencing layout has been made public so you can set up your own cross referencing site and start to use this great tool.
In this article, I will give you every step you need to go through on an Ubuntu 10.10 clean system. Before we start, you need to download some packages from your Ubuntu universe repositories. You need to have apache2, MySQL and exuberant-ctags. To get them, simply use the following line of code:
sudo apt-get install apache2 mysql-server exuberant-ctags
While installing mysql-server package, you will be asked to create a root user password for MySQL. Go ahead and create your password and finish installation. After that, you need to install a perl library, in order for your XR to connect with your database. Overall, you need to download the following packages related to perl, as it is our backbone for the cross referencing system:
sudo apt-get install libdbi-perl libdbd-mysql-perl libfile-mmagic-perl libapache2-mod-perl2
In this instructions, I will use the latest avaible realse of LXR, which is 0.9.10 at the moment. LXR source codes are hosted on sourforge.net with the following address http://sourceforge.net/projects/lxr/files/stable . You will download the code in a tgz format from here. I will assume that you download that package to your Desktop with a directory path of ~/Desktop/lxr-0.9.10.tgz .
First of all you need to open the contents of this compressed package in to /usr/local/share . I will also change the name of the resulting directory to “lxr” for ease. Use the following code in order to do that:
cd /usr/local/share sudo mv /home/aselsan/Desktop/LXR/lxr-0.9.10.tgz . sudo tar -zxf lxr-0.9.10.tgz sudo mv lxr-0.9.10 lxr cd lxr
Now, at this point of time, we are done with getting the required packages to their places. Now, it is time for configuration of the system components. Firstly, we will start with MySQL. You will login to your MySQL sever with the root user using the password that you created previously.
mysql -u root -p #Enter the password when asked.
Inside your MySQL server, run the following queries to set up the system for your cross referencing site:
CREATE USER 'lxr'@'localhost' identified by 'foo'; #Creates the user "lxr" with the password "foo" \. initdb-mysql #Initializes the lxr database by creating the tables and give "lxr" user the needed permissions.
What you need to be careful about is that, to pay attention where you are when you log in to your MySQL. In order to do that initialization process you need to be in your lxr directory which is /usr/local/share/lxr. Otherwise, when you try to run the initdb-mysql script, it will complain about can’t finding the script. After the process is completed leave your MySQL server with the command “exit”. That’s all you need to do for configuring your database.
Now we need to configure the LXR. Remember, we are still in the lxr directory. In order to configure, we need to copy the default lxr.conf file to the root of lxr directory.
sudo cp templates/lxr.conf .
Now we can configure. Use the editor of your choice to do the configuration. I will use vim.
sudo vim lxr.conf
Now firstly, we will use swish-e instead of glimse for our sort of text search engine. The reason is, glimpse is not free for commercial use whereas swish-e is completely free software. Thus comment out the glimpse related lines as the following:
#, 'glimpsebin' =>'/usr/local/bin/glimpse' #, 'glimpseindex' => '/usr/local/bin/glimpseindex'
Now we will give the executable swish-e path to lxr, which you can find out by using “which swish-e” command configure it according to your system. My system yielded the following result:
, 'swishbin' => '/usr/bin/swish-e'
You should pay attention if the user can reach that executable. If you face any problems regarding to that, just “chmod” and enable users to have execution rights. Next step will be the configuration of ctags and configuration of lxr:
, 'ectagsconf' => '/usr/local/share/lxr/lib/LXR/Lang/ectags.conf' , 'genericconf' => '/usr/local/share/lxr/lib/LXR/Lang/generic.conf'
Now we need to define the address for our server’s URL. Firstly give the base URL as:
,'baseurl' => ' http://localhost/lxr'
Then you can add IP addresses of your server and hostname related network addresses to URL aliases like the following:
, 'baseurl_aliases' => [ 'http://18.104.22.168/lxr']
Now it is time for us to point out the source folder that we would like to keep our source code and index for cross referencing. This will be the main folder which will contain subdirectories like v1, v2 and so on. So that you can add your respective version source under these subdirectories. As an example, I will be using a directory that I just created on my desktop like the following:
, 'sourceroot' => '/Desktop/lxr_source'
Now I have subdirectories named v1 and v2. I have my source code for my application under these subdirectories. We also need to tell LXR to look into these directories and index them. We will set this in the lxr.conf file as the following line:
, 'range' => [qw(v1 v2)]
You can also put the character “/” in order to tell LXR to index everything under your main source directory. However, if you do that, you will lose the option to have different versions to compare. Thus, I do not suggest that. Lets also set which version should come up when we get on the site by default. I will set it to version 2:
, 'default' => 'v2'
Now that we are almost done expect the database connection details and swish-e index directory. You can see that MySQL login information related part is commented out. Delete “#” characters and open them and set them to your regarding MySQL username and password. You have to use the username lxr, however if you set the password for this user, you need to use that instead of foo:
,'dbpass' =>'foo' , 'dbuser' => 'lxr'
Finally, we only need to give swish-e a bit of privacy and point out a designated folder for it to keep its search indexes. Don’t put this directory under your main lxr path or the directory where you put your source codes. Setup another location. For this tutorial, I just used a directory on my desktop as the following:
, 'swishdir' => '/home/aselsan/Desktop/swish/'
Now we are practically done with the lxr.conf file. For more customization, you can read the file as it is pretty self explanatory and adjust it according to your needs. Now we can start our indexing operation. I would like to remind you that if you have a large source code, this process can take a bit of time and can load your cpu, so just be aware and while you are still in /usr/local/share/lxr directory execute the following command:
./genxref --url=http://localhost/lxr --allversions
This wil index every non-indexed part of your source code versions. If you want to only index one version, instead of “–allversions” you need to use a syntax like “–version=v1″. Once you index the whole of your source code whenever you run this indexing operation it will skip the already indexed part and only index the newly changed and added parts. Also, remember to index every time you add or take out something from your source directory.
Now I will move on to the apache2 server configuration. Just create a lxrserver.conf file with your editor:
sudo vim lxrserver.conf
Inside this file, write the following piece of apache directives:
Alias /lxr /usr/local/share/lxr AllowOverride All
Now we need to move this file into apache2’s conf.d directory which can be done as given below:
sudo cp lxrserver.conf /etc/apache2/conf.d/
To finish apache2 configuration, we just need to use one more command while we are still in the directory /usr/local/share/lxr :
sudo cp .htaccess_cgi .htaccess
Now, theoretically, you should be able to see your cross reference site at http://localhost/lxr/source . However, I faced some problems regarding to perl in my experience. After couple of hours of messing around and reading through a lot of material and code, I came to the a solution. If your LXR is not working at this point, it is mostly likely perl related and in order to solve this you need to change the first line in source, ident, diff and search files in /usr/local/share/lxr . You can see the in the first line of these files the following line is written:
This caused confusion for my operating system and failed the execution of perl scripts. So that I decided to look up where my perl executable is with “which perl” command. I saw that it is located in /usr/bin/perl. So I changed all the first lines for these 4 lines as the following:
Also, make sure that you reboot your apache2 web server as it adjust itself to the changes are reboots. You can use the following command:
Now, you should have a fully functional cross reference site for your projects and please remember to give back to the open source community! If you want you can also check for more information on LXR’s website. Enjoy!