LXR Installation on Ubuntu 10.10

If you are into programming and dealt with large scale projects I’m sure you had your time thinking about how great it would have been to have an application that you can look up your previous functions, layouts and definitions. While developing and maintaining a large scale on going project, it is crucial to have access and chance to compare your current version with the previous versions with ease. Cross referencing is what you are totally looking for. It came out mainly from GNU/Linux kernel development process where there are constant version releases and tons of lines of code.

Linux cross referencing (LXR) let programmers to browse the different versions of the code, look up functions by their names, see where you used that exact same variable or function definition or even just search random words that you hope to find in the code. It gives the programmer and opportunity to quickly skim the previous code, find the needed references and continue his development process. Luckily, cross referencing layout has been made public so you can set up your own cross referencing site and start to use this great tool.

In this article, I will give you every step you need to go through on an Ubuntu 10.10 clean system. Before we start, you need to download some packages from your Ubuntu universe repositories. You need to have apache2, MySQL and exuberant-ctags. To get them, simply use the following line of code:

sudo apt-get install apache2 mysql-server exuberant-ctags

While installing mysql-server package, you will be asked to create a root user password for MySQL. Go ahead and create your password and finish installation. After that, you need to install a perl library, in order for your XR to connect with your database. Overall, you need to download the following packages related to perl, as it is our backbone for the cross referencing system:

sudo apt-get install libdbi-perl libdbd-mysql-perl libfile-mmagic-perl libapache2-mod-perl2

 

In this instructions, I will use the latest avaible realse of LXR, which is 0.9.10 at the moment. LXR source codes are hosted on sourforge.net with the following address http://sourceforge.net/projects/lxr/files/stable . You will download the code in a tgz format from here. I will assume that you download that package to your Desktop with a directory path of ~/Desktop/lxr-0.9.10.tgz .

First of all you need to open the contents of this compressed package in to /usr/local/share . I will also change the name of the resulting directory to “lxr” for ease. Use the following code in order to do that:

cd /usr/local/share
sudo mv /home/aselsan/Desktop/LXR/lxr-0.9.10.tgz .
sudo tar -zxf lxr-0.9.10.tgz
sudo mv lxr-0.9.10 lxr
cd lxr

Now, at this point of time, we are done with getting the required packages to their places. Now, it is time for configuration of the system components. Firstly, we will start with MySQL. You will login to your MySQL sever with the root user using the password that you created previously.

mysql -u root -p #Enter the password when asked.

Inside your MySQL server, run the following queries to set up the system for your cross referencing site:

CREATE USER 'lxr'@'localhost' identified by 'foo'; #Creates the user "lxr" with the password "foo"
\. initdb-mysql #Initializes the lxr database by creating the tables and give "lxr" user the needed permissions.

What you need to be careful about is that, to pay attention where you are when you log in to your MySQL. In order to do that initialization process you need to be in your lxr directory which is /usr/local/share/lxr. Otherwise, when you try to run the initdb-mysql script, it will complain about can’t finding the script. After the process is completed leave your MySQL server with the command “exit”. That’s all you need to do for configuring your database.

Now we need to configure the LXR. Remember, we are still in the lxr directory. In order to configure, we need to copy the default lxr.conf file to the root of lxr directory.

sudo cp templates/lxr.conf .

Now we can configure. Use the editor of your choice to do the configuration. I will use vim.

sudo vim lxr.conf

Now firstly, we will use swish-e instead of glimse for our sort of text search engine. The reason is, glimpse is not free for commercial use whereas swish-e is completely free software. Thus comment out the glimpse related lines as the following:

#, 'glimpsebin'   =>'/usr/local/bin/glimpse'
#, 'glimpseindex'   => '/usr/local/bin/glimpseindex'

Now we will give the executable swish-e path to lxr, which you can find out by using “which swish-e” command configure it according to your system. My system yielded the following result:

, 'swishbin'   => '/usr/bin/swish-e'

You should pay attention if the user can reach that executable. If you face any problems regarding to that, just “chmod” and enable users to have execution rights. Next step will be the configuration of ctags and configuration of lxr:

, 'ectagsconf' => '/usr/local/share/lxr/lib/LXR/Lang/ectags.conf'
, 'genericconf' => '/usr/local/share/lxr/lib/LXR/Lang/generic.conf'

Now we need to define the address for our server’s URL. Firstly give the base URL as:

,'baseurl' => '
http://localhost/lxr'

Then you can add IP addresses of your server and hostname related network addresses to URL aliases like the following:

, 'baseurl_aliases' =>
                        [ 'http://17.2.0.193/lxr']

Now it is time for us to point out the source folder that we would like to keep our source code and index for cross referencing. This will be the main folder which will contain subdirectories like v1, v2 and so on. So that you can add your respective version source under these subdirectories. As an example, I will be using a directory that I just created on my desktop like the following:

, 'sourceroot' => '/Desktop/lxr_source'

Now I have subdirectories named v1 and v2. I have my source code for my application under these subdirectories. We also need to tell LXR to look into these directories and index them. We will set this in the lxr.conf file as the following line:

, 'range' => [qw(v1 v2)]

You can also put the character “/” in order to tell LXR to index everything under your main source directory. However, if you do that, you will lose the option to have different versions to compare. Thus, I do not suggest that. Lets also set which version should come up when we get on the site by default. I will set it to version 2:

, 'default' => 'v2'

Now that we are almost done expect the database connection details and swish-e index directory. You can see that MySQL login information related part is commented out. Delete “#” characters and open them and set them to your regarding MySQL username and password. You have to use the username lxr, however if you set the password for this user, you need to use that instead of foo:

,'dbpass' =>'foo'
, 'dbuser' => 'lxr'

Finally, we only need to give swish-e a bit of privacy and point out a designated folder for it to keep its search indexes. Don’t put this directory under your main lxr path or the directory where you put your source codes. Setup another location. For this tutorial, I just used a directory on my desktop as the following:

, 'swishdir' => '/home/aselsan/Desktop/swish/'

Now we are practically done with the lxr.conf file. For more customization, you can read the file as it is pretty self explanatory and adjust it according to your needs. Now we can start our indexing operation. I would like to remind you that if you have a large source code, this process can take a bit of time and can load your cpu, so just be aware and while you are still in /usr/local/share/lxr directory execute the following command:

./genxref --url=http://localhost/lxr --allversions

This wil index every non-indexed part of your source code versions. If you want to only index one version, instead of “–allversions” you need to use a syntax like “–version=v1″. Once you index the whole of your source code whenever you run this indexing operation it will skip the already indexed part and only index the newly changed and added parts. Also, remember to index every time you add or take out something from your source directory.

Now I will move on to the apache2 server configuration. Just create a lxrserver.conf file with your editor:

sudo vim lxrserver.conf

Inside this file, write the following piece of apache directives:

Alias /lxr /usr/local/share/lxr
 
 AllowOverride All

Now we need to move this file into apache2′s conf.d directory which can be done as given below:

sudo cp lxrserver.conf /etc/apache2/conf.d/

To finish apache2 configuration, we just need to use one more command while we are still in the directory /usr/local/share/lxr :

sudo cp .htaccess_cgi .htaccess

Now, theoretically, you should be able to see your cross reference site at http://localhost/lxr/source . However, I faced some problems regarding to perl in my experience. After couple of hours of messing around and reading through a lot of material and code, I came to the a solution. If your LXR is not working at this point, it is mostly likely perl related and in order to solve this you need to change the first line in source, ident, diff and search files in /usr/local/share/lxr . You can see the in the first line of these files the following line is written:

#!perl -T

This caused confusion for my operating system and failed the execution of perl scripts. So that I decided to look up where my perl executable is with “which perl” command. I saw that it is located in /usr/bin/perl. So I changed all the first lines for these 4 lines as the following:

#!/usr/bin/perl

Also, make sure that you reboot your apache2 web server as it adjust itself to the changes are reboots. You can use the following command:

apache2ctl restart
LXR logo

http://lxr.linux.no/

Now, you should have a fully functional cross reference site for your projects and please remember to give back to the open source community! If you want you can also check for more information on LXR’s website. Enjoy!

No Trackbacks

You can leave a trackback using this URL: http://egemengozoglu.com/blog/2011/07/lxr-installation-on-ubuntu-10-10/trackback/

21 Comments

  1. Fine!
    But there is a syntax error in baseurl_aliases: remove the trailing ”

    I would not advise to remove the -T option in the first lines of the scripts. This option allows Perl to make many “safety” tests on the data and detect early some inconsistencies. Unless it causes big trouble with your system/Perl, it is better to leave it on.

    You could add a link to the main LXR site where I try to maintain uptodate information and a troubleshooting guide fed by user feedback.

    Best regards,
    ajl
    LXR administrator

    Posted August 3, 2011 at 6:39 am | Permalink | Reply
    • egemengozoglu

      Thanks for you interest ajl!

      I corrected the syntax error in baseurl_aliases now! About the “-T” option, I have tried to install the system over and over again in other Ubuntu 10.10 distros yet I have came across with the same problem, so I used my solution to keep the system up. Do you have any idea what might be the problem? I have also added the link to http://lxr.sourceforge.net/ as you suggested! You can put my article up there as a guide if you like too! I would like to contribute to the project as much as I can!

      Best regards,

      Egemen

      Posted August 4, 2011 at 12:21 pm | Permalink | Reply
    • Surprisingly well-written and informative for a free onnlie article.

      Posted December 18, 2011 at 4:27 am | Permalink | Reply
  2. crazier

    Hello,this is a good guide,but there is some syntax error
    “. initdb-mysql” , It should like this “\. initdb-mysql” .

    Posted September 26, 2011 at 11:21 am | Permalink | Reply
    • egemengozoglu

      Thanks for pointing out the typo! I have just fixed it! I’m glad that its useful for you! Enjoy!

      Posted September 26, 2011 at 11:37 am | Permalink | Reply
      • crazier

        Hi, My Linux distribution is ubuntu 11.04, so there is something not identical with your guide, “sudo apt-get install libdbi-perl libdbi-mysql-perl libfile-mmagic-perl”
        I just can find the package “libdbd-mysql-perl” rather libdbi-mysql-perl , So I’m not sure it’s a a slip of a pen or because of the different distro.

        And I follow your guide,and succeded making my own lxr. but I find it’s not convenient like the site “http://lxr.linux.no” which have a small window on the upper left . So how can I have the same window? Thanks!

        Posted September 26, 2011 at 12:40 pm | Permalink | Reply
        • egemengozoglu

          Thanks for careful reading! It is indeed a typo and it should be “libdbd-mysql-perl” I have just fixed that error. And about the page layout, I think you meant upper right instead of upper left as there is a drop down list and a search bar on the upper right corner. The guide is for installing the LXR distro and what you see on that webpage is a polished version of LXR. You can create that by playing around with the HTML and CSS a little bit and add some jquery if you would like to have a better front-end. However, I have not created a theme like that so far. I am glad the guide worked for Ubuntu 11.04 as well.

          Posted September 26, 2011 at 12:52 pm | Permalink | Reply
          • crazier

            thank your reply! I knowed! And Have a nice day!

            Posted September 26, 2011 at 1:07 pm | Permalink
        • Yeah that’s what I’m taknilg about baby–nice work!

          Posted December 17, 2011 at 7:10 pm | Permalink | Reply
  3. Some genuinely interesting details you have written. Aided me a lot, just what I was looking for : D.

    Posted October 2, 2011 at 4:06 am | Permalink | Reply
  4. Hello,
    I added a link to this page on the LXR site.
    You still have HTML problems: the “greater-than” sign is displayed as character entity name “>”. Can you fix it?
    You made a serious semantic error on your ‘sourceroot’ example. You write the directory path starting with ~, meaning the user home directory. This is fine at the time you configure lxr.conf, but wrong when you query LXR with http://localhost/lxr/source. At that time, Apache is in control. Its user-id is generally apache, not yours. Then LXR cannot access a non-existent directory (since Apache has no /home/apache directory, which anyway would not be yours). This is why ‘sourceroot’ must be expressed as an absolute path.
    More details at http://lxr.sf.net/LxrConf/lxrconffilepaths.shtml.

    Best regards,
    ajl
    LXR administrator

    Posted December 10, 2011 at 4:21 pm | Permalink | Reply
    • egemengozoglu

      Hi,

      I hope this guide will be useful for others as well! Thank you for adding it to the main LXR website!

      I fixed the “>” character problem, I don’t know how it occurred but it was a quick fix! I also changed the “~” to avoid any confusion. At the time I was writing this article I assumed people would just use absolute paths and used “~” for indicating it can vary for your paths.

      Thanks for the help!

      Best Regards,

      Egemen

      Posted December 11, 2011 at 7:41 pm | Permalink | Reply
  5. Heck of a job there, it abosultely helps me out.

    Posted December 18, 2011 at 12:18 am | Permalink | Reply
  6. pankaj

    Hi egemen,
    Thanks! your blog help me to set up lxr,

    After performing all the steps suggested by you, i was unable to see anything on
    http://localhost/lxr/source

    Later on i found that there was mod_perl was missing in the setup!
    i perform following steps:

    apt-get install libapache2-mod-perl2
    #restart apache so that it loads mod_perl
    apache2ctl restart

    And everything worked.

    please include this in your guide, it may help others.

    Posted December 28, 2011 at 9:49 am | Permalink | Reply
    • egemengozoglu

      Hi,

      Glad you found the guide useful! I will add those steps regarding to mod_perl and apache reboot to the guide! Thanks for the feedback!

      Best Regards,

      Egemen

      Posted December 30, 2011 at 4:05 pm | Permalink | Reply
  7. xujinna

    hi,
    I read this guide and build the lxr successfully.I want to use it to check android source code, but I find that it doesn’t support java code index, please help.Thanks

    Best regards,
    jinna xu

    Posted December 30, 2011 at 11:37 am | Permalink | Reply
    • egemengozoglu

      Hi,

      Glad this guide was helpful for you! Even though I haven’t used it myself to browse Java code, LXR should be able to index Java. It is stated in the website that it has support for Java: http://sourceforge.net/projects/lxr/

      Please leave feedback here if you solve your problem so it can be a help for others in the future.

      Best Regards,

      Egemen

      Posted December 30, 2011 at 4:02 pm | Permalink | Reply
  8. Dharmesh Jani

    I followed the exact procedure as mentioned here. But I think the apache server is not working correctly.
    It gives me an error:
    “Syntax error on line 2 of /etc/apache2/conf.d/lxrserver.conf:
    AllowOverride not allowed here
    Action ‘configtest’ failed.
    The Apache error log may have more information.
    …fail!”

    If I uncomment it, when I access http://localhost/lxr/source it is showing me the source script that is installed under /usr/local/share/lxr.

    Is there anything wrong I am doing here?

    Posted January 11, 2012 at 11:32 pm | Permalink | Reply
    • egemengozoglu

      Hi,

      Have you fiddled around with your apache2 configurations beforehand? Also, maybe I can be more help, if you can post your Apache error log. It might have more information about what’s going wrong.

      Egemen

      Posted January 12, 2012 at 8:35 pm | Permalink | Reply
  9. akam

    I am getting following error, trying to configure lxr on U12.04LTS

    mysql> \. initdb-mysql
    ERROR:
    Failed to open file ‘initdb-mysql’, error: 2
    mysql>
    What am I missing ?

    Posted March 19, 2013 at 5:51 pm | Permalink | Reply
  10. Vadiraj Panchamukhi

    Hi,
    Nice step by step writeup. I followed the steps mentioned here. When I access http://localhost/lxr/source I get the contents of the file being displayed in the browser. But when I access http://localhost/lxr I get “Forbidden” error. The apache error log says “Directory index forbidden by Options directive: /usr/local/share/lxr/”. Am I missing something?

    Posted August 23, 2013 at 5:16 am | Permalink | Reply

Post a Comment

Your email is never shared. Required fields are marked *

*
*