UPDATE I've written a script called bundle_script which does most of this (maybe even all) for you. You need to be running an OS with yum, since it'll be grabbing packages from there; other than that, no restrictions. The script installs the packages you need, gives you the right reaper.so for your architecture, and customizes the setup for your machine. Right now it uses a slightly dated version of the GN code and gets all its stuff from rhodes.utmem.edu, so you'll need an account on that machine.

Today I set up a new bundle server using Fan Zhang's modified code; this document will outline the procedure I used. Much of this is based on Evan Williams' installation procedure.

What You Need to Know

I'm going to reiterate what Evan said. If you can use a command line in Linux and have even moderate experience in MySQL, you should be fine.

Operating System

I have successfully installed the bundle on Centos 5, both 32-bit and 64-bit. The 64-bit version takes a little more work; you'll have to download the source code for the QTLReaper project from Sourceforge, compile it yourself, and copy the new reaper.so in place of the old one; otherwise everything is the same.

Software

Apache should come preinstalled, along with Python. I installed all updates for these (this may or may not be necessary). There are a bunch of extra software packages you'll need to get for things to work; I used yum for this. Make sure port 80 is open for Apache; to do this, modify /etc/sysconfig/iptables. Copy a line that opens a port, but replace the port number with 80. On my system, the line looks like

-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT

• The python-to-mysql interface is called MySQLdb? when imported in the code, or MySQL-Python on yum. Install it.
• Any other python package that looks even vaguely relevant. (Examples! Sanskrit language support--irrelevant. C language support--relevant. Support for a package used solely to portray, in ASCII, the faces of ex-U.S. presidents--probably irrelevant, but install it anyway. You'd be surprised what packages are essential for Genenetwork.)
• Install gcc (the gnu C compiler) and everything associated with it (also found on yum). Be warned, some of these start with "compat".
• Install a package called "lapack" (on yum).

The Database

The GN database is massive. Setting this up locally gave me the distinct advantage of not having to install a new database; rather, I used one already set up on the network. We're currently using binary logs to backup, so there isn't an official way to get the latest database dump. Talk to one of us.

GN Code Importing

The modified code can be found on our ftp site, tyche.utmem.edu. Copy the latest gnshare*.tbz and extract it to "/" (i.e. "cd /gnshare" should put you in the gnshare directory). Copy the file "admin.py" from /gnshare/gn/support/admin into /gnshare/gn/web/webqtl/webqtl.

Additionally, you should grab the Apache configuration file from that machine, namely "httpd.conf" and located in /etc/httpd/conf/, and replace the file on your machine (also located, hopefully, in /etc/httpd/conf/) with our file.

You'll need a few more packages from our server. Some of them are old and hard to find; some of them have newer versions and you won't be using ours. Either way, they're all in a folder called thirdparty/, and it's not on olympia. Go to webqtl.utmem.edu and get it, the full path being /gnshare/gn/thirdparty/. There are two more files of interest, direct.so and reaper.so; they're located in /gnshare/gn/support/binary/. Get them as well. I'll discuss what to do with this stuff in a moment.

GN Code Customization

Thanks to Fan Zhang, you no longer have to run a script to convert from the old machine's ip to the new one. Yay! You still have to make a few changes, though.

• First, make sure Python can access your database. Open the file "webqtlConfig.py", located in gnshare/gn/web/webqtl/webqtl/, and change the four constants in the section labelled "Added by Fan Zhang" to match your configuration. Specifically: set MYSQL_SERVER to the IP of the computer on which you installed the database (for example, '132.192.47.22', or 'localhost' if the database is on the same machine); leave DB_NAME as 'db_webqtl' (unless for some foolish reason you changed the name when you imported the database--don't do that); and DB_USER and DB_PASSWD should be set to the user name and password of an account that can access the db_webqtl database. Also change the IP's under the "Added by Hongqiang Li" section.

• Now modify the Apache configuration file we grabbed earlier. Go to /etc/httpd/conf/, open httpd.conf, go the end of the file, and change the IP used in the virtual server definition to your own IP.

• Finally, we're going to install some of the python modules from earlier. Go to thirdparty/ on your machine (wherever you put it; it doesn't matter). There should be 10 folders there, each representing a Python module. To attempt to install one, go to its directory and execute "python init.py build", followed by "python init.py install". Before you do this, keep in mind that it will return errors for many of them; that a few don't need to be installed this way anyway; and that even if it looks like it was successful, it probably wasn't. Among the ones you don't need to install like this are MySQLdb? and cairo. Htmlgen may be something you can find on yum; I forget. If it gives you any lip, copy it into /usr/lib/python2.4/site-packages/ (or wherever python packages go on your system) and forget about it. The same goes for the rest. Piddle, in particular, will pretend like it's fine with the build/install procedure; do not be fooled. You must copy it as well. And while you're at it, copy direct.so and reaper.so into the same directory.

Final Setup and Debugging

Hopefully everything's ok now. Start up Apache with the command "service httpd start" (or, if for some reason /sbin isn't in your path, cd to /sbin and do "./service httpd start"). For reference, you can perform the similar "service httpd stop" to stop Apache, or "service httpd restart" to, well, restart it. It's good practice to set up Apache to start when the computer does.

If you're on the machine, you can now test stuff by opening a brower to http://localhost. It should display the Genenetwork home page. If the pulldowns look ok, you're probably good. If they don't, they should be replaced with an error thrown by an AJAX menu handler. The bottom of the error will likely say something like "Couldn't find X.so". If X reminds you of something from thirdparty/, copy that folder into your python packages directory and all will be well. If X looks unfamiliar, try yum. Parse the name intelligently to get what you're looking for; i.e. if it can't find "liblapack", search for "lapack"; if it complains about *.g2c, search for "gcc". If yum doesn't work, the Internets may be your only choice. I've had surprisingly good results googling for the package it's missing and finding an appropriate RPM. Not that you should have to do any of that, unless I forgot something.

Tips for 64bit system:

1. Go to gn/support, unzip direct-1.1.0.tar and rebuild it, then copy direct.so to /usr/lib/python2.4/site-packages/

2. Download graphviz source package from http://www.graphviz.org/Download_source.php, rebuild and install, copy /usr/local/bin/neato to usr/local/bin

3. Download and install other 64bit packages: python-numarray

-- HongqiangLi - 20 May 2008