How do I static compile python?

E.g. you're working in an "enterprise-y" environment with servers running Centos 5.xxx.

You'd like to compile a newer version of python into a nfs share that can be mounted on all servers and not have any lissing libraries, you dont need to set the LD_LIBRARY_PATH.

Or do something like what ActiveState does with their ActivePython distro.

Any ideas people?

So far, I've found 2 links that could potentially help:- StaticPython StackOverflow

StaticPython seems nice, but I'd like to do something like what ActiveState does, so I could install other libraries myself.

11 comments · 67% upvoted · other discussions

comments

lost-theory

Use virtualenv. The same virtualenv can be moved around to different machines if they have the same architecture.

shavenwarthog

Seconded. Using pip and virtualenv your app can use a "private" Python, with specific versions of packages that don't interfere with the system-level libraries.

The terminology can be confusing. Here's my post that minimally goes over them: http://johntellsall.blogspot.com/2011/06/proper-way-to-install-pip-virtualenv.html

lost-theory

Also, I don't think StaticPython you linked to will work for what you need:

distutils is not included, there is no site-packages directory, loading .so files is not supported

if dependencies are or need shared libraries (.so files) -- StaticPython doesn't support loading .so files, not even with the dl or ctypes modules

mindrunner [OP]

Thanks. But assuming I want a newer version of Python, I'd have to compile it into a separate path, e.g by using the ./configure --prefix=/opt/something

That would result in me having to set the LD_LIBRARY_PATH right? My main aim is to not touch the rpm database (hence no EPEL packages) or add stuff to the /etc/ld.conf.

Or am I understanding this wrong?

shavenwarthog

My understanding is that pip/virtualenv will also recompile a private Python binary for you. If that doesn't work, try "buildout", which is also awesome.

rfkelly

No, pip/virtualenv don't recompile Python, they just symlink to an existing install. This is usually what you want.

I have an on-again-off-again project called "myppy" that compiles a private python into a virtualenv-like setup:

https://github.com/rfk/myppy/

It tries to make things portable across different flavours of linux, by removing certain options and using the autopackage compiler tools to avoid dependencies on newer glibc features. Might do what the OP wants slightly better than virtualenv.

mindrunner [OP]

This looks interesting. I'll give it a try :)

rfkelly

Be warned, it's pretty rough round the edges and underdocumented. That said, it should do what it says on the tin. Feel free to shoot me any questions or bugs you run into.

lost-theory

I've been able to install some C extensions nicely without LD_LIBRARY_PATH (lxml, mysqldb), but others (libyaml) I've had to set LD_LIBRARY_PATH manually before running the interpreter.

If you want to avoid using RPM, ldconfig, the system-wide version of Python, etc. then I think you pretty much have to do something like this.

What C extensions are you going to install? For the bigger ones (numpy, lxml, etc.) there should be a pretty well accepted way to install to a virtualenv with a locally built version of python.

fullouterjoin

I do this

ROOT=/nfs/nsistore01/hadoop/ci04/opt
export PATH=$ROOT/python266/bin:$PATH
export LD_LIBRARY_PATH=$ROOT/python266/lib:$LD_LIBRARY_PATH
export MANPATH=$ROOT/python266/share/man:$MANPATH

and then source ( . ) this in my .bashrc

You don't need a static python. This python is writable by me, and I can pip install anything I want.

mindrunner [OP]

this is a pretty easy way of doing it. All the other blogs seems to do it in variations of this way.