pip2pi - Python Package Caching

I ran into an issue recently where I needed to be able to install Python packages on servers without internet access. I researched various options like devpi and other tools that mirror the PyPi servers. However, I didn’t want to download every single PyPi package, for both space and security reasons. Devpi says it has package whitelisting, but I couldn’t get it to work and the documentation was very lacking.

I found pip2pi to be the best solution. I feed a list of packages (or a requirements.txt file) to pip2pi and it downloads all the packages AND their dependencies into a folder. The best part is that the final folder layout matches the PyPi index layout. You can front-end the folder with nginx or Apache and use the -i parameter for pip to specify your server as the index. Now you can point pip at an internal server. Example:

pip install -i http://pythoncache/ requirements.txt

Pip2pi also ensures that every server gets the exact same code when they install packages, especially when not specifying a particular version like package == 0.8.