Separating Development and Production Dependencies
00:00 Let’s discuss separating development and production dependencies for your programs. A common challenge with Python dependency management in the real world is that your development and continuous integration environments need additional dependencies that the production environment doesn’t need.
00:19 For example, your development environment might need additional testing frameworks, debuggers or profiling tools that are not really necessary in production, and might actually slow things down.
00:30 The same would be true for you continuous integration environment. If you’re running any kind of automated tests, you probably want to include testing frameworks and all kinds of other supporting tools for that on your build server environment.
00:44 But again, with all of this, the goal is for the production environment to still run lean and mean. Now the good news is that there is actually a requirements files workflow that is well known and often used in the Python community that solves this problem.
There is a feature in pip that allows you to link up requirements files so you can make this workflow very convenient for developers. They only need to remember to install the
requirements-dev.txt file and that will automatically pull in and install all of the production dependencies as well.
01:36 Let’s take a look at a real world example for this technique. So I am looking at Love here, which is a Python web application that was created by the fine folks at Yelp, and you are going to see in a minute now that they are using the exact same pattern for managing their development in production dependencies.
So right now, I am in the root folder of this repository here and when I scroll down, I’ll find those two requirements files, so there is a
requirements.txt and then there is also a
And this is exactly the split that I talked about earlier, so when you look at
requirements.txt you can see that this includes all of the production dependencies in their exact versions, and nicely enough, they actually put a little comment here to explain what is going on, I think this is a really great use of the commenting feature that you can use in requirements files, by the way.
So there is also this
requirements-dev.txt and what this does, it first of all pulls in the production requirements and then it installs a bunch of additional development dependencies, so it pulls in a testing framework and the
mock module, and some other modules that are useful during development but not really on a production machine.
03:08 And what’s interesting here is that for some of the dependencies they actually pin them to specific version, so every time you specify an exact version that is called pinning, and this is our version pinning and this is exactly what they do here.
But then for this other dependency called
ipdb which is an enhanced version of the Python debugger, they do not pin the dependency. And this makes a lot of sense because the
ipdb debugger would never actually run on an automated test server, but it’s more something that a developer would run on their local machine.
03:39 So it might make sense to leave that dependency unpinned so it will always upgrade to the latest version. Now, again you can see here that they put this little header explaining what is going on in this file, and I think this is really helpful.
Now these two requirements files cover the two usecases we discussed earlier. So if a developer wants to set this up on a new machine, they would clone the repository and then run
pip install -r requirements-dev.txt and this brings in all of the development dependencies.
If you’re building a web application this workflow is specifically supported by platforms like Heroku. So what Heroku does when you deploy a web application to it, is it will look for a
requirements.txt file in the root of your source repository and try to install those dependencies, so if you follow the
requirements-dev.txt split, it will only install the production dependencies and not the testing dependencies which is exactly what you want.
Become a Member to join the conversation.