Yo ! Today, we’ll see how to build a django application with a continuous integration system. With this installation maybe you will loose few times, but you will save time on every deployment of the application, and we know how it can be frustrating to deploy something. So you’ll never have this feeling again because all will be automatized, you’ll just need to do a little click and let jenkins rocks :). Let’s see how to do that !!!
Jenkins is continuous integration server. He will give us the possibility to automatize few task like pull the sources of your project, send an email to the guy who broke the build (my favorite feature), etc… In our case, we want to use Jenkins to deploy automatically our application every time a functionnality is developped on it. It can seem easy, but not really.
For Arch Linux :
sudo pacman -S jenkins sudo systemctl start jenkins
For Ubuntu :
wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add - sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list' sudo apt-get update sudo apt-get install jenkins
Normally, jenkins should run on your machine. His default port is the 8090, so you should be able to go on this URLhttp://localhost:8090/. (Too easy this tutorial)
Virtual Environnement installation
I give you the advice to create a virtual environment for your project. So that will protect you from every python update or just system update because your application will have his own python environment !
To install it we will do :
sudo pacman -S python-virtualenv
And after that we can create an environment easily with this command :
And we will use this command to activate it :
Easy no ?
To launch our django application, we will need of UWSGI.
For that, install uwsgi and uwsgi-python (python module for uwsgi) :
sudo pacman -S uwsgi sudo pacman -S uwsgi-plugin-python
For Debian/Ubuntu :
sudo apt-get install uwsgi sudo apt-get install uwsgi-plugin-python
Don’t forget to adapt that in function of your python version. In my case, I’m using the third version.
However the problem is, our installation is dependant of the system again. So we should install uwsgi directly from the virtual environnement to keep a safe environment. For that we have to use pip or add uwsgi to our requirements.txt file and we will not need the plugin now.
pip install uwsgi
Launching our django application
Now, you have to choice, configure jenkins and wait his first deploy to have the project sources on your server, or you have already a version of it on your server. Here we’ll suppose we have it already. To easily run our application, we will need an .ini file to configure properly UWSGI execution. Here’s an example .ini file for my project mon-amour-pour-jean-travolta
#mon-amour-pour-jean-travolta.ini file [uwsgi] # We specify the module to use (no need if you installed uwsgi with pip) plugins = python # Django-related settings # Project root (where is manage.py) chdir = /path/to/your/application/mon-amour-pour-jean-travolta/ # wsgi file module = mon-amour-pour-jean-travolta.wsgi:application # Absolute path to your virtual environmentChemin absolu vers l'environnement virtuel home = /path/to/your/application/mon-amour-pour-jean-travolta/venv-python3-application/ # process-related settings # master master = true # Number of worker for your application processes = 10 # Unix socket socket = /chemin/vers/lapplication/mon-amour-pour-jean-travolta/juanwolfs-blog.sock # If you want to let open your application on this server without an nginx, uncomment the line below # socket = 127.0.0.1:8888 # Socket permission chmod-socket = 644 # processus user uid = jean-travolta # processus group gid = jean-travolta # socket owner chown_socket = jean-travolta # Create a server for your application (let this value commented the time to debug everything) # daemonize = /var/log/uwsgi/juanwolfs-blog.log # Clean the environment when we kill the application vacuum = true
If everything is well done, your application should run with the command :
uwsgi --ini mon-amour-pour-jean-travolta.ini
Here, the problem is we can’t properly restart automatically the application, we need to kill the process and relaunch it. After few useless researches, I had to ask the advice of an expert. Here’s what he said :
— Dominic Rodger (@dominicrodger) March 17, 2015
Thanks a lot @dominicrodger for the help ! So let’s add the option to our command:
uwsgi --ini mon-amour-pour-jean-travolta.ini --touch-reload mon-amour-pour-jean-travolta.ini # And we use this command to reload it touch mon-amour-pour-jean-travolta.ini
Now, we have the tools we need to deploy our application with only one click. We need to create the job which will execute all the command we should execute if we did the deployment manually. It’s simple, we want: pull the source, launch the virtual environment, download the dependencies, launch the test, change the database if it needs to (I use a django module for that called South), generate translation files, restart the application and leave the virtual environment.
To pull the sources of our application, we will need to host this code somewhere using git or svn or cvs (or dropbox, LOL, just kidding). If you’re using git, you’ll need to install a plugin for it. Just go in Manage Jenkins -> Manage Plugin -> “Availables” -> “GIT plugin”. And if you wanted to install your application to an other server by ssh, install the “Publish Over SSH” plugin.
We will create our job, for that click on “new item”. For this example, I will call my job “jean-travolta-l-integration-qui-envoie-du-pate” (don’t worry if you don’t get a clue of what it means), and select free-style project. Now we have to configure the job to make him doing all the task we say previously. Let’s start with “pulling the sources”.
Pulling the sources
Rendez-vous to the section “gestion du code source”. You have many options (CVS, Subversion, Git (If you installed the plugin)). In my case, I will use git and put this values :
Put some attention on the branch option, you can specify which branch you want to pull and that’s really usefull when you’re working on different branch for different environment. For the last section we will say to jenkins to check if there’s modifications on the git repository every two minutes and if there’s it will launch the build.
Now, we just need to do all the rest. So we will create a little script (bash) which will execute everything for us. Mine looks like that :
#!/bin/bash virtualenv -q venv-python3-blog # Virtual Environment creation if it doesn't exist source ./venv-python3-blog/bin/activate # Activate the virtual environment pip install -r $WORKSPACE/requirements.txt # Install all the dependencies for the application python $WORKSPACE/mon-amour-pour-jean-travolta/manage.py test blogengine # Launching tests python $WORKSPACE/mon-amour-pour-jean-travolta/manage.py schemamigration blogengine --auto --update # Find if there's modification in the database (SOUTH) python $WORKSPACE/mon-amour-pour-jean-travolta/manage.py migrate # apply modification in the database cd $WORKSPACE/mon-amour-pour-jean-travolta/; python manage.py compilemessages # Generate translations files touch $WORKSPACE/mon-amour-pour-jean-travolta.ini # Restart the application deactivate # We leave the virtual environment
Launch the build, and normally we should have something like that :
Et voilà, We have our continuous integration platform for our django application ! However, about tests, we don’t have any reports or good metrics about the quality of our production (except in the console, but that’s not mean anything).
BONUS - Create tests reports
The goal here is to use tools to evaluate the quality of our code. for that,, we will need to modify a little bit our project and our jenkins build. Firstly, install the JUnit, Cobertura and Violation plugin inside Jenkins! For the project, you’ll need to add django-jenkins and coverage as requirements inside your … requirements file. This django module will create the report which will be rendered in jenkins (nice, isn’t it?). So let’s add that to our requirement.txt :
So now, we will modify a little bit our build. Firstly, change in your bash script the line where we were running the tests to use our new depency “django-jenkins” :
#OLD #python $WORKSPACE/mon-amour-pour-jean-travolta/manage.py test blogengine # New python $WORKSPACE/mon-amour-pour-jean-travolta/manage.py jenkins --enable-coverage blogengine
Now, let’s add our report rendering like that :
And if we launch few times the build (JUnit plugin need to be built twice to render), we will have this nice layout :
So I’m able now to say that my 22 tests cover more than 90% of the code of my application. YEAH YEAH YEAH.
Now you have no excuse to not develop your django application now you have nothing to do to put it in production ! You just install a software factory which give quality indications on your application and that’s quite good if you want to build a software with many people so you are able to reject some commit if the quality is not good enough. If you know other metrics / renderer / informations really useful to integrate inside jenkins, let me know I will love you ! Well I let you code ! Codez bien, Ciao !