At the end of my three-week vacation, I was asked for a seemingly easy task. We were going to release the new version of the project we developed:

• Is today is the last working of the week? No.
• Is there any problem with the unit tests? No.
• Are all changes confirmed by the product manager, in the staging server? Yes.

I always note everything, and so I write this article to remind this experience. My other habit is to define dosh sub-commands to access frequently used commands in my whole projects. It doesn’t matter if the project is written in .NET Core or used Django;

• I know that the project will start in my development environment with dosh start,
• I know that the project will build with dosh build,
• I know that dosh deploy command will be running if I push my commits to some specific git branches. All these things are common sub-commands in my projects.

So what was not a standard?

I manage everything in a project except the server maintenance. Today the site was offline about an hour because of the communication failure. Sometimes there are small but critical upgrades on the servers, and one of them broke SSL module of the project’s psycopg2 dependency. I had only these logs:

DAMN ! worker 1 (pid: 28018) died :( trying respawn ...
Respawned uWSGI worker 1 (new pid: 28021)
!!! uWSGI process 27988 got Segmentation Fault !!!
*** backtrace of 27988 ***
/usr/bin/uwsgi-core(uwsgi_backtrace+0x2a) [0x55a0ad5c498a]
/usr/bin/uwsgi-core(uwsgi_segfault+0x23) [0x55a0ad5c4d73]
/lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7f4811ecaf20]
/project/venv/lib/python3.6/site-packages/psycopg2/.libs/./libssl-8bb9b3dd.so.1.0.2o(ssl3_cleanup_key_block+0xb) [0x7f480a747cbb]
/project/venv/lib/python3.6/site-packages/psycopg2/.libs/./libssl-8bb9b3dd.so.1.0.2o(ssl3_clear+0x16) [0x7f480a7455b6]
/project/venv/lib/python3.6/site-packages/psycopg2/.libs/./libssl-8bb9b3dd.so.1.0.2o(tls1_clear+0x9) [0x7f480a751229]
/usr/lib/x86_64-linux-gnu/libssl.so.1.1(SSL_new+0x3f2) [0x7f4812b429d2]
/project/venv/lib/python3.6/site-packages/psycopg2/.libs/libpq-239f9ac8.so.5.10(+0x23399) [0x7f480a9dc399]

You may know Segmentation Fault if you coded in C or if you worked with a package manager. Some included libraries that are coming with psycopg2 binary package are linked to the old version of libssl, so the last server update broke the dependencies¹.

lib/python3.6/site-packages/psycopg2/.libs/libssl-8bb9b3dd.so.1.0.2o   # from 1.0
lib/python3.6/site-packages/psycopg2/.libs/libssl-7a80b951.so.1.1.1b   # to 1.1

In some projects, it’s not possible to solve the problems as we wanted because of some regulations and source limits. I could find prevention for that case, but somehow, I forgot to check the updates in the source code and server packages and tried to make the deployment. And the worst thing, I didn’t realise the problem until someone warned me.

So, I added one more rule for myself to make a deployment:

• Is today the first day after the vacation? Yes (so, don’t touch it.).

---

1. “Don’t break the dependencies”, I missed contributing a Linux distro like Pardus. ↩︎