6 World-Class Companies using Python
There are over 500 current programming languages, with more being written every day. Admittedly, the majority of these overlap and a large number were never meant to be used outside of a theoretical or lab setting. But for the programming languages that are used in everyday coding and businesses, you have to make a choice. What languages should you learn, and why should you invest your time in learning them?
We’ve already told you why Python is a great language to learn. While it’s easy to see how you can tinker with Python, you might be wondering how this translates to actual business and real world applications.
What we’re going to do now is tell you about six top-tier companies that you know that use Python. That way you can see what great real world opportunities there are for Python developers out there.
Google has been a supporter of Python from nearly the very beginning. In the beginning, the founders of Google made the decision of “Python where we can, C++ where we must.” This meant that C++ was used where memory control was imperative and low latency was desired. In the other facets, Python enabled for ease of maintenance and relatively fast delivery.
Even when other scripts were written for Google in Perl or Bash, these were often recoded into Python. The reason was because of the ease of deployment and how simple Python is to maintain. In fact, according to Steven Levy – author of “In the Plex,” Google’s very first web-crawling spider was first written in Java 1.0 and was so difficult that they rewrote it into Python.
Python is now one of the official Google server-side languages—C++, Java, and Go are the other three—that are allowed to be deployed to production. And in case you aren’t really sure about how important Python is to Google, Python’s very own BDFL, Guido van Rossum, worked at Google from 2005 to 2012.
To top it all off, Peter Norvig said:
“Python has been an important part of Google since the beginning, and remains so as the system grows and evolves. Today dozens of Google engineers use Python, and we’re looking for more people with skills in this language.”
Facebook production engineers are exceptionally keen on Python, making it the third most popular language at the social media giant (just behind C++ and their proprietary PHP dialect, Hack). On average, there are over 5,000 commits to utilities and services at Facebook, managing infrastructure, binary distribution, hardware imaging, and operational automation.
The ease of using Python libraries means that the production engineers don’t have to write or maintain as much code, allowing them to focus on getting improvements live. It also ensures that the infrastructure of Facebook is able to scale efficiently.
According to a 2016 post by Facebook, Python is currently responsible for multiple services in infrastructure management. These include using TORconfig to handle network switch setup and imaging, FBOSS for whitebox switch CLIs, and using Dapper for scheduling and execution of maintenance work.
Facebook has published numerous open-source Python projects written for Py3 including a Facebook Ads API and a Python Async IRCbot framework.
In 2016, the Instagram engineering team boasted that they were running the world’s largest deployment of the Django web framework, which is written entirely in Python.
This likely still holds true today. Min Ni, a software engineer at Instagram says this about their production-use of Python:
“We initially chose to use Python because of its reputation for simplicity and practicality, which aligns well with our philosophy of ‘do the simple thing first.’“
Since then, Instagram’s engineering team has invested time and resources into keeping their Python deployment viable at the massive scale (~800 million monthly active users) they’re operating at:
“With the work we’ve put into building the efficiency framework for Instagram’s web service, we are confident that we will keep scaling our service infrastructure using Python. We’ve also started to invest more into the Python language itself, and are beginning to explore moving our Python from version 2 to 3.”
In 2017, Instagram migrated most of their Python code base from Python 2.7 to Python 3. You can watch the PyCon 2017 keynote talk that Lisa Guo and Hui Ding gave and hear about their experience with this massive code migration.
This music streaming giant is a huge proponent of Python, using the language primarily for data analysis and back end services. On the back end, there are a large number of services that all communicate over 0MQ, or ZeroMQ, an open source networking library and framework that is written in Python and C++(among other languages).
The reason that the services are written in Python is because Spotify likes how fast the development pipeline is when writing and coding in Python. The most recent updates to Spotify’s architecture have all been using gevent, which provides a fast event loop with a high-level synchronous API.
To provide suggestions and recommendations for users, Spotify relies on a large volume of analytics. To interpret these, Spotify makes use of Luigi, a Python module that synchs with Hadoop. This open source module handles how the libraries work together, and consolidates error logs quickly to allow troubleshooting and redeployment.In total, Spotify uses over 6000 individual Python processes that work together over the nodes of the Hadoop cluster.
Netflix uses Python in a very similar manner to Spotify, relying on the language to power its data analysis on the server side. It doesn’t just stop there, however. Netflix allows their software engineers to choose what language to code in, and have noticed a large upsurge in the number of Python applications.
When surveyed, Netflix engineers cite the standard library, the extremely active development community, and the rich variety of third party libraries available to solve nearly any given problem. Additionally, because Python is so easy to develop, it has become a linchpin in many of Netflix’s other services.
One of the primary places that Python is used is in the Central Alert Gateway. This RESTful web app processes alerts from anywhere, and then route them to people or groups that would need to see them. Additionally, the app has the power to suppress duplicate alerts that have already been handled and in some cases, perform automated solutions such as rebooting a process or terminating something that is starting to look shaky. This app is a huge win for Netflix considering the sheer volume of alerts. Handling them intelligently means that developers and engineers aren’t flooded with redundant calls.
Another area that Python is used at Netflix is with monkey apps used to track security changes and history. These monkeys are used to track and alert any changes in EC2 security related policies in any groups, tracking any changes in these environments. They are also used to ensure to track the dozens of SSL certificates attached to Netflix’s multiple domains. In tracking, Netflix has seen a reduction in unexpected expirations drop from one very quarter to none since 2012.
This cloud based storage system uses Python in its desktop client. If you have any doubt at all about how invested Dropbox is in Python, consider that in 2012, they managed to convince Guido van Rossum, Python’s creator and benevolent dictator for life, away from Google and into the Dropbox fold.
Rossum joined Dropbox under the condition that he would be an engineer, not a lead or even a manager. In his first year, he was able to help bring about the ability to share datastores with other users within the Dropbox community.
While a great many of Dropbox’s libraries and internals are proprietary and not open source, the company has released a very efficient API coded in python that allows you to see how their engineers are thinking. You can also read between the lines when you read interviews with Dropbox engineers about how a huge percentage of their server side code is Python.
It’s also interesting to note that while the client side programs are coded in Python, they make use of various libraries on both Mac and Windows machines to allow a unified experience. This is because Python doesn’t come preinstalled on Windows and depending on your Mac, your Python version will vary.
In this post we looked at 6 world-class and successful software companies that use Python in production. But they’re not the only ones. As of 2018 Python adoption has reached a new peak and continues to climb.