With thousands of companies using Datadog to track their infrastructure, we can see software trends emerging in real time. Today we're excited to share what we can see about true Docker adoption—no hype, just the facts.
Docker is probably the most talked-about infrastructure technology of 2015. We started this project to investigate how much Docker is used in production, and how fast real adoption is growing. We found the answers to these questions—and more that we discovered along the way—to be fascinating.
Our research was based on a sample of 7,000 companies and tracks real usage, not just anecdotally-reported usage. As far as we know, this is the largest and most accurate review of Docker adoption that has ever been published.
Throughout this article we refer to companies' adoption status: "adopted", "dabbling", or "abandoned". Our method for determining adoption status is described in the Methodology section below.
Real Docker Adoption
Is Up 5x in One Year
At the beginning of September 2014, 1.8 percent of Datadog's customers had adopted Docker. One year later that number has grown to 8.3 percent. That's almost 5x growth in 12 months.
Docker Now Runs on 6%
of the Hosts We Monitor
This is an amazing fact: one year ago Docker had almost no market share, and now it's running on six percent of the hosts we monitor. Six percent may not sound high as an absolute number, but since we monitor 120+ different technologies, it represents a meaningful part of our users' stacks.
However, as seen below, growth as a percentage of hosts seems to have stalled last quarter. This could be seasonal; major infrastructure initiatives are often paused until after summer vacations. We will be interested to see what happens this quarter.
Are the Early Adopters
This one bucks the stereotype that larger companies are slower to move. The more hosts a company uses, the more likely it is to have tried Docker, and the more likely it is to be have adopted Docker. This fact is particularly surprising since the more hosts a company uses, the higher the number of Docker containers the company must use to be considered an “adopter”. As discussed in Methodology below, this finding is resilient to different infrastructure-size segmentation thresholds.
2/3 of Companies
That Try Docker Adopt It
Good news for Docker just keeps coming. We were surprised to find how many companies who try Docker actually adopt it, and fast. Most companies who will adopt have already done so within 30 days of initial production usage, and almost all the remaining adopters convert within 60 days.
Users Triple the Number
of Containers They Use
within 5 Months
Both adopters and dabblers approximately triple the average number of running containers they have in production between their first and sixth month of use. This phenomenal increase in usage—even amongst dabblers—is great news for Docker.
The Most Widely Used Images Are
Registry, NGINX, and Redis
The most common technologies running in Docker are:
- Registry: Fully 25% of companies running Docker are using Registry, presumably instead of Docker Hub.
- NGINX: Docker is being used to contain a lot of HTTP servers, it seems. It is interesting that Apache (httpd) didn’t make the top 10.
- Redis: This popular in-memory key/value data store is often used as an in-memory database, message queue, or cache.
- Ubuntu: Still the default to build images.
- Logspout: For collecting logs from all containers on a host, and routing them to wherever they need to go.
- MongoDB: The widely-used NoSQL datastore.
- Elasticsearch: Full text search.
- CAdvisor: Used by Kubernetes to collect metrics from containers.
- MySQL: The most widely used open source database in the world.
- Postgres: The second-most widely used open source database in the world. Adding the Postgres and MySQL numbers, it appears that using Docker to run relational databases is surprisingly common.
Docker Hosts Often Run
Four Containers at a Time
The median company that adopts Docker runs four containers simultaneously on each host. This finding seems to indicate that Docker is in fact commonly used as a lightweight way to share compute resources; it is not solely valued for providing a knowable, versioned runtime environment.
VMs Live 4x Longer
At companies that adopt Docker, containers have an average lifespan of 3 days, while across all companies, traditional and cloud-based VMs have an average lifespan of 12 days.
As discussed in fact #7, it is common to run 4 containers per host simultaneously. So you might expect that the median VM would run 16 containers total in its lifetime (4 generations of 4 simultaneous containers). But due to uneven distributions, the median VM actually runs 9 containers in its life.
Containers' short lifetimes and increased density have significant implications for infrastructure monitoring. They represent an order-of-magnitude increase in the number of things that need to be individually monitored. Monitoring solutions that are host-centric, rather than role-centric, quickly become unusable. We thus expect Docker to continue to drive the sea change in monitoring practices that the cloud began several years ago.