Scaling Docker Usage with JFrog

Earlier last month the development industry was preparing for rate limit changes at Docker Hub. Ultimately, any rate limit changes were put on hold. Many JFrog customers have asked us, “How would Docker Hub rate limit changes impact us?” In this post we’ll discuss what you can do to ensure uninterrupted usage of Docker, now and into the future, regardless of rate limits.

What role does Docker play in modern software delivery?

Containerization has become a standard practice in software development. As long as the base image is consistent with the underlying OS, containerization allows developers to deploy their application anywhere — and in a way that updates and scaling can be performed efficiently.

For many Docker is synonymous with containers. That’s why Docker, Docker Hub, and the container ecosystem they’ve built are considered a cornerstone to deliver modern cloud-native software.

In addition to Docker laying the groundwork for modern containers, Docker Hub hosts millions of images in both public and private registries. Development teams use Docker Hub to retrieve Docker Official images and those contributed by Verified Publishers with both developers and CI tools regularly download images.

And, of course, in addition to the free and open source tools Docker provides, they offer a number of paid services to aid in container development.

Why use JFrog as your container registry?

The JFrog Platform, with JFrog Artifactory at its core, serves as the universal artifact manager and container registry for thousands of development organizations. Teams rely on JFrog as their private container registry for a number of essential benefits, including:

• Ensuring availability of critical dependencies by proxying requests to upstream public registries, and then maintaining a cache of those artifacts.
• Better visibility and control over the SDLC by managing containers, helm charts, and all of the packages/files that are part of your applications and their deployment in one place.
• Providing full traceability of applications from the development process through to a container image running in production.
• Advanced container management based on image layers, which optimizes consumption for faster transfers and scans while providing detailed visibility into the application layer.
• Contextualized container security, providing layer specific guidance on where vulnerabilities reside in containers and whether they’re applicable.
• The ability to deploy JFrog anywhere, including hybrid cloud/on-prem configurations to speed up development and delivery of your applications.

What challenges do Docker Hub rate limits pose for DevOps teams?

Best practice recommends that you leverage an authenticated connection to Docker Hub for your development workflows. However, if you’re like many development organizations, chances are some portion of your pipelines are pulling from Docker Hub as an unauthenticated user.

Even if you are using a CI solution with a dependency proxy, that proxy may also be pulling from Docker Hub as an unauthenticated user. And while that proxy may help reduce some of the calls to Docker Hub, if your developers like to pull latest then your frequently running pipelines are likely to quickly run up against the rate limits. Those frequent calls to Docker Hub over the public internet also introduce delays into the performance of your pipelines.

How does JFrog help DevOps teams scale their usage of Docker?

We’ve long understood the important role Docker plays in software delivery and have even partnered with Docker to help organizations get the most out of the technology. JFrog helps protect you against uncertainty due to rate limits, allowing you to confidently scale in three primary ways:

1. Proxy Caching Docker Hub limits pull requests
2. Enabling authentication removes or augments pull limits
3. Unlimited public Docker Hub registry downloads are available via JFrog SaaS

Proxy Caching Docker Hub

One of the most common use cases development teams leverage JFrog for is to proxy public registries. JFrog caches requested packages close to where development happens and serves up those assets to developers and CI systems wherever, whenever needed. By using JFrog as your pull through cache you limit the number of unique calls to Docker Hub, further optimizing IT resources and improving pipeline performance. It’s perhaps no surprise that Docker is one of the most widely used repository types by JFrog customers.

Enabling Authentication

JFrog makes it easy to authenticate your access to Docker Hub via the JFrog Platform. By authenticating, you can use either a free or paid account to adjust or bypass pull rate limits altogether. Also, by authenticating, you gain greater flexibility, security, and control in how you interact with Docker Hub. Authenticated requests provide better accountability and allow pulling images from private repositories. We highly recommend that you authenticate your JFrog Platform with Docker Hub if you have not done so already.

Unlimited public Docker Hub registry downloads

To ensure continuity of development pipelines, JFrog has partnered with Docker to allow JFrog Cloud subscribers to avoid Docker Hub’s top limit (pulls per hour) for downloads from the public Docker Hub registry for unauthenticated users as long as those pull requests are coming via our managed JFrog Platform (JFrog SaaS). This enables pipelines to continue without disruptions, but is not recommended as a long term strategy.

Those self-hosting the JFrog Platform would potentially face more disruption from a Docker Hub rate limit change. We recommend that those organizations start to take steps to review where their pipelines are pulling from Docker Hub and ensure they are properly authenticated and leveraging JFrog as their cache proxy. Here are few other best practices both self-hosted and SaaS JFrog customers can implement to help reduce the amount of traffic to Docker Hub:

• Ensure every environment is pulling dependencies via JFrog – development, CI, production
• Educate developers on the benefits of pulling specific versions of images rather than using the latest tag as that can cause additional requests to Docker Hub.
• Standardize on a single base operating system image to improve cache performance and reduce the number of image layers that must be pulled from Docker Hu