Search the Community

By leveraging the wide array of public images available on Docker Hub, developers can accelerate development workflows, enhance productivity, and, ultimately, ship scalable applications that run like clockwork. When building with public content, acknowledging the potential operational risks associated with using that content without proper authentication is crucial. In this post, we will describe best practices for mitigating these risks and ensuring the security and reliability of your containers. Import public content locally There are several advantages to importing public content locally. Doing so improves the availability and reliability of your public content pipeline and protects you from failed CI builds. By importing your public content, you can easily validate, verify, and deploy images to help run your business more reliably. For more information on this best practice, check out the Open Container Initiative’s guide on Consuming Public Content. Configure Artifact Cache to consume public content Another best practice is to configure Artifact Cache to consume public content. Azure Container Registry’s (ACR) Artifact Cache feature allows you to cache your container artifacts in your own Azure Container Registry, even for private networks. This approach limits the impact of rate limits and dramatically increases pull reliability when combined with geo-replicated ACR, allowing you to pull artifacts from the region closest to your Azure resource. Additionally, ACR offers various security features, such as private networks, firewall configuration, service principals, and more, which can help you secure your container workloads. For complete information on using public content with ACR Artifact Cache, refer to the Artifact Cache technical documentation. Authenticate pulls with public registries We recommend authenticating your pull requests to Docker Hub using subscription credentials. Docker Hub offers developers the ability to authenticate when building with public library content. Authenticated users also have access to pull content directly from private repositories. For more information, visit the Docker subscriptions page. Microsoft Artifact Cache also supports authenticating with other public registries, providing an additional layer of security for your container workloads. Following these best practices when using public content from Docker Hub can help mitigate security and reliability risks in your development and operational cycles. By importing public content locally, configuring Artifact Cache, and setting up preferred authentication methods, you can ensure your container workloads are secure and reliable. Learn more about securing containers Try Docker Scout to assess your images for security risks. Looking to get up and running? Use our Quickstart guide. Have questions? The Docker community is here to help. Subscribe to the Docker Newsletter to stay updated with Docker news and announcements. Additional resources for improving container security for Microsoft and Docker customers Visit Microsoft Learn. Read the introduction to Microsoft’s framework for securing containers. Learn how to manage public content with Azure Container Registry. View the full article

The rise of open source software has led to more collaborative development, but it’s not without challenges. While public container images offer convenience and access to a vast library of prebuilt components, their lack of control and potential vulnerabilities can introduce security and reliability risks into your CI/CD pipeline. This blog post delves into best practices that your teams can implement to mitigate these risks and maintain a secure and reliable software delivery process. By following these guidelines, you can leverage the benefits of open source software while safeguarding your development workflow. 1. Store local copies of public containers To minimize risks and improve security and reliability, consider storing local copies of public container images whenever feasible. The Open Containers Initiative offers guidelines on consuming public content, which you can access for further information. 2. Use authentication when accessing Docker Hub For secure and reliable CI/CD pipelines, authenticating with Docker Hub instead of using anonymous access is recommended. Anonymous access exposes you to security vulnerabilities and increases the risk of hitting rate limits, hindering your pipeline’s performance. The specific authentication method depends on your CI/CD infrastructure and Google Cloud services used. Fortunately, several options are available to ensure secure and efficient interactions with Docker Hub. 3. Use Artifact Registry remote repositories Instead of directly referencing Docker Hub repositories in your build processes, opt for Artifact Registry remote repositories for secure and efficient access. This approach leverages Docker Hub access tokens, minimizing the risk of vulnerabilities and facilitating a seamless workflow. Detailed instructions on configuring this setup can be found in the following Artifact Registry documentation: Configure remote repository authentication to Docker Hub. 4. Use Google Cloud Build to interact with Docker images Google Cloud Build offers robust authentication mechanisms to pull Docker Hub images seamlessly within your build steps. These mechanisms are essential if your container images rely on external dependencies hosted on Docker Hub. By implementing these features, you can ensure secure and reliable access to the necessary resources while streamlining your CI/CD pipeline. Implementing the best practices outlined above offers significant benefits for your CI/CD pipelines. You’ll achieve a stronger security posture and reduced reliability risks, ensuring smooth and efficient software delivery. Additionally, establishing robust authentication controls for your development environments prevents potential roadblocks that could arise later in production. As a result, you can be confident that your processes comply with or surpass corporate security standards, further solidifying your development foundation. Learn more Visit the following product pages to learn more about the features that assist you in implementing these steps. Take control of your supply chain with Artifact Registry remote and virtual repositories Analyze images to prioritize and remediate software supply chain issues with Docker Scout Artifact Registry Product Page Google Cloud Build Product Page View the full article

We are excited to announce the latest feature for Docker Pro and Team users, our new Advanced Image Management Dashboard available on Docker Hub. The new dashboard provides developers with a new level of access to all of the content you have stored in Docker Hub providing you with more fine grained control over removing old content and exploring old versions of pushed images. Historically in Docker Hub we have had visibility into the latest version of a tag that a user has pushed, but what has been very hard to see or even understand is what happened to all of those old things that you pushed. When you push an image to Docker Hub you are pushing a manifest, a list of all of the layers of your image, and the layers themselves. When you are updating an existing tag, only the new layers will be pushed along with the new manifest which references these layers. This new manifest will be given the tag you specify when you push, such as bengotch/simplewhale:latest. But this does mean that all of those old manifests which point at the previous layers that made up your image are removed from Hub. These are still here, there is just no way to easily see them or to manage that content. You can in fact still use and reference these using the digest of the manifest if you know it. You can kind of think of this like your commit history (the old digests) to a particular branch (your tag) of your repo (your image repo!). This means you can have hundreds of old versions of images which your systems can still be pulling by hash rather than by the tag and you may be unaware which old versions are still in use. Along with this the only way until now to remove these old versions was to delete the entire repo and start again! With the release of the image management dashboard we have provided a new GUI with all of this information available to you including whether those currently ‘untagged old manifests’ are still ‘active’ (have been pulled in the last month) or whether they are inactive. This combined with the new bulk delete for these objects and current tags provides you a more powerful tool for batch managing your content in Docker Hub. To get started you will find a new banner on your repos page if you have inactive images: This will tell you how many images you have, tagged or old, which have not been pushed or pulled to in the last month. By clicking view you can go through to the new Advanced Image Management Dashboard to check out all your content, from here you can see what the tags of certain manifests used to be and use the multi-selector option to bulk delete these. For a full product tour check out our overview video of the feature below. We hope that you are excited for the first step of us providing greater insight into your content on Docker Hub, if you want to get started exploring your content then all users can see how many inactive images they have and Pro & Team users can see which tags these used to be associated with, what the hashes of these are and start removing these today. To find out more about becoming a Pro or Team user check out this page. The post Advanced Image Management in Docker Hub appeared first on Docker Blog. View the full article

We are excited to let you know that we have released a new experimental tool. We would love to get your feedback on it. Today we have released an experimental Docker Hub CLI tool, the hub-tool. The new Hub CLI tool lets you explore, inspect and manage your content on Docker Hub as well as work with your teams and manage your account. The new tool is available as of today for Docker Desktop for Mac and Windows users and we will be releasing this for Linux in early 2021. The hub-tool is designed to map as closely to the top level features we know people are using in Docker Hub and provide a new way for people to start interacting with and managing their content. Let’s start by taking a look at the top level options we have. What you can do We can see that we have the ability to jump into your account, your content, your orgs and your personal access tokens. From here I can dive into one of my repos And from here I can then decide to list the tags in one of those repos. This also now lets me see when these images were last pulled Changing focus, I can go over and look at some of the teams I am a member of to see what permissions people have Or I can have a look at my access tokens Why a standalone tool? I also wanted to mention why we have decided to do this as a standalone tool rather than a Docker command with something like docker registry. We know that Docker Hub has some unique features and we wanted to bring these out as part of this tool and get feedback on whether this is something that would be valuable to add (or which bits of this we should add!) to the Docker CLI in the future. Given that some of these are unique to Hub, that we wanted feedback before adding more top level commands into the Docker CLI and that we wanted to do something quick, we decided to go with a stand alone tool. This does mean that this tool is going to be an experiment so we do expect it to go away sometime in 2021. We plan to use the lessons we learn here to make something awesome as part of the Docker CLI. Give us feedback! If you have feedback or want to see this move into the existing Docker CLI, please let us know on the roadmap item. To get started trying out the tool, sign up for a Hub account and start using the tool in the Edge version of Docker Desktop. The post Docker Hub Experimental CLI tool appeared first on Docker Blog. View the full article

Last week, we announced that the Docker Desktop Stable release includes vulnerability scanning, the latest milestone in our container security solution that we are building with our partner Snyk. You can now run Snyk vulnerability scans directly from the Docker Desktop CLI. Combining this functionality with Docker Hub scanning functionality that we launched in October provides you with the flexibility of including vulnerability scanning along multiple points of your development inner loop, and provides better tooling for deploying secure applications. You can decide if you want to run your first scans from the Desktop CLI side, or from the Hub. Customers that have used Docker for a while tend to prefer starting from the Hub. The easiest way to jump in is to configure the Docker Hub repos to automatically trigger scanning every time that you push an image into that repo. This option is configurable for each repository, so that you can decide how to onboard these scans into your security program. (Docker Hub image is available only for Docker Pro and Team subscribers, for more information about subscriptions visit the Docker Pricing Page.) Once you enable scanning, you can view the scanning results either in the Docker Hub, or directly from the Docker Desktop app as described in this blog. From the scan results summary you can drill down to first view the more detailed data for each scan and get more detailed information about each vulnerability type. The most useful information in vulnerability data is the Snyk recommendation on how to remediate the detected vulnerability, and if a higher package version is available where the specific vulnerability has already been addressed. Detect, Then Remediate If you are viewing vulnerability data from the Docker Desktop, you can start remediating vulnerabilities, and testing remediations directly from your CLI. Triggering scans from Docker Desktop is simple – just run docker scan, and you can run iterative tests that confirm successful remediation before pushing the image back into the Hub. For new Docker users, consider running your first scans from the Desktop CLI. Docker Desktop Vulnerability Scanning CLI Cheat Sheet is a fantastic resource for getting started. The CLI Cheat Sheet starts from the basics, which are also described in the Docker Documentation page on Vulnerability scanning for Docker local images – including steps for running your first scans, description of the vulnerability information included with each scan result, and docker scan flags that help you specify the scan results that you want to view. Some of these docker scan flags are – --dependency-tree - displaying the list of all the package underlying dependencies that include the reported vulnerability --exclude base - running an image scan, without reporting vulnerabilities associated with the base layer --f - including the vulnerability data for the associated Dockerfile --json - displaying the vulnerability data in JSON format The really cool thing about this Cheat Sheet is that it shows you how to combine these flags to create a number of options for viewing your data – Show only high severity vulnerabilities from layers other than the base image: $ docker scan myapp:mytag --exclude-base \ --file path/to/Dockerfile --json | \ jq '[.vulnerabilities[] | select(.severity=="high")]' Show high severity vulnerabilities with an CVSSv3 network attack vector: $ docker scan myapp:mytag --json | \ jq '[.vulnerabilities[] | select(.severity=="high") | select(.CVSSv3 | contains("AV:N"))]' Show high severity vulnerabilities with a fix available: $ docker scan myapp:mytag --json | \ jq '[.vulnerabilities[] | select(.nearestFixedInVersion) | select(.severity=="high")]' Running the CLI scans and remediating vulnerabilities before you push your images to the Hub, reduces the number of vulnerabilities reported in the Hub scan, providing your team with a faster and more streamlined build cycle To learn more about running vulnerability scanning on Docker images, you can watch “Securing Containers Directly from Docker Desktop” session, presented during SnykCon. This is a joint presentation by Justin Cormack, Docker security lead, and Danielle Inbar, Snyk product manager, discussing what you can do to leverage this new solution in the security programs of your organization The post Combining Snyk Scans in Docker Desktop and Docker Hub to Deploy Secure Containers appeared first on Docker Blog. View the full article

The gradual enforcement of the Docker Hub progressive rate limiting enforcement on container image pulls for anonymous and free users began Monday, November 2nd. The next three hour enforcement window on Wednesday, November 4th from 9am to 12 noon Pacific time. During this window, the eventual final limit of 100 container pull requests per six hours for unauthenticated users and 200 for free users with Docker IDs will be enforced. After that window, the limit will rise to 2,500 container pull requests per six hours. As we implement this policy, we are looking at the core technologies, platforms and tools used in app pipelines to ensure a transition that supports developers across their entire development lifecycle. We have been working with leading cloud platforms, CI/CD providers and other ISVs to ensure their customers and end users who use Docker have uninterrupted access to Docker Hub images. Among these partners are the major cloud hosting providers, CI/CD vendors such as CircleCI, and OSS entities such as Apache Software Foundation (ASF). You can find more information about programs on our Pricing Page as well as links to contact us for information about programs for ISVs and companies with more than 500 users. Besides the Apache Software Foundation, we are working with many Open Source Software projects from Cloud Foundry and Jenkins to many other open source projects of all sizes, so they can freely use Docker in their project development and distribution. Updates and details on the program are available in this blog from Docker’s Marina Kvitnitsky. We have assembled a page of information updates, as well as relevant resources to understand and manage the transition, at https://www.docker.com/increase-rate-limits. We’ve had a big week delivering new features and integrations for developers this week. Along with the changes outlined above, we also announced new vulnerability scan results incorporated into Docker Desktop, a faster, integrated path into production from Desktop into Microsoft Azure, and improved support for Docker Pro and Team subscribers. We are singularly focused on creating a sustainable, innovative company focused on the success of developers and development teams, both today and tomorrow, and we welcome your feedback. The post Updates on Hub Rate Limits, Partners and Customer Exemptions appeared first on Docker Blog. View the full article

On August 13th, we announced the implementation of rate limiting for Docker container pulls for some users. Beginning November 2, Docker will begin phasing in limits of Docker container pull requests for anonymous and free authenticated users. The limits will be gradually reduced over a number of weeks until the final levels (where anonymous users are limited to 100 container pulls per six hours and free users limited to 200 container pulls per six hours) are reached. All paid Docker accounts (Pro, Team or Legacy subscribers) are exempt from rate limiting. The rationale behind the phased implementation periods is to allow our anonymous and free tier users and integrators to see the places where anonymous CI/CD processes are pulling container images. This will allow Docker users to address the limitations in one of two ways: upgrade to an unlimited Docker Pro or Docker Team subscription, or adjust application pipelines to accommodate the container image request limits. After a lot of thought and discussion, we’ve decided on this gradual, phased increase over the upcoming weeks instead of an abrupt implementation of the policy. An up-do-date status update on rate limitations is available at https://www.docker.com/increase-rate-limits. Docker users can get an up-to-date view of their usage limits and updated status messages in the CLI, in terms of querying for current pulls used as well as header messages returned from Docker Hub. This blog post walks developers through how they can access their current account usage as well as understanding the header messages. And finally, Docker users can avoid rate limits completely by upgrading to a Pro or Team subscription: subscription details and upgrade information is available at https://docker.com/pricing. And open source projects can apply for a sponsored no-cost Docker account by filling out this application. The post What you need to know about upcoming Docker Hub rate limiting appeared first on Docker Blog. View the full article

Continuing with our move towards consumption-based limits, customers will see the new rate limits for Docker pulls of container images at each tier of Docker subscriptions starting from November 2, 2020. Anonymous free users will be limited to 100 pulls per six hours, and authenticated free users will be limited to 200 pulls per six hours. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction as long as the quantities are not excessive or abusive. In this article, we’ll take a look at determining where you currently fall within the rate limiting policy using some command line tools. Determining your current rate limit Requests to Docker Hub now include rate limit information in the response headers for requests that count towards the limit. These are named as follows: RateLimit-Limit RateLimit-Remaining The RateLimit-Limit header contains the total number of pulls that can be performed within a six hour window. The RateLimit-Remaining header contains the number of pulls remaining for the six hour rolling window. Let’s take a look at these headers using the terminal. But before we can make a request to Docker Hub, we need to obtain a bearer token. We will then use this bearer token when we make requests to a specific image using curl. Anonymous Requests Let’s first take a look at finding our limit for anonymous requests. The following command makes a request to auth.docker.io for an authentication token for the ratelimitpreview/test image and saves that token in an environment variable named TOKEN. You’ll notice that we do not pass a username and password as we will for authenticated requests. $ TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) Now that we have a TOKEN, we can decode it and take a look at what’s inside. We’ll use the jwt tool to do this. You can also paste your TOKEN into the online tool located on jwt.io $ jwt decode $TOKEN Token header ------------ { "typ": "JWT", "alg": "RS256" } Token claims ------------ { "access": [ { "actions": [ "pull" ], "name": "ratelimitpreview/test", "parameters": { "pull_limit": "100", "pull_limit_interval": "21600" }, "type": "repository" } ], ... } Under the Token header section, you see a pull_limit and a pull_limit_interval. These values are relative to you as an anonymous user and the image being requested. In the above example, we can see that the pull_limit is set to 100 and the pull_limit_interval is set to 21600 which is the number of seconds for the limit. Now make a request for the test image, ratelimitpreview/test, passing the TOKEN from above. NOTE: The following curl command emulates a real pull and therefore will count as a request. Please run this command with caution. $ curl -v -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest 2>&1 | grep RateLimit < RateLimit-Limit: 100;w=21600 < RateLimit-Remaining: 96;w=21600 The output shows that our RateLimit-Limit is set to 100 pulls every six hours – as we saw in the output of the JWT. We can also see that the RateLimit-Remaining value tells us that we now have 96 remaining pulls for the six hour rolling window. If you were to perform this same curl command multiple times, you would observe the RateLimit-Remaining value decrease. Authenticated requests For authenticated requests, we need to update our token to be one that is authenticated. Make sure you replace username:password with your Docker ID and password in the command below. $ TOKEN=$(curl --user 'username:password' "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) Below is the decoded token we just retrieved. $ jwt decode $TOKEN Token header ------------ { "typ": "JWT", "alg": "RS256" } Token claims ------------ { "access": [ { "actions": [ "pull" ], "name": "ratelimitpreview/test", "parameters": { "pull_limit": "200", "pull_limit_interval": "21600" }, "type": "repository" } ], ... } The authenticated JWT contains the same fields as the anonymous JWT but now the pull_limit value is set to 200 which is the limit for authenticated free users. Let’s make a request for the ratelimitpreview/test image using our authenticated token. NOTE: The following curl command emulates a real pull and therefore will count as a request. Please run this command with caution. $ curl -v -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest 2>&1 | grep RateLimit < RateLimit-Limit: 200;w=21600 < RateLimit-Remaining: 176;w=21600 You can see that our RateLimit-Limit value has risen to 200 per six hours and our remaining pulls are at 176 for the next six hours. Just like with an anonymous request, If you were to perform this same curl command multiple times, you would observe the RateLimit-Remaining value decrease. Error messages When you have reached your Docker pull rate limit, the resulting response will have a http status code of 429 and include the below message. HTTP/1.1 429 Too Many Requests Cache-Control: no-cache Connection: close Content-Type: application/json Retry-After: 21600 { "errors": [{ "code": "DENIED", "message": "You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" }] } Conclusion In this article we took a look at determining the number of image pulls allowed based on whether we are an authenticated user or anonymous user. Anonymous free users will be limited to 100 pulls per six hours, and authenticated free users will be limited to 200 pulls per six hours. If you would like to avoid rate limits completely, you can purchase or upgrade to a Pro or Team subscription: subscription details and upgrade information is available at https://docker.com/pricing. For more information and common questions, please read our docs page and FAQ. And as always, please feel free to reach out to us on Twitter (@docker) or to me directly (@pmckee). To get started using Docker, sign up for a free Docker account and take a look at our getting started guide. The post Checking Your Current Docker Pull Rate Limits and Status appeared first on Docker Blog. View the full article