Experiment to better understand Azure Container Registry geo-replication behavior
In this article, we perform a simple experiment to better understand Azure Container Registry (ACR) geo-replication behavior.
Say we are using ACR premium-tier with geo-replication enabled and “docker push” image1:tag1 to the registry from East US 2 region. What happens when we try to “docker pull” image1:tag1 from Southeast Asia region before the replication is finished?
According to the “Considerations for using a geo-replicated registry”, Azure Traffic Manager is used internally by the registry to redirect the request to the registry in the closest region. Therefore, the pull request from a VM in Southeast Asia will go to the registry in that region and will likely fail since it will not find the image there until ACR finishes replicating the new image there in an eventually consistent way.
This is why the ACR documentation recommends to configure regional ACR webhooks to track push events as they complete across the geo-replicated regions.
Simple Experiment
Below, we perform a simple test and confirm that in a geo-replicated ACR after “docker push” completes from a VM in region1 the image is right away ready to be used in that region, but it does not become available in the other regions until a bit later, due to replication.
Our ACR premium-tier is deployed in East US 2 region and replication is enabled to Southeast Asia.
Internally, ACR uses Azure Traffic Manager to resolve its DNS to the closest location.
From VM near East US 2, ACR domain name resolves to r1001eus2.eastus2.cloudapp.azure.com
From VM in Southeast Asia region, the same domain resolves to: r1001sea.southeastasia.cloudapp.azure.com
Now, we push 1.7GB image (tagged v3) from a VM in East US 2 to the registry
From the VM in East US 2, image with tag “v3” becomes visible right after “docker push” completes
On the other hand, from a VM in Southeast Asia, image with tag “v3” is not visible for some 30–60 seconds. This delay time likely depends on the size of the image, which in our experiment was about 1.7GB, and probably on other factors like network conditions and what else is happening in the registry.
Thank you!
Please leave feedback and questions below or on Twitter https://twitter.com/ArsenVlad
