Container registries seem like the simplest part of a container platform. They store images. You push to them. You pull from them. What could go wrong?
When you are dealing with multiple terabytes of container images across hundreds of repositories, spanning multiple years of organic growth, the answer is: a lot.
I have led several large-scale container registry migrations into Mirantis Secure Registry (MSR). The smallest was a few hundred gigabytes. The largest was over twenty terabytes. Every migration was different, but the lessons were consistent. Here is what I learned.
Why registry migrations are harder than they look
A container registry is not just storage. It is a governance boundary, a distribution endpoint, and — in many organizations — an audit trail. Teams have built CI/CD pipelines around it. Developers have hardcoded image references. Security scanners are configured to scan it. Backup policies assume it works.
When you migrate, you are not moving bytes. You are migrating every team’s workflow, every pipeline’s assumption, every script that contains the old registry URL.
The first lesson is: inventory everything before you move anything. Do not assume you know what is in the registry. We found images that were five years old, still referenced in dormant pipelines. We found images that were pulled daily but had no documented owner. We found images that were critical to production but had no tags — just dangling digest references.
A complete inventory is the foundation of a migration plan. Without it, you are guessing.
Planning the migration
The planning phase was always longer than the execution phase. For a multi-terabyte migration, I typically spent four to six weeks on planning alone.
The first step was categorizing images. Active and critical. Active and non-critical. Inactive but retained for compliance. Unknown. Each category had a different migration strategy. Critical images were migrated first, with rollback plans. Inactive images were migrated last, sometimes archived instead of migrated.
The second step was understanding the consumption patterns. Which teams pull which images? What is the pull frequency? Are there automated systems that depend on the registry being available at a specific endpoint? We mapped every consumer we could find, and we still missed some.
The third step was defining the target architecture. MSR supports multiple storage backends, garbage collection policies, and retention rules. We designed the target registry to be better than the source — not just a replacement. Better retention policies. Better access controls. Better integration with the existing authentication infrastructure.
The inventory challenge
Inventory was consistently the hardest part. Enterprise container registries grow organically. Teams push images with inconsistent naming conventions. Some images have labels. Most do not. Some repositories are actively maintained. Many are abandoned.
We built scripts to enumerate every repository, every tag, every manifest. We extracted metadata where it existed. We correlated image usage with pipeline logs where we could access them. We interviewed team leads to identify orphaned repositories.
The most useful tool was not technical. It was asking: “Who owns this repository?” If nobody could answer within a week, the repository was treated as inactive.
Governance decisions
Every migration forced governance conversations that should have happened years earlier.
Who is allowed to push to production repositories? Who is allowed to delete images? What is the retention policy for images that are older than twelve months? How do we handle images that contain vulnerable dependencies?
These questions had technical answers — RBAC policies, retention rules, vulnerability scanning configuration — but they were fundamentally organizational decisions. The registry migration became the forcing function for governance that had been deferred.
My advice: do not treat governance as a prerequisite for migration. Treat the migration as an opportunity to establish governance. The old registry was ungoverned. The new one does not have to be.
Cutover strategy
We never did a big-bang cutover for registries larger than a few terabytes. The risk was too high. If something went wrong during the cutover, every team that needed to push or pull images would be blocked.
Instead, we ran the old and new registries in parallel for a transition period. The duration varied from two weeks to two months, depending on the complexity of the environment.
During the transition, we:
- Configured the old registry as a pull-through cache for the new registry
- Redirected CI/CD pipelines to push to the new registry first, with the old registry as fallback
- Monitored pull failures and access patterns daily
- Communicated status updates weekly to every team lead
The parallel run gave us confidence that the new registry was working correctly before we committed to it. It also gave teams time to update their workflows without pressure.
Lessons learned
Several specific lessons stand out from these migrations:
Tag immutability matters. In the source registry, tags could be overwritten. This caused confusion during migration because the same tag pointed to different content at different points in time. We enabled tag immutability in MSR and communicated the change clearly.
Garbage collection is not optional. Unmanaged registries accumulate unreferenced blobs. We ran garbage collection after migration and recovered significant storage — in one case, over 30 percent of total registry size.
Network matters more than you expect. Pushing and pulling terabytes of images across wide-area networks is slow. We staged the data transfer in the same数据中心 where possible, using direct-attached storage for the initial seed.
Final thought
Registry migrations are not glamorous. They are plumbing. But plumbing matters. A well-designed registry migration improves security, governance, and operational efficiency for years. Approach it with humility, invest in planning, and treat it as an organizational change project — not just a technical one.