On January 15, 2025, at approximately 11:00 UTC, Block Storage clone operations began failing in ORD (Chicago) and IAD (Washington, D.C.) regions due to a missing critical dependency required for the volume cloning feature. The absence of this dependency prevented the successful execution of cloning operations, affecting users attempting to perform clones in the impacted regions.
The first impact to external customers was observed at 11:00 UTC on January 15, 2025. Upon identifying the root cause, we took immediate action to resolve the issue by restoring the missing dependency and updating the Block Storage infrastructure to a newer release. This action restored the functionality of the cloning feature.
The issue was fully resolved by 15:30 UTC on January 16, 2025, after the necessary updates were deployed and the missing dependency was restored. As a result, the Block Storage clone operations returned to normal functionality in the affected regions.
This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.