Beginning on February 19, 2025, at 23:00 UTC, we observed 403 errors when customers tried to access E2 and E3 endpoint buckets with "Public Read" permissions in Object Storage. The issue was traced to a recent software update that changed the default behavior for "Public Read" access, now requiring explicit bucket policies. Previously, legacy access allowed “Public Read” access without these policies. Customers transitioning from older to newer and enhanced Object Storage infrastructure versions were not expecting this change, causing the errors. We rolled back the enhanced version at 12:42 UTC on February 20, 2025, and confirmed resolution by 13:16 UTC.
To prevent future issues, we are enhancing communication around default behavior changes, improving testing for public access scenarios, and updating documentation to clarify the need for bucket policies in the enhanced version compared to the legacy version.
This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.