Cellar performance issues in Roubaix and Gravelines due to OSD crash

The recovery process for the Object Storage Daemon (OSD) in our Cellar offering is now underway in the Roubaix and Gravelines regions. While the affected daemon is back in operation, we are continuing to stabilize the environment and restore full performance. Our engineers remain on-site to complete the recovery phase. We will share another status update as soon as the process has advanced (eta 60 mins).

The recovery process for the Object Storage Daemon (OSD) in Cellar is progressing in our Roubaix and Gravelines regions. While the affected daemon has been recovered and is now operational, our engineers continue to stabilize the environment to restore normal service performance. Additional updates will be provided once the recovery phase has advanced (eta 60 mins).

The recovery process for the Object Storage Daemon (OSD) in Cellar continues in our Roubaix and Gravelines regions. While the affected daemon remains recovered and functional, our engineering team is still stabilising the infrastructure to restore full performance. We will provide another update once the process advances.

The recovery of the Object Storage Daemon (OSD) in Cellar is still in progress for our Roubaix and Gravelines regions. While the daemon itself is operational, our engineers are working on recovering full data redundancy across affected storage units. This process is expected to take a few more hours, and some placement group instances remain impacted during the final recovery steps. We will provide another update once the redundancy restoration progresses further.

The redundancy recovery process across the affected Cellar units in Roubaix and Gravelines has now completed, and rebalancing operations are actively in progress. During this phase, customers may continue to experience normal operations with ongoing background tasks as the platform stabilizes. Our engineering team is closely monitoring the rebalancing process to ensure full performance is restored. We will provide another status update once rebalancing completes.

The rebalancing phase of the Ceph cluster in the Roubaix and Gravelines regions is now fully in effect as the recovery from the OSD crash completes. During this process, customers may continue to observe background operations such as data redistribution across storage nodes. The engineering team reports the estimated time to finish the full rebalancing remains approximately six days. We are closely watching system stability and performance throughout this period. Further developments will be communicated as needed.

The ongoing rebalancing phase for Cellar in the Roubaix and Gravelines regions is still in progress under the hood. All Object Storage Daemon (OSD) components have been stabilized, and data redundancy has been fully restored across storage units. Cellar services are operating at normal levels with no performance issues reported. The status of this issue is being updated to Resolved. No customer impact is expected going forward.