Incident History

Since 14:46 UTC, requests to applications hosted in the Montréal region fail on our shared load balancers. Intermittent errors started at 14:11 UTC.

The root cause has been identified and a fix is being applied.

Since the PHP update, deployments may be slow to complete or return HTTP 503 errors on the affected regions. We have identified the cause and are working to restore normal performance. Already-running applications and services are not impacted. We will provide an update within the next 30 minutes.

We have identified a DDoS attack targeting one of our shared load balancer instances in the Paris region. To mitigate its effects, we have temporarily taken the affected instance out of DNS while we investigate and implement countermeasures. Customers may experience degraded load balancing performance during this incident. We will share further updates once mitigation is in effect.

We are currently observing a high error rate on our Pulsar cluster hosted in the Paris region. Customers relying on Pulsar for message processing or streaming may encounter issues or timeouts. Our engineering team is actively investigating the root cause. We will provide an update within the next 30 minutes.

We have identified the root cause of the issues affecting Pulsar clients when producing or consuming messages. Applications using Pulsar may currently encounter failures or delays during message operations. The team is now applying a fix to restore normal service. We will provide an update within the next 30 minutes.

Public load balancers in SNC Gouv are experiencing intermittent issues since 07:24 UTC, resulting in possible failed requests or TLS handshake issues.

We are investigating the root cause.

We are currently investigating a DDoS attack targeting our infrastructure in the Paris (PAR) region. This is causing degraded performance and connectivity issues for applications hosted in Paris. Our team is actively working to mitigate the impact and restore normal service.

We are currently observing timeouts and elevated 503 errors affecting the Cellar S3-compatible object storage service in our Paris (PAR) region. Customers using Cellar for file storage, backups, or static assets may experience errors or degraded performance. Our engineering team is actively investigating the root cause. We will provide an update within the next 30 minutes.

Our systems have detected increased errors and service disruptions following a recent load balancer update in our GOV and SCW regions. The issue stems from reload operations that exceeded normal duration during deployment.

Customers may encounter connection errors or degraded service availability in affected regions. We are actively watching the situation to assess full impact.

We will provide an update within the next 30 minutes.

We are currently experiencing an infrastructure issue in our Paris (PAR) region, resulting in degraded service availability. Our team is actively working to restore full service. We will provide an update within the next 30 minutes.

We are currently recovering from metrics lag. As a result, you may experience delays when viewing your metrics. We are monitoring the process closely.

We identified the issue to be a network link failure between datacenters that we've quickly identified & disabled.

We are currently observing 503 Service Unavailable errors affecting the Cellar object storage service in our Paris (PAR) region. Customers using Cellar may encounter issues accessing their stored objects. Our team is actively investigating the root cause. We will provide an update within the next 30 minutes.

Some deployments are currently failing to initiate after a commit is pushed to GitHub repositories. We are investigating the issue. Already-running applications and services are not impacted.

We have identified performance degradation affecting Cellar in our Roubaix and Gravelines regions. This issue originated from an OSD (Object Storage Daemon) crash that occurred yesterday. Although the affected daemon has been recovered, instability during the process could lead to further disruptions. Our team is actively monitoring the recovery phase and working to restore normal performance. We will provide an update within the next 30 minutes.

Some deployments were currently failing on the affected regions. We investigated the issue and rolled back the component update that caused it.

It was always possible to start a successful deployment from the console or the cli. Not from a push on github for the duration of the incident.

The monitoring has detected that an hypervisor is unreachable on Paris since 10 minutes. We are investigating. Add-ons and services hosted on this node are currently unreachable.

One hypervisor in the Paris region experienced a brief issue between 12:20:09 UTC and 12:21:41 UTC. During this time, services hosted on it experienced both inbound and outbound network connectivity issues. Users might have encountered errors such as "TTL exceeded" or "Host is unreachable." The issue is now resolved.

We are currently investigating issues affecting our load balancers. This may impact the availability and performance of applications hosted on our platform. Our team is working to identify the root cause. We will provide an update within the next 30 minutes.

We are currently investigating delays in sending email and webhook notifications. Some users may experience a delay in receiving these notifications. Our team is actively working to identify the root cause. We will provide an update within the next 30 minutes.

We are currently experiencing issues with deployment statuses. Applications that are already running are unaffected. Our team is investigating the problem. We will provide an update within the next 30 minutes.