Logs service unavailable
The logs stack is currently unavailable. We are looking into it. Logs viewer from the console, clever-tools (CLI) and the API are impacted.
Updates
The issue has been found. We are working on fixing it. The incident also impacts the following services and has been escalated to Major Outage:
- Pulsar product
- Access logs
- Logs Drains
If you wish to view the logs of your applications, you can use the CLI to SSH to the instance
- https://www.clever.cloud/developers/doc/administrate/ssh-clever-tools/#using-clever-tools-cli
- https://www.clever.cloud/developers/doc/administrate/ssh-clever-tools/#show-your-applications-logs
The team is still working on fixing the issue.
The team is currently restarting the underlying services before allowing clients access again.
We are starting to allow clients connect to the pulsar cluster gradually.
We had to shut down the pulsar proxies again because of the underlying issue with the pulsar proxy happening again. We are restarting the various components and will retry allowing clients connects to the cluster.
We started allowing clients reconnecting to the pulsar cluster at 12:24 UTC. We are monitoring the situation and looking into restarting the various services used by the pulsar cluster (logs, access logs, drains).
The logs services (logs, drains) are available again. We are restarting the access logs services. The underlying pulsar cluster is also healthy.
The logs stack is operational. Some drains were disabled because of the incident, we are in the process of re-enabling them. The access logs stack is currently being restarted.
The access logs queue is currently being catch up. We will keep monitoring the platform for the next few hours. If you have issues with any of the impacted services, please reach out to our support with the impacted service.
All disabled drains due to this incident have been enabled again.