Back to overview
Downtime

RESOLVED: Service disruption April 7

Apr 7, 2026 at 8:53am UTC
Affected services
Velatir API

Resolved
Apr 7, 2026 at 9:28am UTC

Post mortem (April 7 2026, 8:54 - 11:16):

At 8:54 CEST, our infrastructure provider (Scaleway) began experiencing issues with their internal access management service. This caused the database and our Kubernetes cluster to fail on connections.

At 10:46 CEST, Scaleway had restored access.

However, due to our API endpoints returning errors, all browser extensions that are deployed were restarting their requests to our API. This exhausted the database resources, since the endpoint that the browser extensions hit was not cached. Essentially this crashed the backend.

We have bumped the database to allow for a larger amount of connections, tweaked the connection pooling in the API, and added caching to the service discovery endpoint the browser extensions use.

At 11:16 we were back at normal operations.

Created
Apr 7, 2026 at 8:53am UTC

The API has been unavailable due to an incident at our infrastructure provider. We are gradually scaling up operations again at this moment.