Incident on November 10th, 2026

What happened

On November 10th, 9:25 UTC, the Impero platform was unavailable for 11 minutes, due to a server error. Users attempting to access the platform got a HTTP 502 - Bad Gateway response. No data, controls or other assets were lost, and operation resumed normally after the incident.

Why did it happen

A combination of a server upgrade in October and the implementation of a new web framework meant that the web servers could potentially run out of RAM, needing a restart. The driver of the RAM issue could be upload of large files to the platform. The Impero-platform operates on 2 separate web servers, and if one is down, the other will take over the load, to ensure the platform is online. On this day, the usage hit both servers, meaning that both were unavailable for a short period. The issue was resolved at 9:36 UTC.

The issue had already been identified ahead of the incident, and a fix was ready, but it hadn't been deployed.

How did we respond

We responded immediately by rebooting one of the web servers, bringing the platform back online. Once the platform was online, we investigated the issue, to confirm that the cause was the known RAM issue. We immediately applied a configuration change that resolved the issue, while still planning to deploy the long-term-fix code change on next deploy. I addition, we set up additional alerts in our monitoring systems for RAM consumption.