Back to overview

Errors downloading weights on model startup

Feb 08 at 09:53pm UTC
Affected services
Prediction serving

Feb 08 at 10:00pm UTC

Mitigations are in place and now models are again downloading weights as expected.

All model startups within the affected region have returned to normal.

Feb 08 at 09:53pm UTC

We are seeing elevated incidences of weights failing to download on model startup within one of our regions. A problematic bit of code has been identified and a fix is in progress.

Additionally we are putting in place immediate mitigations to help limit the overall impact while the code fix is being worked on.