Previous incidents

Predictions are flowing as expected once again. We'll continue to monitor the situation.

The rollback fixed things and we're back to normal.

We've identified what's causing this issue and have rolled back the change. Affected predictions will have completed successfully and you can re-request their status through the API.

Autoscaling issues have been resolved for all models and everything should be operating normally.

Prediction webhooks are now working as intended.

We made a change intended to fix a subtle bug in webhook handling, and unfortunately introduced a much less subtle bug: "completed" webhooks were dropped and not correctly delivered. We rolled that change back and are now working on a permanent fix for both bugs.

The database issue was automatically resolved. We made some changes to our database schema in the wrong order, and this resulted in a brief interruption of service for the replicate.com website. Running predictions was not affected.