High capacity utilization
Resolved
Feb 19 at 08:07am UTC
We are back at full capacity. Thanks for your patience!
Affected services
API
Prediction serving
Updated
Feb 19 at 06:04am UTC
We identified an unexpected scaling configuration applied to a large number of deployments that seems to account for all of the resources exhaustion. We rolled out a change to our autoscaling to ignore this particular scaling configuration and now are seeing recovery.
Affected services
API
Prediction serving
Updated
Feb 19 at 04:50am UTC
We are continuing the investigation into the source of the incorrect increased demand for hardware. At this time we expect all instances to take significant time to boot.
Affected services
API
Prediction serving
Created
Feb 19 at 04:07am UTC
All hardware types are seeing unexpectedly high demand. We are investigating the root cause of this excessive demand.
Affected services
API
Prediction serving