Back to overview
Degraded

Issues scheduling to certain hardware

Aug 19 at 08:20pm UTC
Affected services
API

Resolved
Aug 19 at 11:41pm UTC

Thank you for your patience. At this time all hung workloads targeted for the T4 hardware should no longer be stuck in starting phase.

Updated
Aug 19 at 09:39pm UTC

We have identified the cause of the delays scheduling workloads to the T4 hardware. We have performed a workaround and are seeing workloads successfully complete at this time.

We will update further once pending workloads have cleared.

Created
Aug 19 at 08:20pm UTC

We are aware and investigating an issue impacting the ability to schedule predictions and trainings to certain hardware. Workloads affected appear to stay in the "starting" phase for extended periods of time.

This is primarily impacting the use of the T4 GPUs. Other hardware types should be unaffected.

This is not directly impacting responses when interacting with the website or API.