Issues scheduling to certain hardware
Resolved
Aug 19 at 11:41pm UTC
Thank you for your patience. At this time all hung workloads targeted for the T4 hardware should no longer be stuck in starting phase.
Affected services
API
Updated
Aug 19 at 09:39pm UTC
We have identified the cause of the delays scheduling workloads to the T4 hardware. We have performed a workaround and are seeing workloads successfully complete at this time.
We will update further once pending workloads have cleared.
Affected services
API
Created
Aug 19 at 08:20pm UTC
We are aware and investigating an issue impacting the ability to schedule predictions and trainings to certain hardware. Workloads affected appear to stay in the "starting" phase for extended periods of time.
This is primarily impacting the use of the T4 GPUs. Other hardware types should be unaffected.
This is not directly impacting responses when interacting with the website or API.
Affected services
API