We have identified a hardware failure and have isolated the affected node(s). We are seeing a return to normal service for H100-targeted predictions and trainings.

Predictions and trainings targeting h100 hardware are currently failing to create. Our engineers are working on identifying the source of these failures and will provide updates as information becomes available.

This incident impacts all h100-class hardware targets.