The issues with inference was limited to select LLM models. At this time the problematic code has been rolled back and all inference should be operating normally at this time.

We are aware of a series of errors occurring when running predictions on the replicate platform. We have started a rollback of the affected code.