WarpBuild - WarpBuild jobs are seeing delays in start – Incident details

All systems operational

WarpBuild jobs are seeing delays in start

Resolved
Major outage
Started about 2 years agoLasted about 1 hour

Affected

GitHub Runners

Operational from 2:30 PM to 2:50 PM, Major outage from 2:50 PM to 3:17 PM, Degraded performance from 3:17 PM to 3:42 PM

Updates
  • Resolved
    UTC
    Resolved

    AWS had an issue which was causing capacity issues for WarpBuild instances.

    https://health.aws.amazon.com/health/status#multipleservices-us-east-1_1720795596

    This has now been resolved and we have seen full recovery of all WarpBuild services.

  • Monitoring
    UTC
    Monitoring

    We're starting to see recovery in the job delays. The backlog should be processed shortly.

  • Update
    UTC
    Update

    AWS EC2 API has an ongoing incident. This is causing instance capacity to run out.

  • Identified
    UTC
    Identified
    We are continuing to work on a fix for this incident.
  • Investigating
    UTC
    Investigating
    We are currently investigating this incident.