tag:warpbuild.instatus.com,2005:/history WarpBuild Status - Incident history 2026-05-18T06:03:49.494+00:00 WarpBuild tag:warpbuild.instatus.com,2005:Incident/cmpastefm08gmqjjg9gh14z40 2026-05-18T06:03:49.494+00:00 2026-05-18T06:03:49.494+00:00 WarpBuild status page migrated to a new home: https://status.warpbuild.com/ Type: Incident

May 18, 06:03:49 GMT+0
Resolved - WarpBuild status page migrated to a new home: <https://status.warpbuild.com/> Please resubscribe for email/slack updates..

]]> tag:warpbuild.instatus.com,2005:Maintenance/cmowce0mx03bysn35yultzgol 2026-05-09T05:30:00.000+00:00 2026-05-09T05:30:00.000+00:00 Scheduled maintenance for infrastructure upgrades Type: Maintenance

Duration: 1 hour

Affected Components: Dashboard, API, GitHub Runners, Docker Builders

May 9, 05:30:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time. There may be \~5m delays in scheduling during this time..

May 9, 05:30:01 GMT+0
Identified - Maintenance is now in progress.

May 9, 06:30:00 GMT+0
Completed - Maintenance has completed successfully.

]]> tag:warpbuild.instatus.com,2005:Incident/cmmdcx5bp0024snpg8m68ivx2 2026-03-05T10:45:00.000+00:00 2026-03-05T10:45:00.000+00:00 Linux runners on x86-64 arch scheduling is delayed Type: Incident

Duration: 19 minutes

Affected Components: GitHub Runners

Mar 5, 10:45:00 GMT+0
Identified - We are seeing delays in job pick up times of up to 10 mins..

Mar 5, 11:03:31 GMT+0
Resolved - This incident has been resolved. The queues are now back to normal..

]]> tag:warpbuild.instatus.com,2005:Incident/cmlku6i9y00pa13evljv1rbb8 2026-02-13T12:00:54.324+00:00 2026-02-13T12:00:54.324+00:00 MacOS runners are seeing delays in scheduling Type: Incident

Duration: 1 hour and 12 minutes

Affected Components: Dashboard, API, GitHub Runners, Docker Builders

Feb 13, 12:00:54 GMT+0
Identified - We have identified the problem and are continuing to work on a fix for this incident..

Feb 13, 12:33:00 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Feb 13, 13:13:20 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmkyo0m7800yuppzbcal5wztt 2026-01-28T23:37:25.887+00:00 2026-01-28T23:37:25.887+00:00 Degradation in upstream GitHub services causing jobs to not get picked up / timeout Type: Incident

Duration: 22 minutes

Affected Components: Dashboard, API, GitHub Runners, Docker Builders

Jan 28, 23:37:25 GMT+0
Investigating - We are currently investigating this incident..

Jan 28, 23:59:40 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmj1mrucu00xcbf5g13fw92h6 2025-12-11T16:06:00.000+00:00 2025-12-11T16:40:01.817+00:00 Upstream GitHub issue - API and actions degraded Type: Incident

Duration: 34 minutes

Affected Components: Third Party: Github → Actions, Third Party: Github → API Requests

Dec 11, 16:40:01 GMT+0
Resolved - The GitHub incident doesn't seem to have any real impact..

Dec 11, 16:06:00 GMT+0
Identified - API Requests and Actions are experiencing degraded performance. <https://www.githubstatus.com/incidents/xntfc1fz5rfb>.

]]> tag:warpbuild.instatus.com,2005:Incident/cmj1lilnm00o7ws65rkaamur1 2025-12-11T14:00:39.267+00:00 2025-12-11T14:00:39.267+00:00 Runners are not allocated or are failing to start Type: Incident

Duration: 1 hour and 19 minutes

Affected Components: Dashboard, API, GitHub Runners, Docker Builders

Dec 11, 14:00:39 GMT+0
Investigating - Runners were either not starting or failing to start with a 503 after allocation..

Dec 11, 14:30:39 GMT+0
Identified - We are working on a fix for this.

Dec 11, 14:50:00 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result. Runners are working as usual now..

Dec 11, 15:20:00 GMT+0
Resolved - This incident has been resolved. A full post-incident report will be published shortly along with mitigation steps..

Dec 11, 15:21:00 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmio8w7f1015udkq5tdrw2dc0 2025-12-02T07:16:59.406+00:00 2025-12-02T07:16:59.406+00:00 WarpBuild Runners are not starting with a "failed to fetch" error Type: Incident

Duration: 7 minutes

Affected Components: GitHub Runners

Dec 2, 07:16:59 GMT+0
Investigating - We are currently investigating this incident..

Dec 2, 07:23:47 GMT+0
Resolved - This incident has been resolved..

Dec 2, 07:48:34 GMT+0
Postmortem - The issue was caused because of a faulty change to the `auth` service during a refactor. This was not expected to have an impact on the runner allocation flow. This was reverted to fix the error. We are also going to add tests to ensure issues of this nature do not creep through..

]]> tag:warpbuild.instatus.com,2005:Incident/cmi6av0rk008x104t5u0pvkzd 2025-11-19T17:52:12.087+00:00 2025-11-19T17:52:12.087+00:00 Github reporting degradation for Actions Type: Incident

Duration: 17 minutes

Affected Components: Third Party: Github → Actions, GitHub Runners

Nov 19, 17:52:12 GMT+0
Identified - GitHub is investigating an actions related outage - [https://www.githubstatus.com/incidents/zs5ccnvqv64m.](https://www.githubstatus.com/incidents/zs5ccnvqv64m) This is leading to queued jobs not getting picked up..

Nov 19, 18:02:35 GMT+0
Monitoring - GitHub applied a mitigation. We are seeing recovery and monitoring the situation..

Nov 19, 18:09:34 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmi52uico004zy41owmyv1h2g 2025-11-18T20:41:00.000+00:00 2025-11-18T20:41:00.000+00:00 GitHub reporting failures with Git operations Type: Incident

Duration: 1 hour and 19 minutes

Affected Components: Third Party: Github → Git Operations, Third Party: Github → Webhooks, Third Party: Github → Actions, Third Party: Github → API Requests

Nov 18, 20:41:00 GMT+0
Identified - There is an increase in job failures because of an incident with Github <https://www.githubstatus.com/incidents/5q7nmlxz30sk>. We are monitoring the situation..

Nov 18, 22:00:17 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmi4qxbc7007wcest7dltfl3y 2025-11-18T11:40:00.000+00:00 2025-11-18T11:40:00.000+00:00 WarpBuild services are down because of cloudflare global outage Type: Incident

Duration: 2 hours and 35 minutes

Affected Components: API, Dashboard, , Third Party: Cloudflare → Cloudflare Sites and Services → R2, Third Party: Cloudflare → Cloudflare Sites and Services → API, GitHub Runners, Docker Builders, Cloudflare →

Nov 18, 11:40:00 GMT+0
Investigating - This issue is being added retroactively because the status page was also down. <https://www.cloudflarestatus.com/incidents/8gmgl950y3h7>.

Nov 18, 13:30:00 GMT+0
Resolved - WarpBuild is seeing full recovery. .

]]> tag:warpbuild.instatus.com,2005:Incident/cmgzgiz8303is55kx8hbmfs2w 2025-10-20T09:00:00.000+00:00 2025-10-20T09:00:00.000+00:00 Remote docker builders and arm64 runners are not functional because of the AWS outage Type: Incident

Duration: 13 hours

Affected Components: GitHub Runners, Docker Builders

Oct 20, 09:00:00 GMT+0
Identified - Remote docker builders and arm64 runners are not functional because of the AWS outage There may be some delay in jobs getting picked up because of queueing and image related delays. Windows, custom runners, and snapshot runners are also affected..

Oct 20, 18:19:25 GMT+0
Identified - Windows runners are also impacted because of the AWS outage..

Oct 20, 21:54:50 GMT+0
Monitoring - We are seeing full recovery in the impacted services: * Docker builders * arm64 runners * custom runners * Snapshot runners * Windows runners x64 runners were not impacted and are continuing to work as usual..

Oct 20, 22:00:00 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmfm0getv0262v7mp2xtdu0kf 2025-09-16T03:46:05.532+00:00 2025-09-16T03:46:05.532+00:00 [No customer impact] Investigating a potential issue impacting some runners Type: Incident

Duration: 19 minutes

Affected Components: API, GitHub Runners

Sep 16, 03:46:05 GMT+0
Investigating - Tailscale has an ongoing incident that may affect some runner types in WarpBuild. <https://status.tailscale.com/incidents/01K589GKH7AGJ2VBY131HM4ZD1>.

Sep 16, 03:52:48 GMT+0
Monitoring - There was no user impact because of the tailscale outage and systems are working as expected. Continuing to monitor the situation..

Sep 16, 04:04:44 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmdfvodwx00u4j4w4klvnuk9w 2025-07-14T10:00:00.000+00:00 2025-07-14T10:00:00.000+00:00 Intermittent issues with network speeds Type: Incident

Duration: 9 days and 51 minutes

Affected Components: GitHub Runners

Jul 14, 10:00:00 GMT+0
Identified - We received sporadic notifications of low network throughput..

Jul 23, 10:51:21 GMT+0
Resolved - A small set of servers had network performance issues. These have been replaced and the network related issues are resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cmcosnqbg000sjzwd8561g28h 2025-07-04T12:15:00.000+00:00 2025-07-04T12:32:15.596+00:00 Service interruption for github runners and docker builders Type: Incident

Duration: 17 minutes

Affected Components: API, Dashboard, GitHub Runners, Docker Builders

Jul 4, 12:32:15 GMT+0
Resolved - This incident has been resolved..

Jul 4, 12:15:00 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

]]> tag:warpbuild.instatus.com,2005:Maintenance/cmc94vr4y0006hbf5ehozu8d3 2025-06-23T14:00:00.000+00:00 2025-06-24T09:37:22.337+00:00 Usage alerts and limits will not work for 24 hours Type: Maintenance

Duration: 19 hours and 37 minutes

Affected Components: GitHub Runners, Docker Builders

Jun 24, 09:37:22 GMT+0
Completed - Maintenance has completed successfully..

Jun 23, 14:00:01 GMT+0
Identified - Maintenance is now in progress.

Jun 23, 14:00:00 GMT+0
Identified - We are planning for a scheduled maintenance during that time..

]]> tag:warpbuild.instatus.com,2005:Incident/cmb0mi3o5001t2pmefcx3q5xo 2025-05-23T08:52:00.000+00:00 2025-05-23T08:52:00.000+00:00 Docker builders using arm64 seeing timeouts Type: Incident

Duration: 1 hour and 4 minutes

Affected Components: Docker Builders

May 23, 08:52:00 GMT+0
Identified - This was caused by a misconfiguration during testing. We have implemented the fix and are monitoring the situation..

May 23, 09:56:18 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/cm8q76kt1005z74oe5sh1kkxg 2025-03-26T17:27:33.013+00:00 2025-03-26T17:27:33.013+00:00 32x instances are seeing delays in startup for some users Type: Incident

Duration: 44 minutes

Affected Components: GitHub Runners

Mar 26, 17:27:33 GMT+0
Monitoring - We implemented a fix and are currently monitoring the result..

Mar 26, 18:11:58 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Maintenance/cm6aby513005lyb8tszyza2qf 2025-01-25T05:00:00.000+00:00 2025-01-25T05:00:01.000+00:00 Scheduled maintenance Type: Maintenance

Duration: 35 minutes

Affected Components: GitHub Runners

Jan 25, 05:00:01 GMT+0
Identified - Maintenance is now in progress.

Jan 25, 05:35:13 GMT+0
Completed - Maintenance has completed successfully..

Jan 25, 05:00:00 GMT+0
Identified - There may be some delays to processing runners during the maintenance window..

]]> tag:warpbuild.instatus.com,2005:Incident/cm5ue3qz700hs1276stvjsc3m 2025-01-12T23:45:00.000+00:00 2025-01-13T03:29:51.695+00:00 Runners are not starting Type: Incident

Duration: 3 hours and 45 minutes

Affected Components: GitHub Runners

Jan 13, 03:29:51 GMT+0
Resolved - This incident has been resolved..

Jan 12, 23:45:00 GMT+0
Investigating - We are currently investigating this incident..

]]> tag:warpbuild.instatus.com,2005:Incident/cm4bs3zol001xafcumum8848v 2024-12-03T10:30:00.000+00:00 2024-12-03T10:30:00.000+00:00 Runner misconfiguration causing some jobs and snapshots to fail Type: Incident

Duration: 19 days

Affected Components: GitHub Runners

Dec 3, 10:30:00 GMT+0
Identified - This incident was because configs for setting up the `runner` user were not applied during a routine runner binary upgrade. This issue affected \~5% of our users. The resolution required a rebuild and rollout of the affected images. This incident has been resolved..

Dec 4, 06:30:00 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/clyitbu6m214706j3o7y6p80yxs 2024-07-12T14:30:00.000+00:00 2024-07-12T14:30:00.000+00:00 WarpBuild jobs are seeing delays in start Type: Incident

Duration: 1 hour and 13 minutes

Affected Components: GitHub Runners

Jul 12, 14:30:00 GMT+0
Investigating - We are currently investigating this incident..

Jul 12, 14:50:49 GMT+0
Identified - We are continuing to work on a fix for this incident..

Jul 12, 15:12:04 GMT+0
Identified - AWS EC2 API has an ongoing incident. This is causing instance capacity to run out..

Jul 12, 15:17:43 GMT+0
Monitoring - We're starting to see recovery in the job delays. The backlog should be processed shortly..

Jul 12, 15:42:35 GMT+0
Resolved - AWS had an issue which was causing capacity issues for WarpBuild instances. <https://health.aws.amazon.com/health/status#multipleservices-us-east-1%5F1720795596> This has now been resolved and we have seen full recovery of all WarpBuild services..

]]> tag:warpbuild.instatus.com,2005:Incident/cly8yd02y132695iclr0wh8ahfi 2024-07-05T16:50:00.000+00:00 2024-07-05T18:19:00.000+00:00 Github is not delivering webhooks Type: Incident

Duration: 1 hour and 29 minutes

Jul 5, 18:19:00 GMT+0
Resolved - This incident has been resolved..

Jul 5, 16:50:00 GMT+0
Identified - We are continuing to monitor Github for a fix - <https://www.githubstatus.com/>.

]]> tag:warpbuild.instatus.com,2005:Incident/clwge6aem7441bsopkqxgl3gd 2024-05-21T12:49:06.981+00:00 2024-05-21T12:49:06.981+00:00 Jobs stuck in queued state Type: Incident

Duration: 6 hours and 17 minutes

Affected Components: Dashboard, API, GitHub Runners

May 21, 12:49:06 GMT+0
Monitoring - GitHub is currently facing an issue with their actions API, where the status of newly created jobs is stuck in the queued state..

May 21, 19:05:45 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/clw9wmd73129841aoopr0ep47oe 2024-05-16T23:51:05.884+00:00 2024-05-16T23:51:05.884+00:00 Runner delays in getting picked up Type: Incident

Duration: 34 minutes

Affected Components: GitHub Runners

May 16, 23:51:05 GMT+0
Investigating - We are seeing a spike in runner pick up delays. Currently investigating this incident..

May 17, 00:19:18 GMT+0
Monitoring - The issue is because of a GCP outage. We are seeing recovery now..

May 17, 00:25:15 GMT+0
Resolved - This incident has been resolved. All pending jobs have been processed. Rootcause: Major service disruption to GCP in the America regions across VPC, GKE, and other services..

]]> tag:warpbuild.instatus.com,2005:Incident/clw5ke8mu37255bmo6ye97z6ey 2024-05-13T22:57:44.633+00:00 2024-05-13T22:57:44.633+00:00 Runners experiencing delays Type: Incident

Duration: 40 minutes

Affected Components: GitHub Runners

May 13, 22:57:44 GMT+0
Investigating - There was a Github actions downtime which led to some of WarpBuild runners seeing delays. We are working on a fix..

May 13, 23:37:23 GMT+0
Resolved - This incident has been resolved..

]]> tag:warpbuild.instatus.com,2005:Incident/clvjlri9i0721cgo8ekixjxqc 2024-04-26T12:00:00.000+00:00 2024-04-26T15:00:00.000+00:00 Intermittent networking slowdown issue with arm64 and amd64 runners Type: Incident

Duration: 3 hours

Affected Components: GitHub Runners

Apr 26, 15:00:00 GMT+0
Resolved - This incident has been resolved..

Apr 26, 12:00:00 GMT+0
Identified - The network interruption was because of a faulty configuration of network MTU pushed to the runner instances. This caused some workloads to drop packets or hang..

]]> tag:warpbuild.instatus.com,2005:Incident/clv6nsvmn239259ben4o5z54p54 2024-04-19T12:41:13.744+00:00 2024-04-19T12:41:13.744+00:00 ARM64 runners are timing out Type: Incident

Duration: 2 hours and 59 minutes

Affected Components: GitHub Runners

Apr 19, 12:41:13 GMT+0
Identified - We are continuing to work on a fix for this incident..

Apr 19, 15:40:21 GMT+0
Resolved - This incident has been resolved..

Apr 19, 13:03:43 GMT+0
Monitoring - Recovery is in progress..

]]>