What happened

1. At 9:50PM PST, all of our worker nodes lost blockchain connectivity.
2. At 9:54PM PST, we were notified of downtime and immediately responded.
3. At 10:03PM PST, we changed RPC configurations to bring Automatos back online. Incident resolved.

Why this happened

1. All of the RPC providers we use with our Automatos workers went offline.

Estimated costs

1. Automatos was down on Blast for about 13 minutes.
2. Automatos did not miss any work that needed to be performed.

How to prevent this in the future

1. We'll add more backup RPC providers.