Ingestion stopped on C25

Incident Report for Iterable

Resolved

Cluster 25 has recovered fully and is now processing normally.
Posted Sep 21, 2023 - 16:13 PDT

Monitoring

Ingestion on cluster 25 is still recovering. We are monitoring the situation and expect it to catch up in the next several hours. Until then you may still experience overall slowness of journey processing and API calls. Next update will be at or before 4pm PDT.
Posted Sep 21, 2023 - 13:03 PDT

Update

As of 9:09 AM PDT we have resumed ingestion on Cluster 25 and are working through the backlog of user events, updates, and triggers. No data has been lost. You may experience overall slowness of journey processing and API calls as we work through this backlog. Next update will be at or before 1PM PDT.
Posted Sep 21, 2023 - 10:43 PDT

Update

Our engineers are continuing to remedy the ingestion issue on C25. Next update will be at 11AM PDT or before.
Posted Sep 21, 2023 - 09:08 PDT

Identified

At 7:48 AM PDT we identified that ingestion has stopped on C25 and our engineers have identified the issue and are working to resolve it. Next update will be at 9AM PDT or sooner.
Posted Sep 21, 2023 - 08:21 PDT