Ingestion lag has subsided. There are no longer any delays for the subset of customers impacted today.
Posted Oct 29, 2019 - 21:43 PDT
Update
Ingestion lag is still present on the affected database cluster, however it is continuing to improve. Next update by 9:45pm Pacific Time.
Posted Oct 29, 2019 - 20:52 PDT
Monitoring
The impacted database cluster has now recovered, and the data ingestion service has restarted. It will take some time to catch up, so there will be ingestion lag until that point. Next update: 8:45pm Pacific Time.
Posted Oct 29, 2019 - 19:47 PDT
Update
We experienced a bit of a speed bump during the recovery process in the last 45 minutes, but our engineering team has negotiated that issue and we are back on the path to recovery. API error rates are declining. Next update by 7:45pm Pacific Time.
Posted Oct 29, 2019 - 19:00 PDT
Identified
We have identified the cause of the ingestion delay and we are recovering steadily. Next update by 7:00pm Pacific Time.
Posted Oct 29, 2019 - 18:12 PDT
Investigating
A subset of our customers is currently experiencing delays with data ingestion and elevated API error rates. It is impacting user updates and CSV uploads. Our engineering team is working diligently to resolve this issue. Next update by 6:30PM PDT.