Delayed performance for workflow processing

Incident Report for Iterable

Resolved

The fix deployed by our engineering team has resolved the issue and workflow processing has fully recovered. The impact window began for a small subset of customers at approximately 8 AM PST and continued throughout the day until the fix was deployed and we were fully recovered by 2 PM PST. If you have any additional questions please contact your account manager or email support at support@iterable.com A customer facing RCA will be made available upon request.
Posted Feb 10, 2022 - 16:10 PST

Monitoring

Our engineering team deployed a fix and workflow performance continues to improve towards recovery, though customers may still be experiencing some processing delays. We will continue monitoring workflow performance with the next update planned at 4 PM PST or sooner.
Posted Feb 10, 2022 - 14:03 PST

Identified

Our engineering team has identified what is causing the degraded performance in workflow processing. Engineering is currently working on a solution to see if that will fix the degraded performance. Next update at 2 pm PT.
Posted Feb 10, 2022 - 13:02 PST

Investigating

We have received multiple reports of delayed performance in regards to users progressing through workflows. Engineering is currently investigating the cause of these performance delays. Next update at 1 pm PT
Posted Feb 10, 2022 - 11:58 PST
This incident affected: Cluster 5 (Journey Processing), Cluster 9 (Journey Processing), Cluster 11 (Journey Processing), Cluster 12 (Journey Processing), Cluster 13 (Journey Processing), Cluster 14 (Journey Processing), Cluster 15 (Journey Processing), Cluster 18 (Journey Processing), Cluster 19 (Journey Processing), and Cluster 20 (Journey Processing).