Iterable Status

All Systems Operational

Web Application Operational

API Operational

Email Sending Operational

Email Links Operational

Workflows Operational

Push Sending Operational

SMS Sending Operational

System Webhooks Operational

Analytics Processing Operational

List Upload Operational

User Updates Operational

Operational

Degraded Performance

Partial Outage

Major Outage

Maintenance

System Metrics Month Week Day

Web App Latency

Fetching

Links API Success Rate

Fetching

Links Latency

Fetching

Message Sending API Success Rate

Fetching

Message Sending API Latency

Fetching

User Update and Track API Latency

Fetching

Past Incidents

Jul 21, 2019

No incidents reported today.

Jul 20, 2019

No incidents reported.

Jul 19, 2019

No incidents reported.

Jul 18, 2019

Data ingestion delay and elevated api 5xx errors

Resolved - We have resolved the ingestion lag and all systems have now fully-caught up at this point. Thank you for your patience. If you have any concerns please contact us at support@iterable.com.
Jul 18, 18:01 PDT

Identified - Our team continues to work on a resolution. Ingestion lag as decreased and is improving both for custom events and api updates. Api error rates have decreased as well. Next Update will be at 6 pm PDT
Jul 18, 16:14 PDT

Investigating - Currently a subset of customers are experience delays in data ingestion for custom events and user api updates. We are also investigating elevated api error rates. Next update will be at 4 pm PDT
Jul 18, 14:55 PDT

Data Ingestion Lag

Resolved - We have been fully caught up for some time now and are going to close this incident out. If you have any concerns please contact us at support@iterable.com.
Jul 18, 04:43 PDT

Update - Data ingestion is now fully caught up and we are going to monitor the queues for a little while longer before closing this incident out. Next update will be before 6 AM Pacific time.
Jul 18, 04:08 PDT

Monitoring - We have implemented changes that will solve the root cause of the issue. We are still experiencing data ingestion lag and are monitoring all queues until we see them return back to a normal state. The next update will be before 4 AM Pacific time.
Jul 18, 03:04 PDT

Identified - One of our server clusters is currently experiencing data ingestion lag of up to an hour. The engineering team has identified what caused the issue and is currently working on a fix. They are also working on reducing the backlog of data ingestion. The next update will be before 3 AM Pacific Time.
Jul 18, 02:00 PDT

Jul 17, 2019

Data ingestion lag for one server cluster

Resolved - Data ingestion is now fully-caught up at this point. Thank you for your patience. If you have any concerns please contact us at support@iterable.com
Jul 17, 10:46 PDT

Investigating - One of our server clusters is currently experiencing data ingestion lag of up to 30 minutes. This is impacting approximately 40% of customers. The engineering team is working on addressing the backlog right now. Impacted customers will see ingestion of API data and CSV uploads delayed. Next update by 10:40am Pacific Time.
Jul 17, 09:39 PDT

Jul 16, 2019

No incidents reported.

Jul 15, 2019

No incidents reported.

Jul 14, 2019

No incidents reported.

Jul 13, 2019

No incidents reported.

Jul 12, 2019

No incidents reported.

Jul 11, 2019

No incidents reported.

Jul 10, 2019

No incidents reported.

Jul 9, 2019

Workflow delays for some customers

Resolved - This incident has been resolved.
Jul 9, 16:03 PDT

Update - We have not encountered any further issues with workflow delays, so we are now closing this incident. Thank you for your patience if you were adversely affected, and if there are any questions or concerns, please open a ticket by emailing support@iterable.com
Jul 9, 10:14 PDT

Monitoring - Workflow queues have now cleared and we are back to operating as normal. We will continue to monitor this through the night. Further actions will be taken tomorrow to continue addressing the root cause of this. Next update by 8:30am Pacific Time, unless more information is identified prior.
Jul 8, 19:37 PDT

Update - Workflow queues for impacted customers continue to improve, reducing by about 35% in the last hour. We are continuing to work on making adjustments to improve throughput. Next update by 7:30pm Pacific Time.
Jul 8, 17:28 PDT

Update - We are down to about ten customers still seeing workflow backups. Those queues are recovering and the engineering team is continuing to work on options for speeding up that recovery. Next update by 5pm Pacific Time.
Jul 8, 15:32 PDT

Update - Most workflow queues are now processing successfully, although there is still some residual backup. We're continuing to work on rectifying a root cause of some of the backup, to ensure the situation does not recur. Next update by 3:30pm Pacific Time.
Jul 8, 14:00 PDT

Identified - We've made some good progress in the last 30 minutes, hence the slight delay in updates. The workflow queue backups are continuing to process, although there are a lot of events to get through. We have identified the cause of the workflows that were delayed for other reasons (as mentioned in the previous update) and are working on deploying new code to fix that root cause. Next update by 1:45pm Pacific Time.
Jul 8, 12:19 PDT

Update - The team is making progress on identifying root causes for the delayed workflows. Most workflow delays are due to yesterday's ingestion issues and the system is making progress on getting through that backlog. There are a few workflows that are delayed for other reasons, and it is those situations which require additional investigation, which is ongoing. Next update by 12pm Pacific Time.
Jul 8, 10:34 PDT

Investigating - We're seeing some lingering issues from yesterday's data ingestion delays that are causing workflow processing delays for some customers. The engineering team is working on resolving the backups right now. We'll post another update by 10:30am Pacific Time.
Jul 8, 09:12 PDT

Jul 7, 2019

Latency Issues

Resolved - We are comfortable now setting this issue to resolved as we no longer are reporting any latency. If you see any issues you believe may be related please reach out to support@iterable.com.
Jul 7, 19:00 PDT

Update - Thank you for everyone's patience while the team was hard at work investigating the root cause of the issue. We are now seeing no latency being reported, and our machines are showing signs of normal health. We will continue monitoring for the time being, and move to resolved once we feel confident. If you see any issues please reach out to support@iterable.com.
Jul 7, 18:01 PDT

Monitoring - We are now showing signs of our machines returning to a normal state with no latency issues. We are still continuing to monitor for the time being, though we are seeing increasing improvements in terms of speed. Next update at 9PM EDT.
Jul 7, 17:03 PDT

Update - We are continuing work on the latency issue, and are starting to see progress in terms of restoring to a normal state. We will continue to monitor progress, and post updates as they become available. Next update will be 8PM EDT.
Jul 7, 16:02 PDT

Update - The team is hard at work exploring options that can be taken to help with the latency issues, and at this time investigation is still ongoing. Next update at 7PM EDT.
Jul 7, 15:03 PDT

Update - Investigation continues on latency issues. Next update will be at 6PM EDT
Jul 7, 14:03 PDT

Update - Investigation continues on latency issues. Next update will be at 5PM EDT
Jul 7, 13:03 PDT

Update - We are still investigating the root cause for the latency issues. Next update will be at 4PM EDT.
Jul 7, 12:01 PDT

Update - We are still investigating the root cause for the latency issues, but have made some progress since the last update. Next update at 3PM EDT.
Jul 7, 11:00 PDT

Update - We are still in the process of investigating the cause of the latency issues, and our team is currently exploring options for returning to a normal state. We have identified that the affected areas are only a subset of machines, and is not affecting the entire platform. Next update at 2PM PDT
Jul 7, 10:07 PDT

Investigating - We have identified latency occurring with a subset of our machines, and are experiencing latency up to 1 hour. Our team is currently looking into the issue and working on resolving the latency to a normal state. Next update will be 1PM EDT
Jul 7, 08:50 PDT