Data ingestion delays
Investigating - A subset of our customers is currently experiencing delays with data ingestion of up to 2 hours. It is impacting campaign sends, event tracking, user updates, and CSV uploads. Our engineering team is working diligently to resolve this issue. Next update by 1pm PDT.
Jun 26, 11:19 PDT
Web Application ? Operational
API ? Degraded Performance
Email Sending ? Degraded Performance
Email Links ? Operational
Workflows ? Operational
Push Sending Operational
SMS Sending Operational
System Webhooks Operational
Analytics Processing ? Operational
List Upload ? Degraded Performance
User Updates ? Degraded Performance
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
System Metrics Month Week Day
Web App Latency
Fetching
Links API Success Rate ?
Fetching
Links Latency
Fetching
Message Sending API Success Rate ?
Fetching
Message Sending API Latency
Fetching
User Update and Track API Latency ?
Fetching
Past Incidents
Jun 25, 2019
Data ingestion delays, with some impact on workflow triggering
Resolved - Workflow processing is now completely caught up, resolving this incident. If you have any questions or concerns, please don't hesitate to reach out to us at support@iterable.com
Jun 25, 19:48 PDT
Update - Data ingestion is now completely caught up, but it has caused a knock-on effect of backing up workflow processing, which may be delayed up to two hours in some cases. We're monitoring to ensure that this delay is continuing to decrease. Next update by 8pm Pacific Time.
Jun 25, 17:36 PDT
Monitoring - We have identified and corrected the cause the of the data ingestion lag, and the lag has now almost completely caught up. Next update by 5:30pm Pacific Time.
Jun 25, 16:31 PDT
Investigating - We're seeing noticeable lag on data ingestion, primarily with list uploads and event ingestion. For projects with workflows that are triggered by events, this could also delay the progression of those workflows. The engineering team is aware of the cause and is working to fix it. Next update by 4:30pm Pacific Time.
Jun 25, 15:31 PDT
Data ingestion delays
Resolved - All ingestion services continue to operate without any delay, so we're going to close out this incident. Thank you for your patience as we worked through this issue today. If you have any questions or concerns you can reach out to us at support@iterable.com
Jun 25, 11:43 PDT
Monitoring - There are no more data ingestion delays. We will continue to monitor to make sure that this issue does not recur.
Jun 25, 11:04 PDT
Update - The ingestion delay has cleared up in most areas and it remains for only a very small subset of customers now. Next update by 11:00am PDT.
Jun 25, 10:07 PDT
Update - The underlying issue has been resolved, and our engineering team is working on mitigating the remaining lag. Next update at 10:00am PDT
Jun 25, 08:58 PDT
Identified - All data ingestion services are currently experiencing lag, which is impacting event tracking, user updates, and CSV uploads. Our engineering team is working to resolve this issue. Next update at 9:00am PDT.
Jun 25, 08:00 PDT
Jun 24, 2019

No incidents reported.

Jun 23, 2019

No incidents reported.

Jun 22, 2019

No incidents reported.

Jun 21, 2019

No incidents reported.

Jun 20, 2019
User update API issues
Resolved - Our API has stabilized and we are seeing no further issues. Thank you for your patience as we worked through this today. If you have any questions or concerns please contact us at support@iterable.com.
Jun 20, 18:10 PDT
Monitoring - Our /user/update API is back to a normal state. We will continue to monitor to ensure there are no further interruptions. Next Update by 6PM PST.
Jun 20, 17:25 PDT
Investigating - We are experiencing high error rate and timeouts with our /users/update API endpoint. Our engineers are working diligently to resolve this issue. Next update by 6PM PST.
Jun 20, 17:02 PDT
Jun 19, 2019
API issues: Bulk APIs, ListSubscribe and ListUnsubscribe
Resolved - The downstream infrastructure issues have been resolved and all API endpoints are ingesting properly. We will endeavor to identify what API calls might have been impacted during the outage and will be in touch with any customers associated with those calls. Thank you for your patience.
Jun 19, 17:55 PDT
Identified - This afternoon from approximately 4:05pm Pacific Time until 4:25pm, calls to the Bulk API endpoint, as well as calls to ListSubscribe and ListUnsubscribe, were failing, due to a downstream issue with our message processing infrastructure. This would also have impacted list uploads. The situation has now been mitigated, and depending upon the client-side retry logic, calls may yet process successfully. We are working to identify a list of calls that failed and connect with impacted customers. We are also continuing to monitor the infrastructure which was responsible to ensure it does not re-occur.

Next update by 6pm Pacific Time.
Jun 19, 16:54 PDT
Jun 18, 2019

No incidents reported.

Jun 17, 2019

No incidents reported.

Jun 16, 2019

No incidents reported.

Jun 15, 2019

No incidents reported.

Jun 14, 2019
Incorrect rendering of email previews
Resolved - The email preview tool is working properly and we are marking this incident as resolved. If you have any questions or concerns please reach out to us at support@iterable.com.
Jun 14, 14:01 PDT
Update - We are continuing to investigate this issue.
Jun 14, 13:06 PDT
Update - We have filtered out a subset of email previews and are continuing to work with our downstream rendering partner to identify the root cause. If you have any questions please reach out to your CSM or support@iterable.com. The next update will be before 2 PM Pacific Time.
Jun 14, 13:02 PDT
Investigating - We're currently investigating an issue related to the rendering of email previews, and are working with our downstream rendering partner to determine the scope and root cause. For now, we're going to turn off the preview function. We'll have an update on this by 1pm Pacific Time.
Jun 14, 12:00 PDT
Jun 13, 2019

No incidents reported.

Jun 12, 2019

No incidents reported.

← Incident History Powered by Statuspage