Intermittent 502 response from Taskflow Status API
Incident
Report for Informatica Intelligent Cloud Services
Postmortem
INCIDENT START/END: Oct-16-2024 5:50 to Oct-17-2024 21:15 UTC
Please find the RCA link available : https://knowledge.informatica.com/s/article/RCA-incident-for-IICS-NA-East-2-Services-IICS-US-West-3-Services-on-Oct-17
Resolved
This incident has been resolved.
Monitoring
The fix implemented in US West 3 at 16:17 PT is working as expected and all new connections are operating normally.
A few existing connections may show the 502 error when calling the taskflow status api call, any such existing connections will be cleared by 19:00 PT.
Taskflow status API calls are operating normally NA East 2 since the fix at 13:54 PT.
Taskflow executions via IDMC scheduler or runajob cli were not impacted by this issue.
We continue monitoring the services.
A few existing connections may show the 502 error when calling the taskflow status api call, any such existing connections will be cleared by 19:00 PT.
Taskflow status API calls are operating normally NA East 2 since the fix at 13:54 PT.
Taskflow executions via IDMC scheduler or runajob cli were not impacted by this issue.
We continue monitoring the services.
Identified
We have identified that the issue with taskflow status api call is seen again in US West 3 pod.
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
Issue is not seen in NA East 2 pod.
We will keep status page updated.
We apologize for the continued interruptions to taskflow status api call.
Taskflow executions via IDMC scheduler or runajob cli are not impacted by this issue.
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
Issue is not seen in NA East 2 pod.
We will keep status page updated.
We apologize for the continued interruptions to taskflow status api call.
Taskflow executions via IDMC scheduler or runajob cli are not impacted by this issue.
Monitoring
We have implemented a fix for the issue and taskflow status API is operating normally in US West 3.
We continue to monitor the services.
NA East 2 has not run into the issue since the update at 13:54 PM PT.
We continue to monitor the services.
NA East 2 has not run into the issue since the update at 13:54 PM PT.
Update
We have implemented a fix for the issue and taskflow status API is operating normally in US West 3.
We continue to monitor the services.
NA East 2 has not run into the issue since the update at 13:54 PM PT.
We continue to monitor the services.
NA East 2 has not run into the issue since the update at 13:54 PM PT.
Update
We are continuing to investigate this issue.
Investigating
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
We are currently investigating this issue on priority.
Identified
We have identified that the issue with taskflow status api call is seen again in US West 3 pod.
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
Issue is not seen in NA East 2 pod.
We will keep status page updated.
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
Issue is not seen in NA East 2 pod.
We will keep status page updated.
Monitoring
We have implemented a fix for the issue and taskflow status API is operating normally.
We continue to monitor the services.
Taskflow executions via IDMC scheduler or runajob cli were not impacted by this issue.
We continue to monitor the services.
Taskflow executions via IDMC scheduler or runajob cli were not impacted by this issue.
Investigating
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
We are currently investigating this issue on priority.
Identified
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority.
We are currently investigating this issue on priority.
Monitoring
We have implemented a fix for the issue and the taskflow API calls are operating normally.
We continue monitoring the services.
We continue monitoring the services.
Investigating
Users may observe 502 errors intermittently when using Taskflow status API [/active-bpel/services/tf/status/].
We are currently investigating this issue on priority. Please reach out to Informatica Global Customer Support if any concerns.
We are currently investigating this issue on priority. Please reach out to Informatica Global Customer Support if any concerns.
This incident affected: IICS NA East 2 Services (Cloud Data Integration) and IICS US West 3 Services (Cloud Data Integration).