Live Status\Incident History
December 2, 2024
Healthcheck Assertion Failed
Outage
Opened: Dec 2, 2024, 7:16 AM UTC
Duration: 10m 6s
Affected services:Workflow Execution Monitoring - US1
- OpenedDec 2, 2024, 7:16 AM UTC
Issue opened on monitor failure.
- ResolvedDec 2, 2024, 7:26 AM UTC
Issue resolved on monitor recovery.
Healthcheck Assertion Failed
Outage
Opened: Dec 2, 2024, 5:05 AM UTC
Duration: 20m 7s
Affected services:Workflow Execution Monitoring - US1
- OpenedDec 2, 2024, 5:05 AM UTC
Issue opened on monitor failure.
- ResolvedDec 2, 2024, 5:20 AM UTC
Issue resolved on monitor recovery.
- UnresolvedDec 2, 2024, 5:36 AM UTC
Issue re-opened on monitor failure.
- ResolvedDec 2, 2024, 5:41 AM UTC
Issue resolved on monitor recovery.
November 25, 2024
Healthcheck Assertion Failed
Outage
Opened: Nov 25, 2024, 4:54 PM UTC
Duration: 44m 14s
Affected services:Workflow Execution Monitoring - US1
- OpenedNov 25, 2024, 4:54 PM UTC
Issue opened on monitor failure.
- ResolvedNov 25, 2024, 5:38 PM UTC
Issue resolved on monitor recovery.
November 22, 2024
Pax8 Rate Limit Issue
Degraded Performance
Opened: Nov 22, 2024, 5:00 PM UTC
Duration: 8h 47m 49s
- IdentifiedNov 22, 2024, 5:00 PM UTC
Shortly before 12:00 PM EST today, we became aware of an issue affecting users of the Pax8 integration on the US East instance of Rewst. This issue is resulting in "429 - Too Many Requests" errors being returned from the Pax8 integration. This issue prevents users from making changes to the Pax8 integration and running workflows utilizing the integration. This will affect users running our User Onboarding and User Offboarding crates that utilize the Pax8 Integration. Our team is actively collaborating with Pax8 to resolve the issue as quickly as possible. We are continuously monitoring the affected services to evaluate the extent and progression of the problem. We greatly appreciate your patience and understanding during this time. Updates will be shared as more information becomes available.
- ResolvedNov 23, 2024, 1:47 AM UTC
Pax8 actions are successfully executing
November 20, 2024
Healthcheck Assertion Failed
Outage
Opened: Nov 20, 2024, 10:12 PM UTC
Duration: 15m 40s
Affected services:Workflow Execution Monitoring - UK
- OpenedNov 20, 2024, 10:12 PM UTC
Issue opened on monitor failure.
- ResolvedNov 20, 2024, 10:28 PM UTC
Issue resolved on monitor recovery.
November 17, 2024
Healthcheck Assertion Failed
Outage
Opened: Nov 17, 2024, 8:43 PM UTC
Duration: 30m 28s
Affected services:Workflow Execution Monitoring - US1
- OpenedNov 17, 2024, 8:43 PM UTC
Issue opened on monitor failure.
- ResolvedNov 17, 2024, 9:14 PM UTC
Issue resolved on monitor recovery.
Healthcheck Assertion Failed
Outage
Opened: Nov 17, 2024, 12:12 AM UTC
Duration: 20m 49s
Affected services:Workflow Execution Monitoring - AU
- OpenedNov 17, 2024, 12:12 AM UTC
Issue opened on monitor failure.
- ResolvedNov 17, 2024, 12:33 AM UTC
Issue resolved on monitor recovery.
November 13, 2024
Live App Builder Sites in Prod US Unavailable
Minor Outage
Opened: Nov 13, 2024, 11:33 PM UTC
Duration: 1h 5m
- InvestigatingNov 13, 2024, 11:33 PM UTC
Live App Builder sites are unavailable.
- ResolvedNov 14, 2024, 12:38 AM UTC
We were performing upgrades across all Rewst instances to how we provision domains for live sites. The upgrades were successful in all of our instances, but failed in prod US at 6:33 PM EST. We were troubleshooting for 50 minutes and managed to roll back at 7:23PM bringing rew.st domains back up. There was additional work needed to get custom domains back, which was completed at 7:38PM.
November 10, 2024
Healthcheck Assertion Failed
Outage
Opened: Nov 10, 2024, 12:04 AM UTC
Duration: 11m 19s
Affected services:Workflow Execution Monitoring - UK
- OpenedNov 10, 2024, 12:04 AM UTC
Issue opened on monitor failure.
- ResolvedNov 10, 2024, 12:16 AM UTC
Issue resolved on monitor recovery.
November 6, 2024
Site Unreachable
Outage
Opened: Nov 6, 2024, 4:00 PM UTC
Duration: 12m 17s
Affected services:Workflow Execution Monitoring - US1
- OpenedNov 6, 2024, 4:00 PM UTC
Issue opened on monitor failure.
- ResolvedNov 6, 2024, 4:12 PM UTC
Issue resolved on monitor recovery.
- ResolvedNov 6, 2024, 9:27 PM UTC
On November 6, 2024, from 10:53 AM EST to 11:40 AM EST, an unintended disruption occurred in our production environment, impacting workflow processing for our US customers. While working towards a resolution, there were failures of workflows running at the time. This temporarily affected real-time processing across parts of our application. The issue was resolved and workflows are running as normal. Thank you for your understanding as we work to strengthen our safeguards. We are committed to maintaining the reliability and resilience of our platform and will continue to enhance our processes to prevent such incidents in the future.
November 1, 2024
Healthcheck Assertion Failed
Outage
Opened: Nov 1, 2024, 8:40 PM UTC
Duration: 25m 2s
Affected services:Workflow Execution Monitoring - US1
- OpenedNov 1, 2024, 8:40 PM UTC
Issue opened on monitor failure.
- ResolvedNov 1, 2024, 9:05 PM UTC
Issue resolved on monitor recovery.
Healthcheck Assertion Failed
Outage
Opened: Nov 1, 2024, 2:33 PM UTC
Duration: 2h 59m 31s
Affected services:Workflow Execution Monitoring - US1
- OpenedNov 1, 2024, 2:33 PM UTC
Issue opened on monitor failure.
- ResolvedNov 1, 2024, 4:31 PM UTC
Issue resolved on monitor recovery.
- UnresolvedNov 1, 2024, 4:56 PM UTC
Issue re-opened on monitor failure.
- ResolvedNov 1, 2024, 5:58 PM UTC
Issue resolved on monitor recovery.
Rewst Experiencing Performance Degradation for US Region
Degraded Performance
Opened: Nov 1, 2024, 2:25 PM UTC
Duration: 6h 47m 16s
Affected services:Workflow Execution Monitoring - US1
- InvestigatingNov 1, 2024, 2:25 PM UTC
Rewst is experiencing performance degradation affecting user experience. Workflows are running, however the user interface experience and workflow execution speeds are degraded. We are investigating and we will share more information as it becomes available.
- IdentifiedNov 1, 2024, 5:04 PM UTC
We have identified the root cause of the performance issues impacting user experience on the Rewst platform. Our team has initiated remediation efforts and we are observing initial improvements in both user interface responsiveness and workflow execution speeds. We will continue to monitor the situation closely and provide further updates as progress is made. Thank you for your patience and understanding.
- ResolvedNov 1, 2024, 9:12 PM UTC
Update: The performance issues affecting Rewst’s user experience and workflow execution speeds have been fully resolved. Our team has completed remediation efforts, and all systems are now operating as expected. We appreciate your patience and understanding throughout the process.
October 28, 2024
Healthcheck Assertion Failed
Outage
Opened: Oct 28, 2024, 8:56 PM UTC
Duration: 9m 35s
Affected services:Workflow Execution Monitoring - US1
- OpenedOct 28, 2024, 8:56 PM UTC
Issue opened on monitor failure.
- ResolvedOct 28, 2024, 9:05 PM UTC
Issue resolved on monitor recovery.
October 15, 2024
Healthcheck Assertion Failed
Outage
Opened: Oct 15, 2024, 12:13 AM UTC
Duration: 25m 19s
Affected services:Workflow Execution Monitoring - US1
- OpenedOct 15, 2024, 12:13 AM UTC
Issue opened on monitor failure.
- ResolvedOct 15, 2024, 12:39 AM UTC
Issue resolved on monitor recovery.
October 14, 2024
Site Unreachable
Outage
Opened: Oct 14, 2024, 10:33 AM UTC
Duration: 8m 52s
Affected services:Workflow Execution Monitoring - UK
- OpenedOct 14, 2024, 10:33 AM UTC
Issue opened on monitor failure.
- ResolvedOct 14, 2024, 10:42 AM UTC
Issue resolved on monitor recovery.
October 13, 2024
Healthcheck Assertion Failed
Outage
Opened: Oct 13, 2024, 3:18 PM UTC
Duration: 36m 29s
Affected services:Workflow Execution Monitoring - US1
- OpenedOct 13, 2024, 3:18 PM UTC
Issue opened on monitor failure.
- ResolvedOct 13, 2024, 3:55 PM UTC
Issue resolved on monitor recovery.
October 10, 2024
Healthcheck Assertion Failed
Outage
Opened: Oct 10, 2024, 2:39 PM UTC
Duration: 20m 45s
Affected services:Workflow Execution Monitoring - UK
- OpenedOct 10, 2024, 2:39 PM UTC
Issue opened on monitor failure.
- ResolvedOct 10, 2024, 3:00 PM UTC
Issue resolved on monitor recovery.
Healthcheck Assertion Failed
Outage
Opened: Oct 10, 2024, 12:14 AM UTC
Duration: 9m 42s
Affected services:Workflow Execution Monitoring - US1
- OpenedOct 10, 2024, 12:14 AM UTC
Issue opened on monitor failure.
- ResolvedOct 10, 2024, 12:24 AM UTC
Issue resolved on monitor recovery.
October 4, 2024
Healthcheck Assertion Failed
Outage
Opened: Oct 4, 2024, 7:20 PM UTC
Duration: 19m 57s
Affected services:Workflow Execution Monitoring - US1
- OpenedOct 4, 2024, 7:20 PM UTC
Issue opened on monitor failure.
- ResolvedOct 4, 2024, 7:40 PM UTC
Issue resolved on monitor recovery.
September 30, 2024
Healthcheck Assertion Failed
Outage
Opened: Sep 30, 2024, 10:46 PM UTC
Duration: 21m 33s
Affected services:Workflow Execution Monitoring - UK
- OpenedSep 30, 2024, 10:46 PM UTC
Issue opened on monitor failure.
- ResolvedSep 30, 2024, 11:08 PM UTC
Issue resolved on monitor recovery.
September 25, 2024
Healthcheck Assertion Failed
Outage
Opened: Sep 25, 2024, 11:00 AM UTC
Duration: 1h 11m 40s
Affected services:Workflow Execution Monitoring - UK
- OpenedSep 25, 2024, 11:00 AM UTC
Issue opened on monitor failure.
- ResolvedSep 25, 2024, 12:12 PM UTC
Issue resolved on monitor recovery.
September 21, 2024
Healthcheck Assertion Failed
Outage
Opened: Sep 21, 2024, 5:36 PM UTC
Duration: 10m 1s
Affected services:Workflow Execution Monitoring - US1
- OpenedSep 21, 2024, 5:36 PM UTC
Issue opened on monitor failure.
- ResolvedSep 21, 2024, 5:46 PM UTC
Issue resolved on monitor recovery.
September 9, 2024
Healthcheck Assertion Failed
Outage
Opened: Sep 9, 2024, 4:19 AM UTC
Duration: 45m 14s
Affected services:Workflow Execution Monitoring - US1
- OpenedSep 9, 2024, 4:19 AM UTC
Issue opened on monitor failure.
- ResolvedSep 9, 2024, 5:04 AM UTC
Issue resolved on monitor recovery.
August 25, 2024
Healthcheck Assertion Failed
Outage
Opened: Aug 25, 2024, 12:15 AM UTC
Duration: 12m 35s
Affected services:Workflow Execution Monitoring - US1
- OpenedAug 25, 2024, 12:15 AM UTC
Issue opened on monitor failure.
- ResolvedAug 25, 2024, 12:28 AM UTC
Issue resolved on monitor recovery.
August 13, 2024
Healthcheck Assertion Failed
Outage
Opened: Aug 13, 2024, 10:37 AM UTC
Duration: 19m 56s
Affected services:Workflow Execution Monitoring - US1
- OpenedAug 13, 2024, 10:37 AM UTC
Issue opened on monitor failure.
- ResolvedAug 13, 2024, 10:57 AM UTC
Issue resolved on monitor recovery.
August 10, 2024
Healthcheck Assertion Failed
Outage
Opened: Aug 10, 2024, 1:29 AM UTC
Duration: 50m 57s
Affected services:Workflow Execution Monitoring - US1
- OpenedAug 10, 2024, 1:29 AM UTC
Issue opened on monitor failure.
- ResolvedAug 10, 2024, 2:20 AM UTC
Issue resolved on monitor recovery.
August 7, 2024
Update Service Ticket Action Failure
Minor Outage
Opened: Aug 7, 2024, 4:22 PM UTC
Duration: 3h 17m 14s
Affected services:Workflow Execution Monitoring - US1
- InvestigatingAug 7, 2024, 4:22 PM UTC
We have detected an issue that is affecting workflows that update PSA tickets. We are actively looking into the issue and will provide a solution and updates as soon as possible. We apologize for any inconvenience.
- IdentifiedAug 7, 2024, 5:50 PM UTC
The action affected is: ConnectWise Update Service Ticket (v1). Workflows that contain this action are failing. This also affects crates that use this action, such as User Onboarding and User Offboarding V1 and V2. Once the recovery process is complete, we will re-run all failed New User Workflows.
- ResolvedAug 7, 2024, 7:39 PM UTC
We are pleased to inform you that the issue affecting workflows containing the ConnectWise Update Service Ticket (v1) action has been successfully resolved. Normal functionality has resumed. All services should now be operating as expected. We will now begin re-running the New User Workflows that failed today. We apologize for any inconvenience caused during this period and appreciate your patience and understanding. If you encounter any further issues or have any questions, please do not hesitate to reach out to our support team.
Healthcheck Assertion Failed
Outage
Opened: Aug 7, 2024, 12:12 AM UTC
Duration: 10m 15s
Affected services:Workflow Execution Monitoring - US1
- OpenedAug 7, 2024, 12:12 AM UTC
Issue opened on monitor failure.
- ResolvedAug 7, 2024, 12:22 AM UTC
Issue resolved on monitor recovery.
July 17, 2024
Healthcheck Assertion Failed
Outage
Opened: Jul 17, 2024, 2:12 PM UTC
Duration: 10m 24s
Affected services:Workflow Execution Monitoring - US1
- OpenedJul 17, 2024, 2:12 PM UTC
Issue opened on monitor failure.
- ResolvedJul 17, 2024, 2:22 PM UTC
Issue resolved on monitor recovery.
Forms users unable to log in
Minor Outage
Opened: Jul 17, 2024, 10:53 AM UTC
Duration: 56h 36m 51s
- InvestigatingJul 17, 2024, 10:53 AM UTC
Forms users are reporting the inability to log in to Rewst or very long wait times for logging in.
- InvestigatingJul 17, 2024, 12:36 PM UTC
Performance for forms-only user logins has been improved, but we are still working on permanent resolution.
- IdentifiedJul 17, 2024, 2:22 PM UTC
We have identified an issue within our forms trigger that could potentially be the source of the intermittent login delays. A fix has been applied to address this, and we are actively monitoring the situation to ensure the effectiveness of the solution. Please report any further login delays.
- IdentifiedJul 17, 2024, 2:25 PM UTC
This issue is also being tracked on our docs site here: https://docs.rewst.help/updates/issue-alerts
- MonitoringJul 17, 2024, 3:46 PM UTC
- InvestigatingJul 18, 2024, 8:17 AM UTC
We are continuing to see some issues and are continuing to investigate
- MonitoringJul 18, 2024, 10:32 AM UTC
The root cause has been identified and a temporary measure has been put in place to resolve the issue. We will update further once a permanent fix has been deployed, but the platform should now be running as expected.
- ResolvedJul 19, 2024, 7:29 PM UTC
This issue has been successfully resolved and we are closing this alert.
July 9, 2024
Site Unreachable
Outage
Opened: Jul 9, 2024, 2:07 AM UTC
Duration: 17m 59s
Affected services:Workflow Execution Monitoring - UK
- OpenedJul 9, 2024, 2:07 AM UTC
Issue opened on monitor failure.
- ResolvedJul 9, 2024, 2:25 AM UTC
Issue resolved on monitor recovery.
June 24, 2024
Site Unreachable
Outage
Opened: Jun 24, 2024, 9:01 AM UTC
Duration: 21m 32s
Affected services:Workflow Execution Monitoring - UK
- OpenedJun 24, 2024, 9:01 AM UTC
Issue opened on monitor failure.
- ResolvedJun 24, 2024, 9:02 AM UTC
Issue resolved on monitor recovery.
- UnresolvedJun 24, 2024, 9:06 AM UTC
Issue re-opened on monitor failure.
- ResolvedJun 24, 2024, 9:26 AM UTC
Issue resolved on monitor recovery.
Site Unreachable
Outage
Opened: Jun 24, 2024, 7:01 AM UTC
Duration: 9m 7s
Affected services:Workflow Execution Monitoring - UK
- OpenedJun 24, 2024, 7:01 AM UTC
Issue opened on monitor failure.
- ResolvedJun 24, 2024, 7:10 AM UTC
Issue resolved on monitor recovery.
June 22, 2024
Healthcheck Assertion Failed
Outage
Opened: Jun 22, 2024, 12:29 AM UTC
Duration: 31m 11s
Affected services:Workflow Execution Monitoring - US1
- OpenedJun 22, 2024, 12:29 AM UTC
Issue opened on monitor failure.
- ResolvedJun 22, 2024, 12:40 AM UTC
Issue resolved on monitor recovery.
- UnresolvedJun 22, 2024, 12:55 AM UTC
Issue re-opened on monitor failure.
- ResolvedJun 22, 2024, 1:11 AM UTC
Issue resolved on monitor recovery.
- UnresolvedJun 22, 2024, 1:47 AM UTC
Issue re-opened on monitor failure.
- ResolvedJun 22, 2024, 1:51 AM UTC
Issue resolved on monitor recovery.
June 21, 2024
Healthcheck Assertion Failed
Outage
Opened: Jun 21, 2024, 12:02 AM UTC
Duration: 8m 54s
Affected services:Workflow Execution Monitoring - US1
- OpenedJun 21, 2024, 12:02 AM UTC
Issue opened on monitor failure.
- ResolvedJun 21, 2024, 12:11 AM UTC
Issue resolved on monitor recovery.
June 14, 2024
Site Unreachable
Outage
Opened: Jun 14, 2024, 6:06 AM UTC
Duration: 1m 22s
Affected services:Workflow Execution Monitoring - UK
- OpenedJun 14, 2024, 6:06 AM UTC
Issue opened on monitor failure.
- ResolvedJun 14, 2024, 6:07 AM UTC
Issue resolved on monitor recovery.
June 12, 2024
Healthcheck Assertion Failed
Outage
Opened: Jun 12, 2024, 3:31 AM UTC
Duration: 16m 51s
Affected services:Workflow Execution Monitoring - US1
- OpenedJun 12, 2024, 3:31 AM UTC
Issue opened on monitor failure.
- ResolvedJun 12, 2024, 3:47 AM UTC
Issue resolved on monitor recovery.
June 8, 2024
Healthcheck Assertion Failed
Outage
Opened: Jun 8, 2024, 11:46 PM UTC
Duration: 9m 29s
Affected services:Workflow Execution Monitoring - US1
- OpenedJun 8, 2024, 11:46 PM UTC
Issue opened on monitor failure.
- ResolvedJun 8, 2024, 11:56 PM UTC
Issue resolved on monitor recovery.
June 7, 2024
Healthcheck Assertion Failed
Outage
Opened: Jun 7, 2024, 6:56 PM UTC
Duration: 1h 22m 48s
Affected services:Workflow Execution Monitoring - US1
- OpenedJun 7, 2024, 6:56 PM UTC
Issue opened on monitor failure.
- ResolvedJun 7, 2024, 8:18 PM UTC
Issue resolved on monitor recovery.
Site Unreachable
Outage
Opened: Jun 7, 2024, 8:46 AM UTC
Duration: 19m 51s
Affected services:Workflow Execution Monitoring - UK
- OpenedJun 7, 2024, 8:46 AM UTC
Issue opened on monitor failure.
- ResolvedJun 7, 2024, 9:06 AM UTC
Issue resolved on monitor recovery.
June 5, 2024
Healthcheck Assertion Failed
Outage
Opened: Jun 5, 2024, 4:19 PM UTC
Duration: 14m 54s
Affected services:Workflow Execution Monitoring - US1
- OpenedJun 5, 2024, 4:19 PM UTC
Issue opened on monitor failure.
- ResolvedJun 5, 2024, 4:33 PM UTC
Issue resolved on monitor recovery.
May 22, 2024
Healthcheck Assertion Failed
Outage
Opened: May 22, 2024, 6:05 AM UTC
Duration: 46s
Affected services:Workflow Execution Monitoring - UK
- OpenedMay 22, 2024, 6:05 AM UTC
Issue opened on monitor failure.
- ResolvedMay 22, 2024, 6:06 AM UTC
Issue resolved on monitor recovery.
April 30, 2024
Healthcheck Assertion Failed
Minor Outage
Opened: Apr 30, 2024, 8:06 PM UTC
Duration: 13m 29s
Affected services:Workflow Execution Monitoring - US1
- OpenedApr 30, 2024, 8:06 PM UTC
Issue opened on monitor failure.
- InvestigatingApr 30, 2024, 8:16 PM UTC
- InvestigatingApr 30, 2024, 8:16 PM UTC
- IdentifiedApr 30, 2024, 8:16 PM UTC
App running, engine work in progress
- ResolvedApr 30, 2024, 8:19 PM UTC
Failed migration from new feature deployment - we are back to normal operations. We were able to fix the migration issue without rolling back. The status page will update automatically when it determines we are back.
- ResolvedApr 30, 2024, 8:19 PM UTC
April 23, 2024
Healthcheck Assertion Failed
Degraded Performance
Opened: Apr 23, 2024, 4:26 PM UTC
Duration: 20m 8s
Affected services:Workflow Execution Monitoring - US1
- OpenedApr 23, 2024, 4:26 PM UTC
Issue opened on monitor failure.
- ResolvedApr 23, 2024, 4:46 PM UTC
Issue resolved on monitor recovery.
April 12, 2024
Healthcheck Assertion Failed
Degraded Performance
Opened: Apr 12, 2024, 1:06 AM UTC
Duration: 20m 30s
Affected services:Workflow Execution Monitoring - US1
- OpenedApr 12, 2024, 1:06 AM UTC
Issue opened on monitor failure.
- ResolvedApr 12, 2024, 1:26 AM UTC
Issue resolved on monitor recovery.
- UnresolvedApr 12, 2024, 1:47 AM UTC
Issue re-opened on monitor failure.
- ResolvedApr 12, 2024, 1:48 AM UTC
Issue resolved on monitor recovery.
April 9, 2024
Healthcheck Assertion Failed
Degraded Performance
Opened: Apr 9, 2024, 6:02 PM UTC
Duration: 19m 47s
Affected services:Workflow Execution Monitoring - US1
- OpenedApr 9, 2024, 6:02 PM UTC
Issue opened on monitor failure.
- ResolvedApr 9, 2024, 6:22 PM UTC
Issue resolved on monitor recovery.
April 4, 2024
Healthcheck Assertion Failed
Degraded Performance
Opened: Apr 4, 2024, 8:15 PM UTC
Duration: 20m 35s
Affected services:Workflow Execution Monitoring - US1
- OpenedApr 4, 2024, 8:15 PM UTC
Issue opened on monitor failure.
- ResolvedApr 4, 2024, 8:36 PM UTC
Issue resolved on monitor recovery.