Platform Service Degradation - Studio Email Alias Error
Incident Report for Firstup
Postmortem

Summary:

On Tuesday August 6th, 2024, starting at approximately 8:24 PM PT, we received user reports that when creating content using a template that includes an assigned email alias that differs from the default, the email alias reverted to a default alias for the program when attempting to save. It was identified that the email alias error was caused by a regression introduced by a bug fix where a restricted user could potentially publish as an unassigned email alias. A hotfix was deployed to rollback this change which resolved the issue.

Impact: 

The scope of this incident affected users who attempted to create and send campaigns between 08.06.2024 8:24 PM PT and 08.07.2024 12:31 PM PT. Impacted users experienced upon selecting an email alias within a campaign, and attempting to save, the email alias reverted to the default if re-opened instead of applying the selected alias. Users additionally could not update or save a draft to fix, this caused campaign delays and prevented impacted users from sending time-sensitive communication from the appropriate alias.

Root Cause: 

The root cause was identified as a regression introduced during the latest deployment of a hotfix to address potential restricted users who publish as an unassigned email alias when using a template. The scoping of the original bug fix was too restrictive, resulting in the inability of any users regardless of permission level to be able to override the default alias in campaign drafts.

Mitigation:

Once the root cause was identified a revert of the deployed hotfix that caused the email alias error was mitigated, restoring functionality by 08.07.2024 12:31 PM PT.

Recurrence Prevention:

A fix will be deployed on the original software change done for the noted bug where a restricted user could potentially publish as an unassigned email alias. Firstup will explicitly test to be certain that author aliases can be set on new drafts and are not overridden by the default alias and will add this to all future regression tests.

Posted Aug 22, 2024 - 18:17 UTC

Resolved
A hotfix has deployed which has been verified to correct the underlying behavior of the author alias reverting on campaign drafts. Marking all components fully operational and resolving incident.
Posted Aug 07, 2024 - 19:45 UTC
Identified
We have determined that this problem is related to a regression introduced by a bug fix where a restricted user could potentially publish as an unassigned email alias. We are going to roll back that change in a forthcoming hotfix which should mitigate the problem, and rework a more comprehensive fix to the original bug. We are going through the change management process now to get the hotfix reviewed, verified, and pushed to production.
Posted Aug 07, 2024 - 18:01 UTC
Update
We are continuing to investigate this Studio service disruption.

Another update will be provided within the next 1 hour.
Posted Aug 07, 2024 - 17:28 UTC
Investigating
We are currently investigating reports of email alias in a campaign results reverting to the default in Studio.

We will provide you with an update with 1 hour.
Posted Aug 07, 2024 - 16:31 UTC
This incident affected: Products (Creator Studio).