AB Test "Delivered" Metric Too High (or Low)

Incident Report for Letterhead

Resolved

This incident has been resolved.
Posted Nov 13, 2025 - 15:22 EST

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Nov 13, 2025 - 14:58 EST

Update

Just a status update: the issue is that an AB test actually contains multiple newsletter campaigns - variant A, variant B, and "everyone else." We have a calculation error (on the display side, not the tracking side) in which those metrics just aren't rolling-up under certain conditions. Thank you for your patience.
Posted Nov 13, 2025 - 12:20 EST

Identified

For people sending AB Tests, we have identified an edge case where the "delivered" metric in your newsletter edition report is either too high or too low. This is a display calculation bug and ostensibly affects only the display. The numbers under the hood remain sound. We are working on a patch. Thanks for your patience.
Posted Nov 13, 2025 - 11:27 EST
This incident affected: Services (Metrics).