Hazard Notice - Office365 Delivery - 19/11/19 3:45pm ongoing
- Antispam mail queues are now clear, and all mail is flowing normally
- We can confirm that this incident was caused by a change by Microsoft to their Office 365 email platform
- Microsoft incident ID EX196121, Exchange Online
- Text of their notice
Users that were receiving emails from external users may have experienced delays. Additionally, we’ve also confirmed that delivery of emails with attachments sent to internal endpoints may have been delayed. During this time, mail will have been queued by the send and is expected to be delivered normally once the situation is resolved.
Final status: We’ve rolled back the change and can confirm that service is restored.
Scope of impact: This issue could affect any mail user capable of receiving mail from external users, or internal mail with included attachments.
Start time: Tuesday, November 19, 2019, 2:40 PM (1:40 AM UTC)
End time: Wednesday, November 20, 2019, 12:50 AM (11/19/2019, 11:50 AM UTC)
Preliminary root cause: A configuration change to our infrastructure caused unexpected impact to mail flow.
- Later today we will update this notice when all queued mail is confirmed to be delivered
- Mail delivery to Office365 now appears to be flowing again with no delay. However there is still a large number of emails queued on our antispam cluster destined for domain hosted on 3rd party mail system, which we are working to clear.
This does not appear have any noticeable impact on regular mailflow, and we are keeping it under observation
Estimated Time of Repair
- This is a Hazard Notice only
- It appears that there a delay in mail delivery to Office365 mailboxes. This is being seen across our 3 outbound email servers, as well as the Antispam cluster
- 19 November 2019 3:45pm - ongoing
- Mail delivery to Office365 mailboxes
- Users with mailboxes hosted on Office365 may see delays in receiving email from external parties