This welcome early Christmas present was flown over, by special delivery, from the US to the UK last year for the very first time and for my team at dotdigital it presented some very interesting engineering challenges!
As a fairly new CTO (joined August 2014) for the UK’s leading email marketing platform it was a little daunting walking into dotdigital just three months before the holiday season. Preparations were well underway for the ramp up over the holiday, but suddenly, all that changed.
Walking into the office on 28th November was an exciting day. The day before had been dotdigital’s busiest sending day ever and the team were all elated with the amount of emails delivered. The start of the holiday season was here!
All was ticking over nicely, however our Platform Engineering Director John looked a little engrossed in our monitoring screens. The platform was very busy to say the least. Little had we realised that the UK was waking up to Black Friday fever for the very first time.
What was going on..!?
Effectively the traffic volumes created by bargain hunters had performed one of the largest legitimate Distributed Denial of Service (DDoS) attacks I have ever witnessed and ecommerce sites were taking the full brunt.
Jumping on the Black Friday fever bandwagon our UK ecommerce customers were sending more than ever with huge campaigns lined up to their subscriber base.
Many were being created before our eyes to ride the wave of hype generated in the news. There were however some delays delivering these emails into the Internet Service Providers (ISPs). Like the crowds being funnelled into the stores with restrictions on how many could get through each minute, the ISPs were throttling the rate at which email was delivered to them. This caused delays for some of our customers hoping to deliver their email campaign into the inbox of the eager bargain hunters.
dotdigital Engagement Cloud had its busiest ever day on the 27th November, on Black Friday it sent an additional 30% more.
The two pink dots represent Black Friday and Cyber Monday. Taking our previous busiest day as a reference point you can see the true effect of Black Friday and Cyber Monday. A huge peak in volume, followed by the usual weekend drop off, followed by another peak on Cyber Monday.
So how did we cope?
We have always invested in the best of breed technology at dotdigital and as I mentioned earlier we had compute reserves on-standby. We were able to quickly spin up these resources and start utilizing them as we saw the peak in demand.
As you can see the sending volume gradually increased with the all hype in the UK… we had our US customers come online sending their early morning Black Friday campaigns which culminated in our busiest hours we had ever seen between 11:00 and 12:00 GMT.
We saw email opens grow much quicker than an average day with much bigger peaks earlier on in the day (all times are in GMT).
So what did we learn and what can you take away?
We started our holiday season (including Black Friday and Cyber Monday) planning earlier this year. We are again making sure we again have enough compute power available to cope with peak demands. We have plans in place to deliver a 100% increase in emails compared to the volume we saw on Black Friday last year.
Email marketing is still the most successful digital channel with one of the highest ROIs. We are anticipating a similar Black Friday fever in the UK and US again and like us, we want our customers to be ready to ride the wave.
Steve, CTO dotmailer