Sitecore Azure Blog Storage

June 06, 2021

TLDR

I'd recently had to install and configure Sitecore's Azure Blob Storage module for a client and it didn't work out of the box for us but it wasn't obvious why. It turns out that for larger databases the SQL Commands may timeout without telling you. 

What is Azure Blob Storage

Have you ever thought how pesky it was to have media library items bulking up your database? What if there was a file server in Azure where you could put them instead and reduce the strain on your CD server! But wait, there is!!

In Azure you can create a Blob Storage account. This is just a bulky file storage (think db backups, large graphics, zipped up Azure templates, container images etc) with an account name you use to reference it. There's an Azure Storage Explorer application to manage your account's files. There's a lot more to it with file states (hot, cold) and setting permission access but it's perfect for putting Sitecore media into. 

Issue

After running the Invoke-Migration.ps1 script we noticed the blob storage size wasn't matching our expectations. On closer inspection the migration script would fail without producing an error. We needed the module working but it would require a research spike to get to the bottom of it. 

What's Happening?

What the blob storage module is trying to do is make a copy of every media item in the blob storage and intercept media requests to look them up in the blob storage instead. There's more to it than that though here's the lifecycle of the blob storage:

  1. Migration script is run
  2. Media files are copied to blob storage
  3. Blob storage ids are stored on media items 
  4. A manual database cleanup task is run from the Sitecore admin tools which removes the media item's blob values from the database
  5. In the event you want to bring a copy of your database back down to your lower environments you need to run the revert script
  6. Blob values are copied back into the database
  7. You can backup and download the database 

Underlying Causes

We first discovered there was a failure when we added some logging to the SQL Commands. It indicated that the request timeout was thrown. This meant the migration failed because of the length of time the script would take to run. The master and web database were so large they could never finish migrating before SQL dropped the request. Once we increased the SQL Command timeout we were able to finally move on.

Solution

Ultimately I provided the script to Sitecore Support and they will incorporate the relevant changes in the upcoming versions so you might already have it but if not, here's what I did. Open Invoke-Migration.ps1 and add '$Command.CommandTimeout = 2000' (adjust 2000 as needed) to all SQLCommand objects in the script. Each environment was different but this was sufficient to complete the migrations.