Sitecore Azure Blob Storage

Tags

Accessibility AI Analytics ARM Template Art Azure Blob Storage Brightcove Cache Caching Catching Exceptions Chatbot Compiling Content Search Continuous Integration Copyleft Creativity CSharp Datasource Design Patterns DevOps DotNet Editor Tabs Experience Editor Extensions Extranet Field Suite First Fun Generics Genetic Algorithm Griping HTML Integration Testing Interfaces Java Javascript JQuery Layouts Linq Localization Logic Lucene Machine Learning Microsoft Microsoft Cognitive Services MongoDB Multi Site MultivariateTesting Page Editor People Personalization PHP Propaganda SDN Security SharedSource Sheer UI Sitecore Sitecore Cognitive Services Sitecore Community Sitecore Dictionary Sitecore Events Sitecore Experience Sitecore Express Sitecore Fields Sitecore Marketplace Sitecore Marketplace Sitecore Modules Sitecore Rules Sitecore Security Sitecore Symposium Sitecore Upgrade Software SQL Stories Strategy Sublayouts SVN Tactics Templates Test Driven Development Umbraco Unit Testing UserManager War Web Controls Web Testing WebApp Website What Whom Why Wysiwyg Xaml xDB XML XPath Yellow Screen Of Death YouTube

< Previous Post Next Post >

June 06, 2021

Tags: Sitecore, Azure, Blob Storage

TLDR

I'd recently had to install and configure Sitecore's Azure Blob Storage module for a client and it didn't work out of the box for us but it wasn't obvious why. It turns out that for larger databases the SQL Commands may timeout without telling you.

What is Azure Blob Storage

Have you ever thought how pesky it was to have media library items bulking up your database? What if there was a file server in Azure where you could put them instead and reduce the strain on your CD server! But wait, there is!!

In Azure you can create a Blob Storage account. This is just a bulky file storage (think db backups, large graphics, zipped up Azure templates, container images etc) with an account name you use to reference it. There's an Azure Storage Explorer application to manage your account's files. There's a lot more to it with file states (hot, cold) and setting permission access but it's perfect for putting Sitecore media into.

Issue

After running the Invoke-Migration.ps1 script we noticed the blob storage size wasn't matching our expectations. On closer inspection the migration script would fail without producing an error. We needed the module working but it would require a research spike to get to the bottom of it.

What's Happening?

What the blob storage module is trying to do is make a copy of every media item in the blob storage and intercept media requests to look them up in the blob storage instead. There's more to it than that though here's the lifecycle of the blob storage:

Migration script is run
Media files are copied to blob storage
Blob storage ids are stored on media items
A manual database cleanup task is run from the Sitecore admin tools which removes the media item's blob values from the database
In the event you want to bring a copy of your database back down to your lower environments you need to run the revert script
Blob values are copied back into the database
You can backup and download the database

Underlying Causes

We first discovered there was a failure when we added some logging to the SQL Commands. It indicated that the request timeout was thrown. This meant the migration failed because of the length of time the script would take to run. The master and web database were so large they could never finish migrating before SQL dropped the request. Once we increased the SQL Command timeout we were able to finally move on.

Solution

Ultimately I provided the script to Sitecore Support and they will incorporate the relevant changes in the upcoming versions so you might already have it but if not, here's what I did. Open Invoke-Migration.ps1 and add '$Command.CommandTimeout = 2000' (adjust 2000 as needed) to all SQLCommand objects in the script. Each environment was different but this was sufficient to complete the migrations.