Home » 2012 » November

Monthly Archives: November 2012

Honey, I Shrunk the Content: Experiments with a Custom Sitecore Cache and Compression

A few days ago, I pondered whether there would be any utility in creating a custom Sitecore cache that compresses data before it’s stored and decompresses data upon retrieval. I wondered whether having such a cache would facilitate in conserving memory resources, thus curtailing the need to beef up servers from a memory perspective.

You’re probably thinking “Mike, who cares? Memory is cheaper today than ever before!” That thought is definitely valid.

However, I would argue we owe it to our clients and to ourselves as developers to push the envelope as much as possible by architecting our solutions to be as efficient and resource conscious as possible.

Plus, I was curious over how expensive compress/decompress operations would be for real-time requests.

The following code showcases my ventures into trying to answer these questions.

First, I defined an interface for compressors. Compressors must define methods to compress and decompress data (duh :)). I also added an additional Decompress method to cast the data object to a particular type — all for the purpose of saving client code the trouble of having to do their own type casting (yeah right, I really did it to be fancy).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Base
{
    public interface ICompressor
    {
        string Name { get; }

        byte[] Compress(object uncompressedData);

        T Decompress<T>(byte[] compressedData) where T : class;

        object Decompress(byte[] compressedData);

        long GetDataSize(object data);
    }
}

I decided to use two compression algorithms available in the System.IO.Compression namespace — Deflate and GZip. Since both algorithms within this namespace implement System.IO.Stream, I found an opportunity to use the Template method pattern.

In the spirit of this design pattern, I created an abstract class — see the CompressionStreamCompressor class below — which contains shared logic for compressing/decompressing data using methods defined by the Stream class. Subclasses only have to “fill in the blanks” by implementing the abstract method CreateNewCompressionStream — a method that returns a new instance of the compression stream represented by the subclass.

CompressionStreamCompressor:

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Base
{
    public abstract class CompressionStreamCompressor : ICompressor
    {
        protected CompressionStreamCompressor()
        {
        }

        public virtual byte[] Compress(object uncompressedData)
        {
            Assert.ArgumentNotNull(uncompressedData, "uncompressedData");
            
            byte[] uncompressedBytes = ConvertObjectToBytes(uncompressedData);
            return Compress(uncompressedBytes);
        }

        private byte[] Compress(byte[] uncompressedData)
        {
            Assert.ArgumentNotNull(uncompressedData, "uncompressedData");

            using (MemoryStream memoryStream = new MemoryStream())
            {
                using (Stream compressionStream = CreateNewCompressionStream(memoryStream, CompressionMode.Compress))
                {
                    compressionStream.Write(uncompressedData, 0, uncompressedData.Length);
                }

                return memoryStream.ToArray();
            }
        }

        public virtual T Decompress<T>(byte[] compressedData) where T : class
        {
            object decompressedData = Decompress(compressedData);
            return decompressedData as T;
        }

        public virtual object Decompress(byte[] compressedBytes)
        {
            Assert.ArgumentNotNull(compressedBytes, "compressedBytes");

            using (MemoryStream inputMemoryStream = new MemoryStream(compressedBytes))
            {
                using (Stream compressionStream = CreateNewCompressionStream(inputMemoryStream, CompressionMode.Decompress))
                {
                    using (MemoryStream outputMemoryStream = new MemoryStream())
                    {
                        compressionStream.CopyTo(outputMemoryStream);
                        return ConvertBytesToObject(outputMemoryStream.ToArray());
                    }
                }
            }
        }

        protected abstract Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode);

        public long GetDataSize(object data)
        {
            if (data == null)
                return 0;

            IFormatter formatter = new BinaryFormatter();
            long size = 0;

            using (MemoryStream memoryStream = new MemoryStream())
            {
                formatter.Serialize(memoryStream, data);
                size = memoryStream.Length;
            }

            return size;
        }

        protected static byte[] ConvertObjectToBytes(object data)
        {
            if (data == null)
                return null;

            byte[] bytes = null;
            IFormatter formatter = new BinaryFormatter();

            using (MemoryStream memoryStream = new MemoryStream())
            {
                formatter.Serialize(memoryStream, data);
                bytes = memoryStream.ToArray();
            }
            
            return bytes;
        }

        protected static object ConvertBytesToObject(byte[] bytes)
        {
            if (bytes == null)
                return null;

            object deserialized = null;
            
            using (MemoryStream memoryStream = new MemoryStream(bytes))
            {
                IFormatter formatter = new BinaryFormatter();
                memoryStream.Position = 0;
                deserialized = formatter.Deserialize(memoryStream);
            }

            return deserialized;
        }
    }
}

DeflateCompressor:

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Compression
{
    public class DeflateCompressor : CompressionStreamCompressor
    {
        private DeflateCompressor()
        {
        }

        protected override Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode)
        {
            return new DeflateStream(stream, compressionMode, false);
        }

        public static ICompressor CreateNewCompressor()
        {
            return new DeflateCompressor();
        }
    }
}

GZipCompressor:

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Compression
{
    public class GZipCompressor : CompressionStreamCompressor
    {
        private GZipCompressor()
        {
        }

        protected override Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode)
        {
            return new GZipStream(stream, compressionMode, false);
        }

        public static ICompressor CreateNewCompressor()
        {
            return new GZipCompressor();
        }
    }
}

If Microsoft ever decides to augment their arsenal of compression streams in System.IO.Compression, we could easily add new Compressor classes for these via the template method paradigm above — as long as these new compression streams implement System.IO.Stream.

After implementing the classes above, I decided I needed a “dummy” Compressor — a compressor that does not execute any compression algorithm but implements the ICompressor interface. My reasoning for doing so is to have a default Compressor be returned via a compressor factory (you will see that I created one further down), and also for ascertaining baseline benchmarks.

Plus, I figured it would be nice to have an object that closely follows the Null Object pattern — albeit in our case, we aren’t truly using this design pattern since our “Null” class is actually executing logic — so client code can avoid having null checks all over the place.

I had to go back and change my Compress and Decompress methods to be virtual in my abstract class so that I can override them within my “Null” Compressor class. The methods just take in the expected parameters and return expected types with no compression or decompression actions in the mix.

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors
{
    class NullCompressor : CompressionStreamCompressor
    {
        private NullCompressor()
        {
        }

        public override byte[] Compress(object uncompressedData)
        {
            Assert.ArgumentNotNull(uncompressedData, "uncompressedData");
            return ConvertObjectToBytes(uncompressedData);
        }

        public override object Decompress(byte[] compressedBytes)
        {
            Assert.ArgumentNotNull(compressedBytes, "compressedBytes");
            return ConvertBytesToObject(compressedBytes);
        }

        protected override Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode)
        {
            return null;
        }

        public static ICompressor CreateNewCompressor()
        {
            return new NullCompressor();
        }
    }
}

Next, I defined my custom Sitecore cache with its interface and settings Data transfer object.

An extremely important thing to keep in mind when creating a custom Sitecore cache is knowing you must subclass CustomCache in Sitecore.Caching — most methods that add or get from cache are protected methods, and you won’t have access to these unless you subclass this abstract class (I wasn’t paying attention when I built my cache for the first time, and had to go back to the drawing board when i discovered my code would not compile due to restricted access rights).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Caching;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Caching.DTO
{
    public class SquashedCacheSettings
    {
        public string Name { get; set; }
        public long MaxSize { get; set; }
        public ICompressor Compressor { get; set; }
        public bool CompressionEnabled { get; set; }
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Caching.Base
{
    public interface ISquashedCache
    {
        ICompressor Compressor { get; }

        void AddToCache(object key, object value);

        T GetFromCache<T>(object key) where T : class;

        object GetFromCache(object key);
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Caching;
using Sitecore.Data;
using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Caching.Base;
using Sitecore.Sandbox.Utilities.Caching.DTO;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Caching
{
    public class SquashedCache : CustomCache, ISquashedCache
    {
        private SquashedCacheSettings SquashedCacheSettings { get; set; }

        public ICompressor Compressor
        {
            get
            {
                return SquashedCacheSettings.Compressor;
            }
        }

        private SquashedCache(SquashedCacheSettings squashedCacheSettings)
            : base(squashedCacheSettings.Name, squashedCacheSettings.MaxSize)
        {
            SetSquashedCacheSettings(squashedCacheSettings);
        }

        private void SetSquashedCacheSettings(SquashedCacheSettings squashedCacheSettings)
        {
            SquashedCacheSettings = squashedCacheSettings;
        }

        public void AddToCache(object key, object value)
        {
            DataInformation dataInformation = GetCompressedDataInformation(value);
            SetObject(key, dataInformation.Data, dataInformation.Size);
        }

        private DataInformation GetCompressedDataInformation(object data)
        {
            long size = SquashedCacheSettings.Compressor.GetDataSize(data);
            return GetCompressedDataInformation(data, size);
        }

        private DataInformation GetCompressedDataInformation(object data, long size)
        {
            if (SquashedCacheSettings.CompressionEnabled)
            {
                data = SquashedCacheSettings.Compressor.Compress(data);
                size = SquashedCacheSettings.Compressor.GetDataSize(data);
            }

            return new DataInformation(data, size);
        }

        public T GetFromCache<T>(object key) where T : class
        {
            object value = GetFromCache(key);
            return value as T;
        }

        public object GetFromCache(object key)
        {
            byte[] value = (byte[])GetObject(key);
            return SquashedCacheSettings.Compressor.Decompress(value);
        }

        private T GetDecompressedData<T>(byte[] data) where T : class
        {
            if (SquashedCacheSettings.CompressionEnabled)
            {
                return SquashedCacheSettings.Compressor.Decompress<T>(data);
            }

            return data as T;
        }

        private object GetDecompressedData(byte[] data)
        {
            if (SquashedCacheSettings.CompressionEnabled)
            {
                return SquashedCacheSettings.Compressor.Decompress(data);
            }

            return data;
        }

        private struct DataInformation
        {
            public object Data;
            public long Size;

            public DataInformation(object data, long size)
            {
                Data = data;
                Size = size;
            }
        }

        public static ISquashedCache CreateNewSquashedCache(SquashedCacheSettings squashedCacheSettings)
        {
            AssertSquashedCacheSettings(squashedCacheSettings);
            return new SquashedCache(squashedCacheSettings);
        }

        private static void AssertSquashedCacheSettings(SquashedCacheSettings squashedCacheSettings)
        {
            Assert.ArgumentNotNull(squashedCacheSettings, "squashedCacheSettings");
            Assert.ArgumentNotNullOrEmpty(squashedCacheSettings.Name, "squashedCacheSettings.Name");
            Assert.ArgumentCondition(squashedCacheSettings.MaxSize > 0, "squashedCacheSettings.MaxSize", "MaxSize must be greater than zero.");
            Assert.ArgumentNotNull(squashedCacheSettings.Compressor, "squashedCacheSettings.Compressor");
        }
    }
}

You’re probably thinking “Mike, what’s up with the name SquashedCache”? Well, truth be told, I was thinking about Thanksgiving here in the United States — it’s just around the corner — and how squash is a usually found on the table for Thanksgiving dinner. The name SquashedCache just fit in perfectly in the spirit of our Thanksgiving holiday.

However, the following class names were considered.

public class SirSquishALot
{
}

public class MiniMeCache
{
}

// this became part of this post’s title instead 🙂
public class HoneyIShrunkTheContent
{
}

I figured having a Factory class for compressor objects would offer a clean and central place for creating them.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Enums
{
    public enum CompressorType
    {
        Deflate,
        GZip,
        Null
    } 
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Enums;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Base
{
    public interface ICompressorFactory
    {
        ICompressor CreateNewCompressor(CompressorType compressorType);
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Enums;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors
{
    public class CompressorFactory : ICompressorFactory
    {
        private CompressorFactory()
        {
        }

        public ICompressor CreateNewCompressor(CompressorType compressorType)
        {
            if (compressorType == CompressorType.Deflate)
            {
                return DeflateCompressor.CreateNewCompressor();
            }
            else if (compressorType == CompressorType.GZip)
            {
                return GZipCompressor.CreateNewCompressor();
            }

            return NullCompressor.CreateNewCompressor();
        }

        public static ICompressorFactory CreateNewCompressorFactory()
        {
            return new CompressorFactory();
        }
    }
}

Now, it’s time to test everything above and look at some statistics. I basically created a sublayout containing some repeaters to highlight how each compressor performed against the others — including the “Null” compressor which serves as the baseline.

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="Squash Test.ascx.cs" Inherits="Sitecore650rev120706.layouts.sublayouts.Squash_Test" %>

<div>
    <h2><u>Compression Test</u></h2>
    Uncompressed Size: <asp:Literal ID="litUncompressedSize" runat="server" /> bytes
    <asp:Repeater ID="rptCompressionTest" runat="server">
        <HeaderTemplate>
            <br /><br />
        </HeaderTemplate>
        <ItemTemplate>
                <div>
                    Compressed Size using <%# Eval("TestName")%>: <%# Eval("CompressedSize")%> bytes <br />
                    Compression Ratio using <%# Eval("TestName")%>: <%# Eval("CompressionRatio","{0:p}") %> of original size
                </div>
        </ItemTemplate>
        <SeparatorTemplate>
            <br />
        </SeparatorTemplate>
    </asp:Repeater>
</div>

<asp:Repeater ID="rptAddToCacheTest" runat="server">
    <HeaderTemplate>
        <div>
            <h2><u>AddToCache() Test</u></h2>
    </HeaderTemplate>
    <ItemTemplate>
            <div>
                AddToCache() Elasped Time for <%# Eval("TestName")%>: <%# Eval("ElapsedMilliseconds")%> ms
            </div>
    </ItemTemplate>
    <FooterTemplate>
        </div>
    </FooterTemplate>
</asp:Repeater>

<asp:Repeater ID="rptGetFromCacheTest" runat="server">
    <HeaderTemplate>
        <div>
            <h2><u>GetFromCache() Test</u></h2>
    </HeaderTemplate>
    <ItemTemplate>
            <div>
                GetFromCache() Elasped Time for <%# Eval("TestName")%>: <%# Eval("ElapsedMilliseconds")%> ms
            </div>
    </ItemTemplate>
    <FooterTemplate>
        </div>
    </FooterTemplate>
</asp:Repeater>

<asp:Repeater ID="rptDataIntegrityTest" runat="server">
    <HeaderTemplate>
        <div>
            <h2><u>Data Integrity Test</u></h2>
    </HeaderTemplate>
    <ItemTemplate>
            <div>
                Data Retrieved From <%# Eval("TestName")%> Equals Original: <%# Eval("AreEqual")%>
            </div>
    </ItemTemplate>
    <FooterTemplate>
        </div>
    </FooterTemplate>
</asp:Repeater>

In my code-behind, I’m grabbing the full text of the book War and Peace by Leo Tolstoy for testing purposes. The full text of this copy of the book is over 3.25 MB.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore;

using Sitecore.Sandbox.Utilities.Caching;
using Sitecore.Sandbox.Utilities.Caching.Base;
using Sitecore.Sandbox.Utilities.Caching.DTO;

using Sitecore.Sandbox.Utilities.Compression.Compressors;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Enums;

namespace Sitecore650rev120706.layouts.sublayouts
{
    public partial class Squash_Test : System.Web.UI.UserControl
    {
        private const string CacheKey = "War and Peace";
        private const string WarAndPeaceUrl = "http://www.gutenberg.org/cache/epub/2600/pg2600.txt";
        private const string MaxSize = "50MB";

        private ICompressorFactory _Factory;
        private ICompressorFactory Factory
        {
            get
            {
                if(_Factory == null)
                    _Factory = CompressorFactory.CreateNewCompressorFactory();

                return _Factory;
            }
        }

        private IEnumerable<ISquashedCache> _SquashedCaches;
        private IEnumerable<ISquashedCache> SquashedCaches
        {
            get
            {
                if(_SquashedCaches == null)
                    _SquashedCaches = CreateAllSquashedCaches();

                return _SquashedCaches;
            }
        }

        private string _WarAndPeaceText;
        private string WarAndPeaceText
        {
            get
            {
                if (string.IsNullOrEmpty(_WarAndPeaceText))
                    _WarAndPeaceText = GetWarAndPeaceText();

                return _WarAndPeaceText;
            }
        }

        private long _UncompressedSize;
        private long UncompressedSize
        {
            get
            {
                if(_UncompressedSize == 0)
                    _UncompressedSize = GetUncompressedSize();

                return _UncompressedSize;
            }
        }


        private Stopwatch _Stopwatch;
        private Stopwatch Stopwatch
        {
            get
            {
                if (_Stopwatch == null)
                    _Stopwatch = Stopwatch.StartNew();

                return _Stopwatch;
            }
        }

        protected void Page_Load(object sender, EventArgs e)
        {
            SetUncomopressedSizeLiteral();
            BindAllRepeaters();
        }

        private void SetUncomopressedSizeLiteral()
        {
            litUncompressedSize.Text = UncompressedSize.ToString();
        }

        private void BindAllRepeaters()
        {
            BindCompressionTestRepeater();
            BindAddToCacheTestRepeater();
            BindGetFromCacheTestRepeater();
            BindDataIntegrityTestRepeater();
        }

        private void BindCompressionTestRepeater()
        {
            rptCompressionTest.DataSource = GetCompressionTestData();
            rptCompressionTest.DataBind();
        }

        private IEnumerable<CompressionRatioAtom> GetCompressionTestData()
        {
            IList<CompressionRatioAtom> compressionRatioAtoms = new List<CompressionRatioAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                byte[] compressed = squashedCache.Compressor.Compress(WarAndPeaceText);
                long compressedSize = squashedCache.Compressor.GetDataSize(compressed);

                CompressionRatioAtom compressionRatioAtom = new CompressionRatioAtom
                {
                    TestName = squashedCache.Name,
                    CompressedSize = compressedSize,
                    CompressionRatio = ((decimal)compressedSize / UncompressedSize)
                };

                compressionRatioAtoms.Add(compressionRatioAtom);
            }

            return compressionRatioAtoms;
        }

        private void BindAddToCacheTestRepeater()
        {
            rptAddToCacheTest.DataSource = GetAddToCacheTestData();
            rptAddToCacheTest.DataBind();
        }

        private IEnumerable<TimeTestAtom> GetAddToCacheTestData()
        {
            IList<TimeTestAtom> timeTestAtoms = new List<TimeTestAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                Stopwatch.Start();
                squashedCache.AddToCache(CacheKey, WarAndPeaceText);
                Stopwatch.Stop();

                TimeTestAtom timeTestAtom = new TimeTestAtom
                {
                    TestName = squashedCache.Name,
                    ElapsedMilliseconds = Stopwatch.Elapsed.TotalMilliseconds
                };

                timeTestAtoms.Add(timeTestAtom);
            }

            return timeTestAtoms;
        }

        private void BindGetFromCacheTestRepeater()
        {
            rptGetFromCacheTest.DataSource = GetGetFromCacheTestData();
            rptGetFromCacheTest.DataBind();
        }

        private IEnumerable<TimeTestAtom> GetGetFromCacheTestData()
        {
            IList<TimeTestAtom> timeTestAtoms = new List<TimeTestAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                Stopwatch.Start();
                squashedCache.GetFromCache<string>(CacheKey);
                Stopwatch.Stop();

                TimeTestAtom timeTestAtom = new TimeTestAtom
                {
                    TestName = squashedCache.Name,
                    ElapsedMilliseconds = Stopwatch.Elapsed.TotalMilliseconds
                };

                timeTestAtoms.Add(timeTestAtom);
            }

            return timeTestAtoms;
        }

        private void BindDataIntegrityTestRepeater()
        {
            rptDataIntegrityTest.DataSource = GetDataIntegrityTestData();
            rptDataIntegrityTest.DataBind();
        }

        private IEnumerable<DataIntegrityTestAtom> GetDataIntegrityTestData()
        {
            IList<DataIntegrityTestAtom> dataIntegrityTestAtoms = new List<DataIntegrityTestAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                string cachedContent = squashedCache.GetFromCache<string>(CacheKey);

                DataIntegrityTestAtom dataIntegrityTestAtom = new DataIntegrityTestAtom
                {
                    TestName = squashedCache.Name,
                    AreEqual = cachedContent == WarAndPeaceText
                };

                dataIntegrityTestAtoms.Add(dataIntegrityTestAtom);
            }

            return dataIntegrityTestAtoms;
            
        }

        private IEnumerable<ISquashedCache> CreateAllSquashedCaches()
        {
            IList<ISquashedCache> squashedCaches = new List<ISquashedCache>();
            squashedCaches.Add(CreateNewNullSquashedCache());
            squashedCaches.Add(CreateNewDeflateSquashedCache());
            squashedCaches.Add(CreateNewGZipSquashedCache());
            return squashedCaches;
        }

        private ISquashedCache CreateNewNullSquashedCache()
        {
            return CreateNewSquashedCache("Null Cache", MaxSize, CompressorType.Null);
        }

        private ISquashedCache CreateNewDeflateSquashedCache()
        {
            return CreateNewSquashedCache("Deflate Cache", MaxSize, CompressorType.Deflate);
        }

        private ISquashedCache CreateNewGZipSquashedCache()
        {
            return CreateNewSquashedCache("GZip Cache", MaxSize, CompressorType.GZip);
        }

        private ISquashedCache CreateNewSquashedCache(string cacheName, string maxSize, CompressorType compressorType)
        {
            SquashedCacheSettings squashedCacheSettings = CreateNewSquashedCacheSettings(cacheName, maxSize, compressorType);
            return SquashedCache.CreateNewSquashedCache(squashedCacheSettings);
        }

        private SquashedCacheSettings CreateNewSquashedCacheSettings(string cacheName, string maxSize, CompressorType compressorType)
        {
            return new SquashedCacheSettings
            {
                Name = cacheName,
                MaxSize = StringUtil.ParseSizeString(maxSize),
                Compressor = Factory.CreateNewCompressor(compressorType),
                CompressionEnabled = true
            };
        }

        private static string GetWarAndPeaceText()
        {
            WebRequest webRequest = (HttpWebRequest)WebRequest.Create(WarAndPeaceUrl);
            HttpWebResponse httpWebResponse = (HttpWebResponse)webRequest.GetResponse();
            
            string warAndPeaceText = string.Empty;

            using (StreamReader streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
            {
                warAndPeaceText = streamReader.ReadToEnd();
            }

            return warAndPeaceText;
        }

        private long GetUncompressedSize()
        {
            return SquashedCaches.FirstOrDefault().Compressor.GetDataSize(WarAndPeaceText);
        }

        private class CompressionRatioAtom
        {
            public string TestName { get; set; }
            public long CompressedSize { get; set; }
            public decimal CompressionRatio { get; set; }
        }

        private class TimeTestAtom
        {
            public string TestName { get; set; }
            public double ElapsedMilliseconds { get; set; }
        }

        private class DataIntegrityTestAtom
        {
            public string TestName { get; set; }
            public bool AreEqual { get; set; }
        }
    }
}

From my screenshot below, we can see that both compression algorithms compress War and Peace down to virtually the same ratio of the original size.

Plus, the add operations are quite expensive for the true compressors over the “Null” compressor — GZip yielding the worst performance next to the others.

However, the get operations don’t appear to be that far off from each other — albeit I cannot truly conclude this I am only showing one test outcome here. It would be best to make such assertions after performing load testing to truly ascertain the performance of these algorithms. My ventures here were only to acquire a rough sense of how these algorithms perform.

In the future, I may want to explore how other compression algorithms stack up next to the two above. LZF — a real-time compression algorithm that promises good performance — is one algorithm I’m considering.

I will let you know my results if I take this algorithm for a dry run.

Gobble Gobble!

Get a Handle on UrlHandle

Over the past year or so, I’ve been having random blobs of information bubbling up into my consciousness and grabbing my attention — please don’t worry, these are pretty geeky things, mostly things I’ve encountered within Sitecore.Kernel.dll using .NET Reflector, or things I’ve discussed with other Sitecore developers. These random tidbits usually commandeer my attention during mundane events — examples include vegging out in front of TV, taking a shower, or during my long walks. I don’t mind when this happens since it has the benefit of keeping my mind engaged in something interesting, and perhaps constructive.

The UrlHandle class — a utility class found within the Sitecore.Web namespace in Sitecore.Kernel.dll — is probably the one thing that grabs my attention the most. This class usurps my attention at least once a day — more often multiple times a day — and does so usually in the context of a question that I will ask you at the end of this post. There is no doubt in my mind you’ll have a good answer to my question.

I remember hearing about the UrlHandle class for the first time at Dreamcore 2010 North America during a talk given by Lars Fløe Nielsen. Nielsen, co-founder and Senior VP of Technical Marketing at Sitecore, sold me on using this class around its core function of transferring a huge set of query string data between two pages without being confined to browser imposed limits.

The UrlHandle class is used within the content editor. An example of its usage can be seen within code for the TreelistEx field. The TreelistEx field passes information to its dialog window via the UrlHandle class.

Below, I created two sublayouts for the purpose of showing you how the UrlHandle class works, and how you may use it in your own solutions.

Sublayout on the originating page:

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="UrlHandle Origin.ascx.cs" Inherits="Sitecore650rev120706.layouts.sublayouts.UrlHandle_Origin" %>

<h2>Querystring:</h2>
<asp:Literal ID="litQueryStringForGettingAHandle" runat="server" /><br /><br />
<asp:Button ID="btnGotoDestinationPage" OnClick="btnGotoDestinationPage_Click" Text="Goto Destination Page" runat="server" />
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore.Text;
using Sitecore.Web;

namespace Sitecore650rev120706.layouts.sublayouts
{
    public partial class UrlHandle_Origin : System.Web.UI.UserControl
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            if (!IsPostBack)
            {
                SetQueryStringLiteral();
            }
        }

        protected void btnGotoDestinationPage_Click(object sender, EventArgs e)
        {
            AddToUrlHandleAndRedirect();
        }

        private string GetBigQueryString()
        {
            const char startLetter = 'a';
            const string queryStringParamFormat = "{0}={1}";
            Random random = new Random();

            IList<string> stringBuffer = new List<string>();

            for (int i = 0; i < 26; i++)
            {
                int ascii = i + (int)startLetter;
                char letter = (char)ascii;
                string queryStringParam = string.Format(queryStringParamFormat, letter.ToString(), random.Next(1000000000));
                stringBuffer.Add(queryStringParam);
            }

            string queryString = string.Join("&", stringBuffer);
            return string.Concat("?", queryString);
        }

        private void SetQueryStringLiteral()
        {
            litQueryStringForGettingAHandle.Text = GetBigQueryString();
        }

        private void AddToUrlHandleAndRedirect()
        {
            UrlHandle urlHandle = new UrlHandle();
            urlHandle["My Big Query String"] = litQueryStringForGettingAHandle.Text;

            UrlString urlString = new UrlString("/urlhandle-destination.aspx");
            urlHandle.Add(urlString);

            Response.Redirect(urlString.ToString());
        }
    }
}

The originating page’s sublayout above generates a querystring using all letters in the English alphabet coupled with random integers. These are placed within an instance of the UrlHandle class, with a key of the destination page’s url via the UrlString class.

If you are unfamiliar with the UrlString class, Jimmi Lyhne Andersen wrote a good blog post on using the UrlString class. I recommend that you check it out.

Output of the originating page:

Sublayout on the destination page:

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="UrlHandle Destination.ascx.cs" Inherits="Sitecore650rev120706.layouts.sublayouts.UrlHandle_Destination" %>

<h2>Querystring in UrlHandle:</h2>
<asp:Literal ID="litQueryStringFromHandle" runat="server" />
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore.Web;

namespace Sitecore650rev120706.layouts.sublayouts
{
    public partial class UrlHandle_Destination : System.Web.UI.UserControl
    {
        private UrlHandle _UrlHandle;
        private UrlHandle UrlHandle
        {
            get
            {
                if (_UrlHandle == null)
                    _UrlHandle = UrlHandle.Get();

                return _UrlHandle;
            }
        }

        protected void Page_Load(object sender, EventArgs e)
        {
            SetLiterals();
        }

        private void SetLiterals()
        {
            litQueryStringFromHandle.Text = UrlHandle["My Big Query String"];
        }
    }
}

In the code-behind of the destination page’s sublayout, we use the parameterless static Get() method on the UrlHandle class to get a UrlHandle instance associated with the current url, and it uses “hdl” as the default handle name. Please be aware that an exception will be thrown if a url passed to this method is not associated with a UrlHandle object — or the current url if we are using the parameterless method. The Get method has overloads for passing in different parameters — one being a UrlString instance, and another being the name of the handle key. You have the ability to override the default handle name of “hdl” if you desire.

The handle value itself is a ShortID as the following screenshot highlights. Basically, this ShortID conjoined with the url of the destination page serve a unique key into the UrlHandle object for retrieving saved information.

Output of the destination page:

Once you pull information out of the UrlHandle class, it is removed from the UrlHandle’s repository of saved information. There is a way to override this default behavior in the object’s Get method by passing a boolean parameter.

So, you may be asking yourself how this class works behind the scenes. Well, it’s quite simple, and even I was surprised to learn how it really worked once I took a peek:

I thought there was some kind of magic happening under the hood but had a reality check after seeing how it really work — this ultimately reminded me of the KISS principle.

Everyday, I ponder and vacillate on whether it would be a good idea to create another class similar to the UrlHandle that would persist across sessions. At this point, I am convinced there is no utility in building such a class. What would be your argument for or against building such a class?

Kicking It Into Overdrive With NVelocity

A few weeks ago, I was tasked with improving one of our Healthcare Specific Website Modules, and felt compelled to share how I used NVelocity to replace tokens set in a Rich Text field — $name and $date are examples of Sitecore tokens. But, before I dive into some code, I will briefly touch upon what NVelocity is.

What exactly is NVelocity? NVelocity is a .NET template engine for replacing string tokens with code references. In other words, NVelocity is a framework that will replace tokens with values returned from code snippets given to the template engine.

There a many examples of using NVelocity in Sitecore across the web. Sitecore provides one such example in this article. This article — although a bit dated since it was written for Sitecore v5.1 — provides code examples that could still be used today.

Alistair Deneys also wrote a good article about NVelocity here. Deneys posits Sitecore no longer uses NVelocity for token replacement. However, three years after Deneys’ penned his article, code within Sitecore.Kernel still references it. Here is an example of it being used within Sitecore.Kernel (v6.5.0 rev. 111123):

Sitecore has created its own API around NVelocity. Sitecore’s wrapper API is defined within Sitecore.NVelocity.dll. In order to use the following code, you must reference this assembly.

Now, let’s get our hands dirty with some code.

First, I created an adapter to encapsulate Sitecore’s static Velocity engine object for doing token replacements.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

using NVelocity.Context;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public interface ITokenContextEvaluator
    {
        bool Evaluate(IContext context, TextWriter writer, string logTag, Stream instream);

        bool Evaluate(IContext context, TextWriter out_Renamed, string logTag, string instring);

        bool Evaluate(IContext context, TextWriter writer, string logTag, TextReader reader);
    }
}
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

using NVelocity.App;
using NVelocity.Context;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class TokenContextEvaluator : ITokenContextEvaluator
    {
        private TokenContextEvaluator()
        {
            Initailize();
        }

        private void Initailize()
        {
            Velocity.Init();
        }

        public bool Evaluate(IContext context, TextWriter writer, string logTag, Stream instream)
        {
            return Velocity.Evaluate(context, writer, logTag, instream);
        }

        public bool Evaluate(IContext context, TextWriter out_Renamed, string logTag, string instring)
        {
            return Velocity.Evaluate(context, out_Renamed, logTag, instring);
        }

        public bool Evaluate(IContext context, TextWriter writer, string logTag, TextReader reader)
        {
            return Velocity.Evaluate(context, writer, logTag, reader);
        }

        public static ITokenContextEvaluator CreateNewTokenContextEvaluator()
        {
            return new TokenContextEvaluator();
        }
    }
}

Next, I created a class that defines an one-to-one mapping between a string token and an object reference — the value of the code given to the template engine.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;

namespace Sitecore.Sandbox.Utilities.StringUtilities.DTO
{
    public class TokenKeyValue
    {
        public string Key { get; private set; }
        public object Value { get; private set; }

        public TokenKeyValue(string key, object value)
        {
            SetKey(key);
            SetValue(value);
        }

        private void SetKey(string key)
        {
            Assert.ArgumentNotNullOrEmpty(key, "key");
            Key = key;
        }

        private void SetValue(object value)
        {
            Value = value;
        }
    }
}

Then I created a Data transfer object to pass an instance of the adapter defined above, a collection of token mappings and an instance of NVelocity’s IContext to the main Tokenator class below — all this being done this way to allow for dependency injection.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using NVelocity.Context;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities.DTO
{
    public class TokenatorSettings
    {
        public ITokenContextEvaluator TokenContextEvaluator { get; set; }

        public IContext TokenContext { get; set; }

        public IEnumerable<TokenKeyValue> TokenKeyValues { get; set; }

        public TokenatorSettings()
        {
        }
    }
}

With all the pieces above floating around, it’s now time to glue them all together in the main Tokenator class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public interface ITokenator
    {
        string ReplaceTokens(string value);
    }
}
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;
using Sitecore.Text.NVelocity;

using NVelocity;
using NVelocity.App;
using NVelocity.Context;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class Tokenator : ITokenator
    {
        private const string LogName = "Tokenator Replacement Action";

        private TokenatorSettings TokenatorSettings { get; set; }

        private Tokenator(IEnumerable<TokenKeyValue> tokenKeyValues)
            : this(CreateNewTokenatorSettingsWithDefaults(tokenKeyValues))
        {
        }

        private Tokenator(TokenatorSettings tokenatorSettings)
        {
            SetTokenatorSettings(tokenatorSettings);
            Initialize();
        }

        private void SetTokenatorSettings(TokenatorSettings tokenatorSettings)
        {
            AssertTokenatorSettings(tokenatorSettings);
            TokenatorSettings = tokenatorSettings;
        }

        private void AssertTokenatorSettings(TokenatorSettings tokenatorSettings)
        {
            Assert.ArgumentNotNull(tokenatorSettings, "tokenatorSettings");
            Assert.ArgumentNotNull(tokenatorSettings.TokenContextEvaluator, "tokenatorSettings.TokenContextEvaluator");
            Assert.ArgumentNotNull(tokenatorSettings.TokenContext, "tokenatorSettings.TokenContext");
            Assert.ArgumentNotNull(tokenatorSettings.TokenKeyValues, "tokenatorSettings.TokenKeyValues");
        }

        private void Initialize()
        {
            foreach (TokenKeyValue tokenKeyValue in TokenatorSettings.TokenKeyValues)
            {
                TokenatorSettings.TokenContext.Put(tokenKeyValue.Key, tokenKeyValue.Value);
            }
        }

        public string ReplaceTokens(string value)
        {
            string tokensReplaced = string.Empty;

            using (StringWriter stringWriter = new StringWriter())
            {
                TokenatorSettings.TokenContextEvaluator.Evaluate(TokenatorSettings.TokenContext, stringWriter, LogName, value);
                tokensReplaced = stringWriter.ToString();
            }

            return tokensReplaced;
        }

        private void LogError(Exception exception)
        {
            Log.Error(this.ToString(), exception, this);
        }

        private static TokenatorSettings CreateNewTokenatorSettingsWithDefaults(IEnumerable<TokenKeyValue> tokenKeyValues)
        {
            return new TokenatorSettings
            {
                TokenContextEvaluator = GetDefaultTokenContextEvaluator(),
                TokenContext = GetDefaultTokenContext(),
                TokenKeyValues = tokenKeyValues
            };
        }

        private static ITokenContextEvaluator GetDefaultTokenContextEvaluator()
        {
            return StringUtilities.TokenContextEvaluator.CreateNewTokenContextEvaluator();
        }

        private static IContext GetDefaultTokenContext()
        {
            return new VelocityContext();
        }

        public static ITokenator CreateNewTokenator(IEnumerable<TokenKeyValue> tokenKeyValues)
        {
            return new Tokenator(tokenKeyValues);
        }

        public static ITokenator CreateNewTokenator(TokenatorSettings tokenatorSettings)
        {
            return new Tokenator(tokenatorSettings);
        }
    }
}

The following sublayout code was used as a test harness for my Tokenator object, and to illustrate how client code would use it.

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="Tokenator Test.ascx.cs" Inherits="Sitecore650rev120706.layouts.sublayouts.Tokenator_Test" %>
<asp:Literal ID="litTokenatorTest" runat="server" />
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore.Sandbox.Utilities.StringUtilities;
using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore650rev120706.layouts.sublayouts
{
    public partial class Tokenator_Test : System.Web.UI.UserControl
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            SetLiteralText();
        }

        private void SetLiteralText()
        {
            litTokenatorTest.Text = GetTokenReplacedContent();
        }

        private string GetTokenReplacedContent()
        {
            string content = Sitecore.Context.Item["Text"];
            return ReplaceTokens(content);
        }

        private string ReplaceTokens(string content)
        {
            IEnumerable<TokenKeyValue> tokenKeyValues = GetTokenKeyValues();
            ITokenator tokenator = Tokenator.CreateNewTokenator(tokenKeyValues);
            return tokenator.ReplaceTokens(content);
        }

        private IEnumerable<TokenKeyValue> GetTokenKeyValues()
        {
            IList<TokenKeyValue> tokenKeyValues = new List<TokenKeyValue>();
            tokenKeyValues.Add(new TokenKeyValue("localtime", DateTime.Now));
            tokenKeyValues.Add(new TokenKeyValue("developer", "@mike_i_reynolds"));
            tokenKeyValues.Add(new TokenKeyValue("random", new Random().Next(10000)));
            return tokenKeyValues;
        }
    }
}

Token content in a Rich Text field:

Test output:

NVelocity is definitely a great tool to tap into and harness — especially to make content more dynamic.

However, if there is one takeaway I would like for you the reader to part with, it would be this: stay relentlessly curious, and keep on digging through the treasure troves in Sitecore’s assemblies. This article would not have been possible if I did not have a copy of .NET Reflector, Sitecore.Kernel.dll, Sitecore.NVelocity.dll and an unquenchable thirst for learning.

Happy coding!