Home » Utilities (Page 2)

Category Archives: Utilities

Empower Your Content Authors to Expand Standard Values Tokens in the Sitecore Client

Have you ever seen a Standard Values token — $name is an example of a Standard Values token — in an item’s field, and ask yourself “how the world did that get there”, or alternatively, “what can be done to replace it with what should be there?”

This can occur when tokens are added to an item’s template’s Standard Values node after the item was created — tokens are expanded once an item is created, not after the fact.

John West wrote a blog article highlighting one solution for eradicating this issue by using the Sitecore Rules Engine.

In this post, I am proposing a completely different solution — one that empowers content authors to expand unexpanded tokens by clicking a link in a custom content editor warning box.

First, I created a series of utility objects that ascertain whether “Source” objects contain substrings that we are interested in. All implement the following interface:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public interface ISubstringsChecker<T>
    {
        T Source { get; set; }
        
        IEnumerable<string> Substrings { get; set; }

        bool ContainsSubstrings();
    }
}

All of our “checkers” will have a “Source” object, a collection of substrings to look for, and a method to convey to calling code whether a substring in the collection of substrings had been found in the “Source” object.

I decided to create a base abstract class with some abstract methods along with one method to serve as a hook for asserting the Source object — albeit I did not use this method anywhere in my solution (come on Mike, you’re forgetting YAGNI — let’s get with the program).

Utimately, I am employing the template method pattern — all subclasses of this abstract base class will just fill in the defined abstract stubs, and the parent class will take care of the rest:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public abstract class SubstringsChecker<T> : ISubstringsChecker<T>
    {
        public T Source { get; set; }

        public virtual IEnumerable<string> Substrings { get; set; }

        protected SubstringsChecker(T source)
        {
            SetSource(source);
        }

        private void SetSource(T source)
        {
            AssertSource(source);
            Source = source;
        }

        protected virtual void AssertSource(T source)
        {
            // a hook for subclasses to assert Source objects
        }

        public bool ContainsSubstrings()
        {
            if (CanDoCheck())
            {
                return DoCheck();
            }

            return false;
        }

        protected abstract bool CanDoCheck();
        protected abstract bool DoCheck();
    }
}

The first “checker” I created will find substrings in a string. It made sense for me to start here, especially when we are ultimately checking strings at the most atomic level in our series of “checker” utility objects.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class StringSubstringsChecker : SubstringsChecker<string>
    {
        private StringSubstringsChecker(IEnumerable<string> substrings)
            : this(null, substrings)
        {
        }

        private StringSubstringsChecker(string source, IEnumerable<string> substrings)
            : base(source)
        {
            SetSubstrings(substrings);
        }

        private void SetSubstrings(IEnumerable<string> substrings)
        {
            AssertSubstrings(substrings);
            Substrings = substrings;
        }

        private static void AssertSubstrings(IEnumerable<string> substrings)
        {
            Assert.ArgumentNotNull(substrings, "substrings");
            Assert.ArgumentCondition(substrings.Any(), "substrings", "substrings must contain as at least one string!");
        }

        protected override bool CanDoCheck()
        {
            return !string.IsNullOrEmpty(Source);
        }

        protected override bool DoCheck()
        {
            Assert.ArgumentNotNullOrEmpty(Source, "Source");

            foreach (string substring in Substrings)
            {
                if (Source.Contains(substring))
                {
                    return true;
                }
            }

            return false;
        }

        public static ISubstringsChecker<string> CreateNewStringSubstringsContainer(IEnumerable<string> substrings)
        {
            return new StringSubstringsChecker(substrings);
        }

        public static ISubstringsChecker<string> CreateNewStringSubstringsContainer(string source, IEnumerable<string> substrings)
        {
            return new StringSubstringsChecker(substrings);
        }
    }
}

The next level up from strings would naturally be Fields. I designed this “checker” to consume an instance of the string “checker” defined above — all for the purposes of reuse. The field “checker” delegates calls to its string “checker” instance.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Data.Fields;
using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class FieldSubstringsChecker : SubstringsChecker<Field>
    {
        private ISubstringsChecker<string> StringSubstringsChecker { get; set; }

        public override IEnumerable<string> Substrings 
        {
            get
            {
                return StringSubstringsChecker.Substrings;
            }
            set
            {
                StringSubstringsChecker.Substrings = value;
            }
        }

        private FieldSubstringsChecker(ISubstringsChecker<string> stringSubstringsChecker)
            : this(null, stringSubstringsChecker)
        {
        }

        private FieldSubstringsChecker(Field source, ISubstringsChecker<string> stringSubstringsChecker)
            : base(source)
        {
            SetStringSubstringsChecker(stringSubstringsChecker);
        }

        private void SetStringSubstringsChecker(ISubstringsChecker<string> stringSubstringsChecker)
        {
            Assert.ArgumentNotNull(stringSubstringsChecker, "stringSubstringsChecker");
            StringSubstringsChecker = stringSubstringsChecker;
        }

        protected override bool CanDoCheck()
        {
            return Source != null;
        }

        protected override bool DoCheck()
        {
            Assert.ArgumentNotNull(Source, "Source");
            StringSubstringsChecker.Source = Source.Value;
            return StringSubstringsChecker.ContainsSubstrings();
        }

        public static ISubstringsChecker<Field> CreateNewFieldSubstringsChecker(ISubstringsChecker<string> stringSubstringsChecker)
        {
            return new FieldSubstringsChecker(stringSubstringsChecker);
        }

        public static ISubstringsChecker<Field> CreateNewFieldSubstringsChecker(Field source, ISubstringsChecker<string> stringSubstringsChecker)
        {
            return new FieldSubstringsChecker(source, stringSubstringsChecker);
        }
    }
}

If I were to ask you what the next level up in our series of “checkers” would be, I hope your answer would be Items. Below is a “checker” that uses an Item as its “Source” object, and delegates calls to an instance of a field “checker”.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class ItemSubstringsChecker : SubstringsChecker<Item>
    {
        private ISubstringsChecker<Field> FieldSubstringsChecker { get; set; }

        public override IEnumerable<string> Substrings 
        {
            get
            {
                return FieldSubstringsChecker.Substrings;
            }
            set
            {
                FieldSubstringsChecker.Substrings = value;
            }
        }

        private ItemSubstringsChecker(ISubstringsChecker<Field> fieldSubstringsChecker)
            : this(null, fieldSubstringsChecker)
        {
        }

        private ItemSubstringsChecker(Item source, ISubstringsChecker<Field> fieldSubstringsChecker)
            : base(source)
        {
            SetFieldSubstringsChecker(fieldSubstringsChecker);
        }

        private void SetFieldSubstringsChecker(ISubstringsChecker<Field> fieldSubstringsChecker)
        {
            Assert.ArgumentNotNull(fieldSubstringsChecker, "fieldSubstringsChecker");
            FieldSubstringsChecker = fieldSubstringsChecker;
        }

        protected override bool CanDoCheck()
        {
            return Source != null && Source.Fields != null && Source.Fields.Any();
        }

        protected override bool DoCheck()
        {
            Assert.ArgumentNotNull(Source, "Source");
            bool containsSubstrings = false;

            for (int i = 0; !containsSubstrings && i < Source.Fields.Count; i++)
            {
                containsSubstrings = containsSubstrings || DoesFieldContainSubstrings(Source.Fields[i]);
            }

            return containsSubstrings;
        }

        private bool DoesFieldContainSubstrings(Field field)
        {
            FieldSubstringsChecker.Source = field;
            return FieldSubstringsChecker.ContainsSubstrings();
        }

        public static ISubstringsChecker<Item> CreateNewItemSubstringsChecker(ISubstringsChecker<Field> fieldSubstringsChecker)
        {
            return new ItemSubstringsChecker(fieldSubstringsChecker);
        }

        public static ISubstringsChecker<Item> CreateNewItemSubstringsChecker(Item source, ISubstringsChecker<Field> fieldSubstringsChecker)
        {
            return new ItemSubstringsChecker(source, fieldSubstringsChecker);
        }
    }
}

Next, I wrote code for a content editor warning box. I’m am completely indebted to .NET Reflector in helping out on this front — I looked at other content editor warning pipelines in Sitecore.Pipelines.GetContentEditorWarnings within Sitecore.Kernel to see how this is done:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Configuration;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Pipelines.GetContentEditorWarnings;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities;

namespace Sitecore.Sandbox.Pipelines.GetContentEditorWarnings
{
    public class HasUnexpandedTokens
    {
        private static readonly ISubstringsChecker<Item> SubstringsChecker = CreateNewItemSubstringsChecker();

        public void Process(GetContentEditorWarningsArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if (CanExpandTokens(args.Item))
            {
                AddHasUnexpandedTokensWarning(args);
            }
        }

        private static bool CanExpandTokens(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            return !IsStandardValues(item) && DoesItemContainUnexpandedTokens(item);
        }

        private static bool DoesItemContainUnexpandedTokens(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            SubstringsChecker.Source = item;
            return SubstringsChecker.ContainsSubstrings();
        }

        private static bool IsStandardValues(Item item)
        {
            return item.Template.StandardValues.ID == item.ID;
        }

        private static void AddHasUnexpandedTokensWarning(GetContentEditorWarningsArgs args)
        {
            GetContentEditorWarningsArgs.ContentEditorWarning warning = args.Add();
            warning.Title = Translate.Text("Some fields contain unexpanded tokens.");
            warning.Text = Translate.Text("To expand tokens, click Expand Tokens.");
            warning.AddOption(Translate.Text("Expand Tokens"), "item:expandtokens");
        }

        private static ISubstringsChecker<Item> CreateNewItemSubstringsChecker()
        {
            return ItemSubstringsChecker.CreateNewItemSubstringsChecker(CreateNewFieldSubstringsChecker());
        }

        private static ISubstringsChecker<Field> CreateNewFieldSubstringsChecker()
        {
            return FieldSubstringsChecker.CreateNewFieldSubstringsChecker(CreateNewStringSubstringsChecker());
        }

        private static ISubstringsChecker<string> CreateNewStringSubstringsChecker()
        {
            return StringSubstringsChecker.CreateNewStringSubstringsContainer(GetTokens());
        }

        private static IEnumerable<string> GetTokens()
        {
            return Factory.GetStringSet("tokens/token");
        }
    }
}

In my pipeline above, I am pulling a collection of token names from Sitecore configuration — these are going to be defined in a patch include file below. I had to go down this road since these tokens are not publically exposed in the Sitecore API.

Plus, we should only allow for the expansion of tokens when not on the Standard values item — it wouldn’t make much sense to expand these tokens here.

Since I’m referencing a new command in the above pipeline — a command that I’ve named “item:expandtokens” — it’s now time to create that new command:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Shell.Framework.Commands;

using Sitecore.Sandbox.Utilities.StringUtilities;
using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore.Sandbox.Commands
{
    public class ExpandTokens : Command
    {
        public override void Execute(CommandContext commandContext)
        {
            if (!DoesCommandContextContainOneItem(commandContext))
            {
                return;
            }

            ExpandTokensInItem(GetCommandContextItem(commandContext));
        }

        private static void ExpandTokensInItem(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            item.Fields.ReadAll();
            item.Editing.BeginEdit();
            ExpandTokensViaMasterVariablesReplacer(item);
            item.Editing.EndEdit();
        }

        private static void ExpandTokensViaMasterVariablesReplacer(Item item)
        {
            MasterVariablesReplacer masterVariablesReplacer = Factory.GetMasterVariablesReplacer();
            masterVariablesReplacer.ReplaceItem(item);
        }

        public override CommandState QueryState(CommandContext commandContext)
        {
            if (ShouldHideCommand(commandContext))
            {
                return CommandState.Hidden;
            }
            
            if (ShouldDisableCommand(commandContext))
            {
                return CommandState.Disabled;
            }

            return base.QueryState(commandContext);
        }

        private bool ShouldHideCommand(CommandContext commandContext)
        {
            Assert.ArgumentNotNull(commandContext, "commandContext");
            return !DoesCommandContextContainOneItem(commandContext) 
                    || !HasField(GetCommandContextItem(commandContext), FieldIDs.ReadOnly);
        }

        private static bool ShouldDisableCommand(CommandContext commandContext)
        {
            AssertCommandContextItems(commandContext);
            Item item = GetCommandContextItem(commandContext);

            return !item.Access.CanWrite()
                    || Command.IsLockedByOther(item)
                    || !Command.CanWriteField(item, FieldIDs.ReadOnly);
        }

        private static Item GetCommandContextItem(CommandContext commandContext)
        {
            Assert.ArgumentNotNull(commandContext, "commandContext");
            Assert.ArgumentNotNull(commandContext.Items, "commandContext.Items");
            return commandContext.Items.FirstOrDefault();
        }

        private static bool DoesCommandContextContainOneItem(CommandContext commandContext)
        {
            AssertCommandContextItems(commandContext);
            return commandContext.Items.Length == 1;
        }

        private static void AssertCommandContextItems(CommandContext commandContext)
        {
            Assert.ArgumentNotNull(commandContext, "commandContext");
            Assert.ArgumentNotNull(commandContext.Items, "commandContext.Items");
        }
    }
}

This new command just uses an instance of Sitecore.Data.MasterVariablesReplacer to expand Standard Values tokens on our item.

I had to register this command in the /App_Config/Commands.config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
	<! -- A bunch of commands defined here -->
	
	<command name="item:expandtokens" type="Sitecore.Sandbox.Commands.ExpandTokens,Sitecore.Sandbox" />
	
	<! -- A bunch more defined here too -->
</configuration>

I then put all the pieces together using a patch include config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <getContentEditorWarnings>
        <processor type="Sitecore.Sandbox.Pipelines.GetContentEditorWarnings.HasUnexpandedTokens, Sitecore.Sandbox"/>
      </getContentEditorWarnings>
    </pipelines>
    <tokens>
      <token>$name</token>
      <token>$id</token>
      <token>$parentid</token>
      <token>$parentname</token>
      <token>$date</token>
      <token>$time</token>
      <token>$now</token>
    </tokens>
  </sitecore>
</configuration>

Let’s create an item for testing:

expand-tokens-new-item

As you can see, the item’s Title field was populated automatically during item creation — the $name token lives on this item’s template’s Standard Values item in Sitecore and was expanded when we created our test item.

Let’s create the problem we are trying to solve by adding new tokens to our test item’s template’s Standard Values node:

expand-tokens-new-tokens

As you can see, these new tokens appear in fields in our test item. However, don’t fret — we now have a way to fix this issue. ๐Ÿ™‚

expand-tokens-content-editor-warning

I clicked on the ‘Expand Tokens’ link, and saw the following:

expand-tokens-expanded-no-warning

Hopefully, this post has given you another weapon to add to your arsenal for solving the unexpanded tokens issue on existing items in Sitecore.

If you find another solution, please drop me a line — I would love to hear about it.

Dude, Where’s My Processor? Filling the Void in the SaveRichTextContent and LoadRichTextContent Sitecore Pipelines

Some of you might be aware that I frequently go through the Web.config of my local instance of Sitecore looking for opportunities to extend or customize class files referenced within it — I may have mentioned this in a previous post, and no doubt have told some Sitecore developers/enthusiasts in person I do this at least once per day. I must confess: I usually do this multiple times a day.

Last night, I was driven to explore something I have noticed in the Web.config of my v6.5 instance — my attention has been usurped many times by the saveRichTextContent and loadRichTextContent pipeline nodes being empty.

These two pipelines allow you to make changes to content within Rich Text fields before any save actions on your item in the Sitecore client.

I remembered that one of them did have a pipeline processor defined within it at one point. It was time to do some research.

After conducting some research — truth be told, I only googled a couple of times — I stumbled upon some release notes on SDN discussing the saveRichTextContent Web.config pipeline, and that this pipeline did contain a processor in it at one point — the Sitecore.Shell.Controls.RichTextEditor.Pipelines.SaveRichTextContent.EmbedInParagraph processor — although I don’t remember what this processor did, and don’t have an older version of Sitecore.Client.dll to investigate. I could download an older version of Sitecore from SDN, but decided to leave that exercise for another snowy weekend.

I decided to explore whether the option to add custom processors to these pipelines still existed. I came up with an idea straight out of the 1990’s — having marquee tags animate content across my pages.

As an aside, back in the 1990’s, almost every webpage — all webpages were called homepages then — had at least one marquee. Most had multiple — it was the cool thing to do back then, asymptotic only to having an ‘Under Construction’ image on your homepage. Employing this practice today would be considered anathema.

I decided to reuse my concept of manipulator from my Manipulate Field Values in a Custom Sitecore Web Forms for Marketers DataProvider article, and created a new manipulator to wrap specified tags in marquee tags:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Manipulators.Base
{
    public interface IWrapHtmlTagsInTagManipulator : IManipulator<string>
    {
    }
}

I thought it would be a good idea to define a DTO for my manipulator to pass objects to it in a clean manner:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Manipulators.DTO
{
    public class WrapHtmlTagsInTagManipulatorSettings
    {
        public string WrapperTag { get; set; }
        public IEnumerable<string> TagsToWrap { get; set; }
    }
}

Next, I built my manipulator:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Manipulators.Base;
using Sitecore.Sandbox.Utilities.Manipulators.DTO;

using HtmlAgilityPack;

namespace Sitecore.Sandbox.Utilities.Manipulators
{
    public class WrapHtmlTagsInTagManipulator : IWrapHtmlTagsInTagManipulator
    {
        private WrapHtmlTagsInTagManipulatorSettings Settings { get; set; }

        private WrapHtmlTagsInTagManipulator(WrapHtmlTagsInTagManipulatorSettings settings)
        {
            SetSettings(settings);
        }

        private void SetSettings(WrapHtmlTagsInTagManipulatorSettings settings)
        {
            AssertSettings(settings);
            Settings = settings;
        }

        private static void AssertSettings(WrapHtmlTagsInTagManipulatorSettings settings)
        {
            Assert.ArgumentNotNull(settings, "settings");
            Assert.ArgumentNotNullOrEmpty(settings.WrapperTag, "settings.WrapperTag");
            Assert.ArgumentNotNull(settings.TagsToWrap, "settings.TagsToWrap");
        }

        public string Manipulate(string html)
        {
            Assert.ArgumentNotNullOrEmpty(html, "html");
            HtmlNode documentNode = GetHtmlDocumentNode(html);

            foreach (string tagToWrap in Settings.TagsToWrap)
            {
                WrapTags(documentNode, tagToWrap);
            }

            return documentNode.InnerHtml;
        }

        private void WrapTags(HtmlNode documentNode, string tagToWrap)
        {
            HtmlNodeCollection htmlNodes = documentNode.SelectNodes(CreateNewDescendantsSelector(tagToWrap));

            foreach(HtmlNode htmlNode in htmlNodes)
            {
                WrapHtmlNodeIfApplicable(documentNode, htmlNode);
            }
        }

        private void WrapHtmlNodeIfApplicable(HtmlNode documentNode, HtmlNode htmlNode)
        {
            if (!AreEqualIgnoreCase(htmlNode.ParentNode.Name, Settings.WrapperTag))
            {
                WrapHtmlNode(documentNode, htmlNode, Settings.WrapperTag);
            }
        }

        private static void WrapHtmlNode(HtmlNode documentNode, HtmlNode htmlNode, string wrapperTag)
        {
            HtmlNode wrapperHtmlNode = documentNode.OwnerDocument.CreateElement(wrapperTag);
            AddNewParent(wrapperHtmlNode, htmlNode);
        }

        private static void AddNewParent(HtmlNode newParentHtmlNode, HtmlNode htmlNode)
        {
            Assert.ArgumentNotNull(newParentHtmlNode, "newParentHtmlNode");
            Assert.ArgumentNotNull(htmlNode, "htmlNode");
            htmlNode.ParentNode.ReplaceChild(newParentHtmlNode, htmlNode);
            newParentHtmlNode.AppendChild(htmlNode);
        }

        private static bool AreEqualIgnoreCase(string stringOne, string stringTwo)
        {
            return string.Equals(stringOne, stringTwo, StringComparison.InvariantCultureIgnoreCase);
        }

        private static string CreateNewDescendantsSelector(string tag)
        {
            Assert.ArgumentNotNullOrEmpty(tag, "tag");
            return string.Format("//{0}", tag);
        }

        private HtmlNode GetHtmlDocumentNode(string html)
        {
            HtmlDocument htmlDocument = CreateNewHtmlDocument(html);
            return htmlDocument.DocumentNode;
        }

        private HtmlDocument CreateNewHtmlDocument(string html)
        {
            HtmlDocument htmlDocument = new HtmlDocument();
            htmlDocument.LoadHtml(html);
            return htmlDocument;
        }

        public static IWrapHtmlTagsInTagManipulator CreateNewWrapHtmlTagsInTagManipulator(WrapHtmlTagsInTagManipulatorSettings settings)
        {
            return new WrapHtmlTagsInTagManipulator(settings);
        }
    }
}

My manipulator class above uses Html Agility Pack to find targeted html elements, and wrap them in newly created marquee tags — which are also created via Html Agility Pack.

I decided to create a base class to contain core logic that will be used across both of my pipeline processors:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Manipulators;
using Sitecore.Sandbox.Utilities.Manipulators.Base;
using Sitecore.Sandbox.Utilities.Manipulators.DTO;

namespace Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.Base
{
    public abstract class AddSomeMarqueesBase
    {
        private IWrapHtmlTagsInTagManipulator _HtmlManipulator;
        private IWrapHtmlTagsInTagManipulator HtmlManipulator
        {
            get
            {
                if(_HtmlManipulator == null)
                {
                    _HtmlManipulator = CreateNewWrapHtmlTagsInTagManipulator();
                }

                return _HtmlManipulator;
            }
        }

        private IWrapHtmlTagsInTagManipulator CreateNewWrapHtmlTagsInTagManipulator()
        {
            return WrapHtmlTagsInTagManipulator.CreateNewWrapHtmlTagsInTagManipulator(CreateNewWrapHtmlTagsInTagManipulatorSettings());
        }

        protected virtual WrapHtmlTagsInTagManipulatorSettings CreateNewWrapHtmlTagsInTagManipulatorSettings()
        {
            return new WrapHtmlTagsInTagManipulatorSettings
            {
                WrapperTag = "marquee",
                TagsToWrap = new string[] { "em", "img" }
            };
        }

        protected virtual string ManipulateHtml(string html)
        {
            if (!string.IsNullOrEmpty(html))
            {
                return HtmlManipulator.Manipulate(html);
            }

            return html;
        }
    }
}

This base class creates an instance of our manipulator class above, passing in the required DTO housing the wrapper tag and tags to wrap settings.

Honestly, while writing this article and looking at this code, I am not completely happy about how I implemented this base class. I should have added a constructor which takes in the manipulator instance — thus allowing subclasses to provide their own manipulators, especially if these subclasses need to use a different manipulator than the one used by default in the base class.

Further, it probably would have been prudent to put the html tags I defined in my DTO instance into a patch config file.

Next, I defined my loadRichTextContent pipeline processor:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Shell.Controls.RichTextEditor.Pipelines.LoadRichTextContent;

using Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.Base;

namespace Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.LoadRichTextContent
{
    public class AddSomeMarquees : AddSomeMarqueesBase
    {
        public void Process(LoadRichTextContentArgs args)
        {
            args.Content = ManipulateHtml(args.Content);
        }
    }
}

Followed by my saveRichTextContentpipeline processor:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Shell.Controls.RichTextEditor.Pipelines.SaveRichTextContent;

using Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.Base;

namespace Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.SaveRichTextContent
{
    public class AddSomeMarquees : AddSomeMarqueesBase
    {
        public void Process(SaveRichTextContentArgs args)
        {
            args.Content = ManipulateHtml(args.Content);
        }
    }
}

Thereafter, I glued everything together via a patch config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <loadRichTextContent>
        <processor type="Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.LoadRichTextContent.AddSomeMarquees, Sitecore.Sandbox"/>
      </loadRichTextContent>
      <saveRichTextContent>
        <processor type="Sitecore.Sandbox.Shell.Controls.RichTextEditor.Pipelines.SaveRichTextContent.AddSomeMarquees, Sitecore.Sandbox"/>
      </saveRichTextContent>
    </pipelines>
  </sitecore>
</configuration>

Time to see the fruits of my labor above.

I’ve added some content in a Rich Text field:

RTF-Design-Before-Marquees

Here’s the html in the Rich Text field:

RTF-Html-Before-Marquees

I clicked the ‘Accept’ button in the Rich Text dialog window, and then saw the targeted content come to life:

RTF-With-Marquees

I launched the dialog window again to investigate what the html now looks like:

RTF-Html-With-Marquees

Mission accomplished — we now have marquees! ๐Ÿ™‚

I do want to point out I could not get my loadRichTextContent pipeline processor to run. I thought it would run when opening the Rich Text dialog, although I was wrong — it did not. I also tried to get it to run via the ‘Edit Html’ button, but to no avail.

If I am looking in the wrong place, or this is a known issue in Sitecore, please drop a comment and let me know.

Shoehorn Sitecore Web Forms for Marketers Field Values During a Custom Save Action

In a previous article, I discussed how one could remove Web Forms for Marketers (WFFM) field values before saving form data into the WFFM database — which ultimately make their way into the Form Reports for your form.

When I penned that article, my intentions were not to write an another highlighting the polar opposite action of inserting field values — I figured one could easily ascertain how to do this from that article.

However, last night I started to feel some guilt for not sharing how one could do this — there is one step in this process that isn’t completely intuitive — so I worked late into the night developing a solution of how one would go about doing this, and this article is the fruit of that effort.

Besides, how often does one get the opportunity to use the word shoehorn? If you aren’t familiar with what a shoehorn is, check out this article. I am using the word as an action verb in this post — meaning to insert values for fields in the WFFM database. ๐Ÿ™‚

I first defined a utility class and its interfaces for shoehorning field values into a AdaptedResultList collection — a collection of WFFM fields:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Shoehorns.Base
{
    public interface IShoehorn<T, U>
    {
        U Shoehorn(T source);
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Form.Core.Client.Data.Submit;

using Sitecore.Sandbox.Utilities.Shoehorns.Base;

namespace Sitecore.Sandbox.Utilities.Shoehorns.Base
{
    public interface IWFFMFieldValuesShoehorn : IShoehorn<AdaptedResultList, AdaptedResultList>
    {
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;
using Sitecore.Form.Core.Client.Data.Submit;
using Sitecore.Form.Core.Controls.Data;

using Sitecore.Sandbox.Utilities.Shoehorns.Base;

namespace Sitecore.Sandbox.Utilities.Shoehorns
{
    public class WFFMFieldValuesShoehorn : IWFFMFieldValuesShoehorn
    {
        private IDictionary<string, string> _FieldsForShoehorning;
        private IDictionary<string, string> FieldsForShoehorning 
        { 
            get
            {
                if (_FieldsForShoehorning == null)
                {
                    _FieldsForShoehorning = new Dictionary<string, string>();
                }

                return _FieldsForShoehorning;
            }
        }

        private WFFMFieldValuesShoehorn(IEnumerable<KeyValuePair<string, string>> fieldsForShoehorning)
        {
            SetFieldsForShoehorning(fieldsForShoehorning);
        }

        private void SetFieldsForShoehorning(IEnumerable<KeyValuePair<string, string>> fieldsForShoehorning)
        {
            AssertFieldsForShoehorning(fieldsForShoehorning);
            
            foreach (var keyValuePair in fieldsForShoehorning)
            {
                AddToDictionaryIfPossible(keyValuePair);
            }
        }

        private void AddToDictionaryIfPossible(KeyValuePair<string, string> keyValuePair)
        {
            if (!FieldsForShoehorning.ContainsKey(keyValuePair.Key))
            {
                FieldsForShoehorning.Add(keyValuePair.Key, keyValuePair.Value);
            }
        }

        private static void AssertFieldsForShoehorning(IEnumerable<KeyValuePair<string, string>> fieldsForShoehorning)
        {
            Assert.ArgumentNotNull(fieldsForShoehorning, "fieldsForShoehorning");

            foreach (var keyValuePair in fieldsForShoehorning)
            {
                Assert.ArgumentNotNullOrEmpty(keyValuePair.Key, "keyValuePair.Key");
            }
        }

        public AdaptedResultList Shoehorn(AdaptedResultList fields)
        {
            if (!CanProcessFields(fields))
            {
                return fields;
            }

            List<AdaptedControlResult> adaptedControlResults = new List<AdaptedControlResult>();

            foreach(AdaptedControlResult field in fields)
            {
                adaptedControlResults.Add(GetShoehornedField(field));
            }

            return adaptedControlResults;
        }

        private static bool CanProcessFields(IEnumerable<ControlResult> fields)
        {
            return fields != null && fields.Any();
        }

        private AdaptedControlResult GetShoehornedField(AdaptedControlResult field)
        {
            string value = string.Empty;

            if (FieldsForShoehorning.TryGetValue(field.FieldName, out value))
            {
                return GetShoehornedField(field, value);
            }

            return field;
        }

        private static AdaptedControlResult GetShoehornedField(ControlResult field, string value)
        {
            return new AdaptedControlResult(CreateNewControlResultWithValue(field, value), true);
        }

        private static ControlResult CreateNewControlResultWithValue(ControlResult field, string value)
        {
            ControlResult controlResult = new ControlResult(field.FieldName, value, field.Parameters);
            controlResult.FieldID = field.FieldID;
            return controlResult;
        }

        public static IWFFMFieldValuesShoehorn CreateNewWFFMFieldValuesShoehorn(IEnumerable<KeyValuePair<string, string>> fieldsForShoehorning)
        {
            return new WFFMFieldValuesShoehorn(fieldsForShoehorning);
        }
    }
}

This class is similar to the utility class I’ve used in my article on ripping out field values, albeit we are inserting values that don’t necessarily have to be the empty string.

I then defined a new WFFM field type — an invisible field that serves as a placeholder in our Form Reports for a given form. This is the non-intuitive step I was referring to above:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Text;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore.Diagnostics;

using Sitecore.Form.Web.UI.Controls;

namespace Sitecore.Sandbox.WFFM.Controls
{
    public class InvisibleField : SingleLineText
    {
        public InvisibleField()
        {
        }

        public InvisibleField(HtmlTextWriterTag tag) 
            : base(tag)
        {
        }

        protected override void OnInit(EventArgs e)
        {
            base.OnInit(e);
            HideControls(new Control[] { title, generalPanel });
        }

        private static void HideControls(IEnumerable<Control> controls)
        {
            Assert.ArgumentNotNull(controls, "controls");

            foreach (Control control in controls)
            {
                HideControl(control);
            }
        }

        private static void HideControl(Control control)
        {
            Assert.ArgumentNotNull(control, "control");
            control.Visible = false;
        }
    }
}

I had to register this new field in WFFM in the Sitecore Client under /sitecore/system/Modules/Web Forms for Marketers/Settings/Field Types/Custom:

invisible-field

Next, I created a custom field action that processes WFFM fields — and shoehorns values into fields where appropriate.

In my example, I will be capturing users’ browser user agent strings and their ASP.NET session identifiers — I strongly recommend defining your field names in a patch config file, although I did not do such a step for this post:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;

using Sitecore.Data;
using Sitecore.Form.Core.Client.Data.Submit;
using Sitecore.Form.Submit;

using Sitecore.Sandbox.Utilities.Shoehorns;
using Sitecore.Sandbox.Utilities.Shoehorns.Base;

namespace Sitecore.Sandbox.WFFM.Actions
{
    public class ShoehornFieldValuesThenSaveToDatabase : SaveToDatabase
    {
        public override void Execute(ID formId, AdaptedResultList fields, object[] data)
        {
            base.Execute(formId, ShoehornFields(fields), data);
        }

        private AdaptedResultList ShoehornFields(AdaptedResultList fields)
        {
            IWFFMFieldValuesShoehorn shoehorn = WFFMFieldValuesShoehorn.CreateNewWFFMFieldValuesShoehorn(CreateNewFieldsForShoehorning());
            return shoehorn.Shoehorn(fields);
        }

        private IEnumerable<KeyValuePair<string, string>> CreateNewFieldsForShoehorning()
        {
            return new KeyValuePair<string, string>[] 
            { 
                new KeyValuePair<string, string>("User Agent", HttpContext.Current.Request.UserAgent),
                new KeyValuePair<string, string>("Session ID", HttpContext.Current.Session.SessionID)
            };
        }
    }
}

I then registered my new Save to Database action in Sitecore under /sitecore/system/Modules/Web Forms for Marketers/Settings/Actions/Save Actions:

shoehorn-save-to-db

Let’s build a form. I created another random form. This one contains some invisible fields:

another-random-form

Next, I mapped my custom Save to Database action to my new form:

another-random-form-add-save-to-db-action

I also created a new page item to hold my WFFM form, and navigated to that page to fill in my form:

another-random-form-filled

I clicked the submit button and got a pleasant confirmation message:

another-random-form-confirmation

Thereafter, I pulled up the Form Reports for this form, and see that field values were shoehorned into it:

another-random-form-reports

That’s all there is to it.

What is the utility in all of this? Well, you might want to use this approach when sending information off to a third-party via a web service where that third-party application returns some kind of transaction identifier. Having this identifier in your Form Reports could help in troubleshoot issues that had arisen during that transaction — third-party payment processors come to mind.

Content Manage Custom Standard Values Tokens in the Sitecore Client

A few days back, John West — Chief Technology Officer at Sitecore USA — blogged about adding custom tokens in a subclass of Sitecore.Data.MasterVariablesReplacer.

One thing that surprised me was how his solution did not use NVelocity, albeit I discovered why: the class Sitecore.Data.MasterVariablesReplacer does not use it, and as John states in this tweet, using NVelocity in his solution would have been overkill — only a finite number of tokens are defined, so why do this?

This kindled an idea — what if we could define such tokens in the Sitecore Client? How would one go about doing that?

This post shows how I did just that, and used NVelocity via a utility class I had built for my article discussing NVelocity

I first created two templates: one that defines the Standard Values variable token — I named this Variable — and the template for a parent Master Variables folder item — this has no fields on it, so I’ve omitted its screenshot:

variable-template

I then defined some Glass.Sitecore.Mapper Models for my Variable and Master Variables templates — if you’re not familiar with Glass, or are but aren’t using it, I strongly recommend you go to http://www.glass.lu/ and check it out!

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Text;

using Sitecore.Diagnostics;
using Sitecore.Reflection;

using Glass.Sitecore.Mapper.Configuration.Attributes;

namespace Sitecore.Sandbox.Model
{
    [SitecoreClass(TemplateId="{E14F91E4-7AF6-42EC-A9A8-E71E47598BA1}")]
    public class Variable
    {
        [SitecoreField(FieldId="{34DCAC45-E5B2-436A-8A09-89F1FB785F60}")]
        public virtual string Token { get; set; }

        [SitecoreField(FieldId = "{CD749583-B111-4E69-90F2-1772B5C96146}")]
        public virtual string Type { get; set; }

        [SitecoreField(FieldId = "{E3AB8CED-1602-4A7F-AC9B-B9031FCA2290}")]
        public virtual string PropertyName { get; set; }

        public object _TypeInstance;
        public object TypeInstance
        {
            get
            {
                bool shouldCreateNewInstance = !string.IsNullOrEmpty(Type) 
                                                && !string.IsNullOrEmpty(PropertyName) 
                                                && _TypeInstance == null;

                if (shouldCreateNewInstance)
                {
                    _TypeInstance = CreateObject(Type, PropertyName);
                }

                return _TypeInstance;
            }
        }

        private object CreateObject(string type, string propertyName)
        {
            try
            {
                return GetStaticPropertyValue(type, propertyName);
            }
            catch (Exception ex)
            {
                Log.Error(this.ToString(), ex, this);
            }

            return null;
        }

        private object GetStaticPropertyValue(string type, string propertyName)
        {
            PropertyInfo propertyInfo = GetPropertyInfo(type, propertyName);
            return GetStaticPropertyValue(propertyInfo);
        }

        private PropertyInfo GetPropertyInfo(string type, string propertyName)
        {
            return GetType(type).GetProperty(propertyName);
        }

        private object GetStaticPropertyValue(PropertyInfo propertyInfo)
        {
            Assert.ArgumentNotNull(propertyInfo, "propertyInfo");
            return propertyInfo.GetValue(null, null);
        }

        private Type GetType(string type)
        {
            return ReflectionUtil.GetTypeInfo(type);
        }
    }
}

In my Variable model class, I added some logic that employs reflection to convert the defined type into an object we can use. This logic at the moment will only work with static properties, although could be extended for instance properties, and even methods on classes.

Here is the model for the Master Variables folder:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Glass.Sitecore.Mapper.Configuration.Attributes;

namespace Sitecore.Sandbox.Model
{
    [SitecoreClass(TemplateId = "{855A1D19-5475-43CB-B3E8-A4960A81FEFF}")]
    public class MasterVariables
    {
        [SitecoreChildren(IsLazy=true)]
        public virtual IEnumerable<Variable> Variables { get; set; }
    }
}

I then hooked up my Glass models in my Global.asax:

<%@Application Language='C#' Inherits="Sitecore.Web.Application" %>
<%@ Import Namespace="Glass.Sitecore.Mapper.Configuration.Attributes" %>
<script runat="server">
    protected void Application_Start(object sender, EventArgs e)
    {
        AttributeConfigurationLoader loader = new AttributeConfigurationLoader
        (
            new string[] { "Sitecore.Sandbox.Model, Sitecore.Sandbox" }
        );

        Glass.Sitecore.Mapper.Context context = new Glass.Sitecore.Mapper.Context(loader);
    }

    public void Application_End()
    {
    }

    public void Application_Error(object sender, EventArgs args)
    {
    }
</script>

Since my solution only works with static properties, I defined a wrapper class that accesses properties from Sitecore.Context for illustration:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Data
{
    public class SitecoreContextProperties
    {
        public static string DomainName
        {
            get
            {
                return Sitecore.Context.Domain.Name;
            }
        }

        public static string Username
        {
            get
            {
                return Sitecore.Context.User.Name;
            }
        }

        public static string DatabaseName
        {
            get
            {
                return Sitecore.Context.Database.Name;
            }
        }

        public static string CultureName
        {
            get
            {
                return Sitecore.Context.Culture.Name;
            }
        }

        public static string LanguageName
        {
            get
            {
                return Sitecore.Context.Language.Name;
            }
        }
    }
}

I then defined my subclass of Sitecore.Data.MasterVariablesReplacer. I let the “out of the box” tokens be expanded by the Sitecore.Data.MasterVariablesReplacer base class, and expand those defined in the Sitecore Client by using my token replacement utility class and content acquired from the master database via my Glass models:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Collections;
using Sitecore.Data;
using Sitecore.Diagnostics;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Text;

using Glass.Sitecore.Mapper;

using Sitecore.Sandbox.Model;
using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore.Sandbox.Data
{
    public class ContentManagedMasterVariablesReplacer : MasterVariablesReplacer
    {
        private const string MasterVariablesDatabase = "master";

        public override string Replace(string text, Item targetItem)
        {
            return CreateNewTokenator().ReplaceTokens(base.Replace(text, targetItem));
        }

        public override void ReplaceField(Item item, Field field)
        {
            base.ReplaceField(item, field);
            field.Value = CreateNewTokenator().ReplaceTokens(field.Value);
        }

        private static ITokenator CreateNewTokenator()
        {
            return Utilities.StringUtilities.Tokenator.CreateNewTokenator(CreateTokenKeyValues());
        }

        private static IEnumerable<TokenKeyValue> CreateTokenKeyValues()
        {
            IEnumerable<Variable> variables = GetVariables();
            IList<TokenKeyValue> tokenKeyValues = new List<TokenKeyValue>();

            foreach (Variable variable in variables)
            {
                AddVariable(tokenKeyValues, variable);
            }
            
            return tokenKeyValues;
        }

        private static void AddVariable(IList<TokenKeyValue> tokenKeyValues, Variable variable)
        {
            Assert.ArgumentNotNull(variable, "variable");
            bool canAddVariable = !string.IsNullOrEmpty(variable.Token) && variable.TypeInstance != null;

            if (canAddVariable)
            {
                tokenKeyValues.Add(new TokenKeyValue(variable.Token, variable.TypeInstance));
            }
        }

        private static IEnumerable<Variable> GetVariables()
        {
            MasterVariables masterVariables = GetMasterVariables();

            if (masterVariables != null)
            {
                return masterVariables.Variables;
            }

            return new List<Variable>();
        }

        private static MasterVariables GetMasterVariables()
        {
            const string masterVariablesPath = "/sitecore/content/Settings/Master Variables"; // hardcoded here for illustration -- please don't hardcode paths!
            ISitecoreService sitecoreService = GetSitecoreService();
            return sitecoreService.GetItem<MasterVariables>(masterVariablesPath);
        }

        private static ISitecoreService GetSitecoreService()
        {
            return new SitecoreService(MasterVariablesDatabase);
        }
    }
}

At first, I tried to lazy instantiate my Tokenator instance in a property, but discovered that this class is only instantiated once — that would prevent newly added tokens from ever making their way into the ContentManagedMasterVariablesReplacer instance. This is why I call CreateNewTokenator() in each place where a Tokenator instance is needed.

I then wedged in my ContentManagedMasterVariablesReplacer class using a patch config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <settings>
      <setting name="MasterVariablesReplacer">
        <patch:attribute name="value">Sitecore.Sandbox.Data.ContentManagedMasterVariablesReplacer,Sitecore.Sandbox</patch:attribute>
      </setting>
    </settings>
  </sitecore>
</configuration>

Now, let’s see if this works.

We first need some variables:

culture-variable

ticks-variable

Next, we’ll need an item for testing. Let’s create a template with fields that will be populated using my Sitecore.Sandbox.Data.ContentManagedMasterVariablesReplacer class:

standard-values-test-page-template

Under /sitecore/content/Home, I created a new item based on this test template.

We can see that hardcoded tokens were populated from the base Sitecore.Data.MasterVariablesReplacer class:

test-item-hardcoded-tokens-populated

The content managed tokens were populated from the Sitecore.Sandbox.Data.ContentManagedMasterVariablesReplacer class:

test-item-content-managed-tokens-populated

That’s all there is to it.

Please keep in mind there could be potential performance issues with the above, especially when there are lots of Variable items — the code always creates an instance of the Tokenator in the ContentManagedMasterVariablesReplacer instance, which is pulling content from Sitecore each time, and we’re using reflection for each Variable. It would probably be best to enhance the above by leveraging Lucene in some way to increase its performance.

Further, these items should only be created/edited by advanced users or developers familiar with ASP.NET code — how else would one be able to populate the Type and Property Name fields in a Variable item?

Despite these, just imagine the big smiley your boss will send your way when you say new Standard Values variables no longer require any code changes coupled with a deployment. I hope that smiley puts a big smile on your face! ๐Ÿ™‚

Put Sitecore to Work for You: Build Custom Task Agents

How many times have you seen some manual process and thought to yourself how much easier your life would be easier if that process were automated?

Custom Sitecore task agents could be of assistance on achieving some automation in Sitecore.

Last night, I built a custom task agent that deletes “expired” items from the recycle bin in all three Sitecore databases — items that have been sitting in the recycle bin after a specified number of days.

I came up with this idea after remembering an instance I had seen in the past where there were so many items in the recycle bin, finding an item to restore would be more difficult than finding a needle in a haystack.

Using .NET reflector, I looked at how Sitecore.Tasks.UrlAgent in Sitecore.Kernel.dll was coded to see if I had to do anything special when building my agent — an example would be ascertaining whether I needed to inherit from a custom base class — and also looked at code in Sitecore.Shell.Applications.Archives.RecycleBin.RecycleBinPage in Sitecore.Client.dll coupled with Sitecore.Shell.Framework.Commands.Archives.Delete in Sitecore.Kernel.dll to figure out how to permanently delete items in the recycle bin.

After doing my research in those two assemblies, I came up with this custom task agent:

using System;
using System.Collections.Generic;
using System.Linq;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Archiving;
using Sitecore.Diagnostics;

namespace Sitecore.Sandbox.Tasks
{
    public class RecycleBinCleanupAgent
    {
        private const string ArchiveName = "recyclebin";
        private static readonly char[] Delimiters = new char[] { ',', '|' };

        public IEnumerable<string> DatabaseNames { get; set; }

        private IEnumerable<Database> _Databases;
        private IEnumerable<Database> Databases
        {
            get
            {
                if (_Databases == null)
                {
                    _Databases = GetDatabases();
                }

                return _Databases;
            }
        }

        public int NumberOfDaysUntilExpiration { get; set; }

        public bool Enabled { get; set; }

        public bool LogActivity { get; set; }

        public RecycleBinCleanupAgent(string databases)
        {
            SetDatabases(databases);
        }

        private void SetDatabases(string databases)
        {
            Assert.ArgumentNotNullOrEmpty(databases, "databases");
            DatabaseNames = databases.Split(Delimiters, StringSplitOptions.RemoveEmptyEntries).Select(database => database.Trim()).ToList();
        }

        public void Run()
        {
            if (Enabled)
            {
                RemoveEntriesInAllDatabases();
            }
        }
        
        private void RemoveEntriesInAllDatabases()
        {
            DateTime expired = GetExpiredDateTime();

            if (expired == DateTime.MinValue)
            {
                return;
            }

            foreach (Database database in Databases)
            {
                RemoveEntries(database, expired);
            }
        }

        private DateTime GetExpiredDateTime()
        {
            if (NumberOfDaysUntilExpiration > 0)
            {
                return DateTime.Now.AddDays(-1 * NumberOfDaysUntilExpiration).ToUniversalTime();
            }

            return DateTime.MinValue;
        }

        private void RemoveEntries(Database database, DateTime expired)
        {
            int deletedEntriesCount = RemoveEntries(GetArchive(database), expired);
            LogInfo(deletedEntriesCount, database.Name);
        }

        private static int RemoveEntries(Archive archive, DateTime expired)
        {
            IEnumerable<ArchiveEntry> archiveEntries = GetAllEntries(archive);
            int deletedEntriesCount = 0;

            foreach (ArchiveEntry archiveEntry in archiveEntries)
            {
                if (ShouldDeleteEntry(archiveEntry, expired))
                {
                    archive.RemoveEntries(CreateNewArchiveQuery(archiveEntry));
                    deletedEntriesCount++;
                }
            }

            return deletedEntriesCount;
        }

        private static IEnumerable<ArchiveEntry> GetAllEntries(Archive archive)
        {
            Assert.ArgumentNotNull(archive, "archive");
            return archive.GetEntries(0, archive.GetEntryCount()); ;
        }

        private static bool ShouldDeleteEntry(ArchiveEntry archiveEntry, DateTime expired)
        {
            Assert.ArgumentNotNull(archiveEntry, "archiveEntry");
            Assert.ArgumentCondition(expired > DateTime.MinValue, "expired", "expired must be set!");
            return archiveEntry.ArchiveDate <= expired;
        }

        private static ArchiveQuery CreateNewArchiveQuery(ArchiveEntry archiveEntry)
        {
            Assert.ArgumentNotNull(archiveEntry, "archiveEntry");
            return CreateNewArchiveQuery(archiveEntry.ArchivalId);
        }

        private static ArchiveQuery CreateNewArchiveQuery(Guid archivalId)
        {
            Assert.ArgumentCondition(archivalId != Guid.Empty, "archivalId", "archivalId must be set!");
            return new ArchiveQuery { ArchivalId = archivalId };
        }

        private void LogInfo(int deletedEntriesCount, string databaseName)
        {
            bool canLogInfo = LogActivity
                              && deletedEntriesCount > 0 
                              && !string.IsNullOrEmpty(databaseName);

            if (canLogInfo)
            {
                Log.Info(CreateNewLogEntry(deletedEntriesCount, databaseName, ArchiveName), this);
            }
        }

        private static string CreateNewLogEntry(int expiredEntryCount, string databaseName, string archiveName)
        {
            return string.Format("{0} expired archive entries permanently deleted (database: {1}, archive: {2})", expiredEntryCount, databaseName, archiveName);
        }

        private IEnumerable<Database> GetDatabases()
        {
            if (DatabaseNames != null)
            {
                return DatabaseNames.Select(database => Factory.GetDatabase(database)).ToList();
            }

            return new List<Database>();
        }

        private static Archive GetArchive(Database database)
        {
            Assert.ArgumentNotNull(database, "database");
            return ArchiveManager.GetArchive(ArchiveName, database);
        }
    }
}

Basically, it loops over all recycle bin archived entries in all specified databases after an allotted time interval — the time interval and target databases are set in a patch config file you will see below — and removes an entry when its archival date is older than the minimum expiration date — a date I derive from the NumberOfDaysUntilExpiration setting:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <scheduling>
      <agent type="Sitecore.Sandbox.Tasks.RecycleBinCleanupAgent, Sitecore.Sandbox" method="Run" interval="01:00:00">
        <param desc="databases">core, master, web</param>
        <NumberOfDaysUntilExpiration>30</NumberOfDaysUntilExpiration>
        <LogActivity>true</LogActivity>
        <Enabled>true</Enabled>
      </agent>
    </scheduling>
  </sitecore>
</configuration>

I had 89 items in my master database’s recycle bin before my task agent ran:

before-recycle-bin-agent-runs-master

I walked away for a bit to watch some television, eat dinner, and surf Twitter for a bit, and phone my brother. I then returned to see the following in my Sitecore log:

after-recycle-bin-agent-runs-master-log

I went into the recycle bin in my master database, and saw there were 5 items left after my task agent executed:

after-recycle-bin-agent-runs-master

As you can see, my custom task agent deleted expired recycle bin items as designed. ๐Ÿ™‚

Manipulate Field Values in a Custom Sitecore Web Forms for Marketers DataProvider

In my Experiments with Field Data Encryption in Sitecore article, I briefly mentioned designing a custom Web Forms for Marketers (WFFM) DataProvider that uses the decorator pattern for encrypting and decrypting field values before saving and retrieving field values from the WFFM database.

From the time I penned that article up until now, I have been feeling a bit guilty that I may have left you hanging by not going into the mechanics around how I did that — I built that DataProvider for my company to be used in one of our healthcare specific content management modules.

To make up for not showing you this, I decided to build another custom WFFM DataProvider — one that will replace periods with smiley faces and the word “Sitecore” with “Sitecoreยฎ”, case insensitively.

First, I defined an interface for utility classes that will manipulate objects for us. All manipulators will consume an object of a specified type, manipulate that object in some way, and then return the maniputed object to the manipulator object’s client:

namespace Sitecore.Sandbox.Utilities.Manipulators.Base
{
    public interface IManipulator<T>
    {
        T Manipulate(T source);
    }
}

The first manipulator I built is a string manipulator. Basically, this object will take in a string and replace a specified substring with another:

namespace Sitecore.Sandbox.Utilities.Manipulators.Base
{
    public interface IStringReplacementManipulator : IManipulator<string>
    {
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Manipulators.Base;

namespace Sitecore.Sandbox.Utilities.Manipulators
{
    public class StringReplacementManipulator : IStringReplacementManipulator
    {
        private string PatternToReplace { get; set; }
        private string ReplacementString { get; set; }
        private bool IgnoreCase { get; set; }

        private StringReplacementManipulator(string patternToReplace, string replacementString)
            : this(patternToReplace, replacementString, false)
        {
        }

        private StringReplacementManipulator(string patternToReplace, string replacementString, bool ignoreCase)
        {
            SetPatternToReplace(patternToReplace);
            SetReplacementString(replacementString);
            SetIgnoreCase(ignoreCase);
        }

        private void SetPatternToReplace(string patternToReplace)
        {
            Assert.ArgumentNotNullOrEmpty(patternToReplace, "patternToReplace");
            PatternToReplace = patternToReplace;
        }

        private void SetReplacementString(string replacementString)
        {
            ReplacementString = replacementString;
        }

        private void SetIgnoreCase(bool ignoreCase)
        {
            IgnoreCase = ignoreCase;
        }

        public string Manipulate(string source)
        {
            Assert.ArgumentNotNullOrEmpty(source, "source");
            RegexOptions regexOptions = RegexOptions.None;

            if (IgnoreCase)
            {
                regexOptions = RegexOptions.IgnoreCase;
            }

            return Regex.Replace(source, PatternToReplace, ReplacementString, regexOptions);
        }

        public static IStringReplacementManipulator CreateNewStringReplacementManipulator(string patternToReplace, string replacementString)
        {
            return new StringReplacementManipulator(patternToReplace, replacementString);
        }

        public static IStringReplacementManipulator CreateNewStringReplacementManipulator(string patternToReplace, string replacementString, bool ignoreCase)
        {
            return new StringReplacementManipulator(patternToReplace, replacementString, ignoreCase);
        }
    }
}

Clients of this class can choose to have substrings replaced in a case sensitive or insensitive manner.

By experimenting in a custom WFFM DataProvider and investigating code via .NET reflector in the WFFM assemblies, I discovered I had to create a custom object that implements Sitecore.Forms.Data.IField.

Out of the box, WFFM uses Sitecore.Forms.Data.DefiniteField — a class that implements Sitecore.Forms.Data.IField, albeit this class is declared internal and cannot be reused outside of the Sitecore.Forms.Core.dll assembly.

When I attempted to change the Value property of this object in a custom WFFM DataProvider, changes did not stick for some reason — a reason that I have not definitely ascertained.

However, to get around these lost Value property changes, I created a custom object that implements Sitecore.Forms.Data.IField:

using System;

using Sitecore.Forms.Data;

namespace Sitecore.Sandbox.Utilities.Manipulators.DTO
{
    public class WFFMField : IField
    {
        public string Data { get; set; }

        public Guid FieldId { get; set; }

        public string FieldName { get; set; }

        public IForm Form { get; set; }

        public Guid Id { get; internal set; }

        public string Value { get; set; }
    }
}

Next, I built a WFFM Field collection manipulator — an IEnumerable of Sitecore.Forms.Data.IField defined in Sitecore.Forms.Core.dll — using the DTO defined above:

using System.Collections.Generic;

using Sitecore.Forms.Data;

namespace Sitecore.Sandbox.Utilities.Manipulators.Base
{
    public interface IWFFMFieldsManipulator : IManipulator<IEnumerable<IField>>
    {
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;
using Sitecore.Forms.Data;

using Sitecore.Sandbox.Utilities.Manipulators.Base;
using Sitecore.Sandbox.Utilities.Manipulators.DTO;

namespace Sitecore.Sandbox.Utilities.Manipulators
{
    public class WFFMFieldsManipulator : IWFFMFieldsManipulator
    {
        private IEnumerable<IStringReplacementManipulator> FieldValueManipulators { get; set; }

        private WFFMFieldsManipulator(params IStringReplacementManipulator[] fieldValueManipulator)
            : this(fieldValueManipulator.AsEnumerable())
        {
        }

        private WFFMFieldsManipulator(IEnumerable<IStringReplacementManipulator> fieldValueManipulators)
        {
            SetFieldValueManipulator(fieldValueManipulators);
        }

        private void SetFieldValueManipulator(IEnumerable<IStringReplacementManipulator> fieldValueManipulators)
        {
            Assert.ArgumentNotNull(fieldValueManipulators, "fieldValueManipulators");
            foreach (IStringReplacementManipulator fieldValueManipulator in fieldValueManipulators)
            {
                Assert.ArgumentNotNull(fieldValueManipulator, "fieldValueManipulator");
            }

            FieldValueManipulators = fieldValueManipulators;
        }

        public IEnumerable<IField> Manipulate(IEnumerable<IField> fields)
        {
            IList<IField> maniuplatdFields = new List<IField>();

            foreach (IField field in fields)
            {
                maniuplatdFields.Add(MainpulateFieldValue(field));
            }

            return maniuplatdFields;
        }

        private IField MainpulateFieldValue(IField field)
        {
            IField maniuplatedField = CreateNewWFFMField(field);

            if (maniuplatedField != null)
            {
                maniuplatedField.Value = ManipulateString(maniuplatedField.Value);
            }

            return maniuplatedField;
        }

        private static IField CreateNewWFFMField(IField field)
        {
            if(field != null)
            {
                return new WFFMField
                {
                    Data = field.Data,
                    FieldId = field.FieldId,
                    FieldName = field.FieldName,
                    Form = field.Form,
                    Id = field.Id,
                    Value = field.Value
                };
            }

            return null;
        }

        private string ManipulateString(string stringToManipulate)
        {
            if (string.IsNullOrEmpty(stringToManipulate))
            {
                return string.Empty;
            }

            string manipulatedString = stringToManipulate;

            foreach(IStringReplacementManipulator fieldValueManipulator in FieldValueManipulators)
            {
                manipulatedString = fieldValueManipulator.Manipulate(manipulatedString);
            }

            return manipulatedString;
        }

        public static IWFFMFieldsManipulator CreateNewWFFMFieldsManipulator(params IStringReplacementManipulator[] fieldValueManipulators)
        {
            return new WFFMFieldsManipulator(fieldValueManipulators);
        }

        public static IWFFMFieldsManipulator CreateNewWFFMFieldsManipulator(IEnumerable<IStringReplacementManipulator> fieldValueManipulators)
        {
            return new WFFMFieldsManipulator(fieldValueManipulators);
        }
    }
}

This manipulator consumes a collection of string manipulators and delegates to these for making changes to WFFM field values.

Now, it’s time to create a custom WFFM DataProvider that uses our manipulators defined above.

In my local sandbox Sitecore instance, I’m using SQLite for my WFFM module, and must decorate an instance of Sitecore.Forms.Data.DataProviders.SQLite.SQLiteWFMDataProvider — although this approach would be the same using MS SQL or Oracle since all WFFM DataProviders should inherit from the abstract class Sitecore.Forms.Data.DataProviders.WFMDataProviderBase:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;
using Sitecore.Forms.Data;
using Sitecore.Forms.Data.DataProviders;
using Sitecore.Forms.Data.DataProviders.SQLite;

using Sitecore.Sandbox.Translation.Base;
using Sitecore.Sandbox.Utilities.Manipulators.Base;
using Sitecore.Sandbox.Utilities.Manipulators;

namespace Sitecore.Sandbox.WFFM.Data.DataProviders
{
    public class FieldValueManipulationSQLiteWFMDataProvider : WFMDataProviderBase
    {
        private static readonly IStringReplacementManipulator RegisteredTrademarkManipulator = StringReplacementManipulator.CreateNewStringReplacementManipulator("sitecore", " Sitecoreยฎ", true);
        private static readonly IStringReplacementManipulator PeriodsToSmiliesManipulator = StringReplacementManipulator.CreateNewStringReplacementManipulator("\\.", " :)");
        private static readonly IWFFMFieldsManipulator FieldsManipulator = WFFMFieldsManipulator.CreateNewWFFMFieldsManipulator(RegisteredTrademarkManipulator, PeriodsToSmiliesManipulator);

	    private WFMDataProviderBase InnerProvider { get; set; }

        public FieldValueManipulationSQLiteWFMDataProvider() 
		    : this(CreateNewSQLiteWFMDataProvider())
        {
        }

        public FieldValueManipulationSQLiteWFMDataProvider(string connectionString)
		    : this(CreateNewSQLiteWFMDataProvider(connectionString))
        {
        }

        public FieldValueManipulationSQLiteWFMDataProvider(WFMDataProviderBase innerProvider)
        {
		    SetInnerProvider(innerProvider);
	    }
	
	    private void SetInnerProvider(WFMDataProviderBase innerProvider)
	    {
		    Assert.ArgumentNotNull(innerProvider, "innerProvider");
		    InnerProvider = innerProvider;
	    }

        private static WFMDataProviderBase CreateNewSQLiteWFMDataProvider()
	    {
            return new SQLiteWFMDataProvider();
	    }

        private static WFMDataProviderBase CreateNewSQLiteWFMDataProvider(string connectionString)
	    {
		    Assert.ArgumentNotNullOrEmpty(connectionString, "connectionString");
            return new SQLiteWFMDataProvider(connectionString);
	    }

        public override void ChangeStorage(Guid formItemId, string newStorage)
        {
            InnerProvider.ChangeStorage(formItemId, newStorage);
        }

        public override void ChangeStorageForForms(IEnumerable<Guid> ids, string storageName)
        {
            InnerProvider.ChangeStorageForForms(ids, storageName);
        }

        public override void DeleteForms(IEnumerable<Guid> formSubmitIds)
        {
            InnerProvider.DeleteForms(formSubmitIds);
        }

        public override void DeleteForms(Guid formItemId, string storageName)
        {
            InnerProvider.DeleteForms(formItemId, storageName);
        }

        public override IEnumerable<IPool> GetAbundantPools(Guid fieldId, int top, out int total)
        {
            return InnerProvider.GetAbundantPools(fieldId, top, out total);
        }

        public override IEnumerable<IForm> GetForms(QueryParams queryParams, out int total)
        {
            return InnerProvider.GetForms(queryParams, out total);
        }

        public override IEnumerable<IForm> GetFormsByIds(IEnumerable<Guid> ids)
        {
            return InnerProvider.GetFormsByIds(ids);
        }

        public override int GetFormsCount(Guid formItemId, string storageName, string filter)
        {
            return InnerProvider.GetFormsCount(formItemId, storageName, filter);
        }

        public override IEnumerable<IPool> GetPools(Guid fieldId)
        {
            return InnerProvider.GetPools(fieldId);
        }

        public override void InsertForm(IForm form)
        {
            ManipulateFields(form);
            InnerProvider.InsertForm(form);
        }

        public override void ResetPool(Guid fieldId)
        {
            InnerProvider.ResetPool(fieldId);
        }

        public override IForm SelectSingleForm(Guid fieldId, string likeValue)
        {
            return InnerProvider.SelectSingleForm(fieldId, likeValue);
        }

        public override bool UpdateForm(IForm form)
        {
            ManipulateFields(form);
            return InnerProvider.UpdateForm(form);
        }

        private static void ManipulateFields(IForm form)
        {
            Assert.ArgumentNotNull(form, "form");
            Assert.ArgumentNotNull(form.Field, "form.Field");
            form.Field = FieldsManipulator.Manipulate(form.Field);
        }
    }
}

This custom DataProvider creates an instance of Sitecore.Forms.Data.DataProviders.SQLite.SQLiteWFMDataProvider and delegates method calls to it.

However, before delegating to insert and update method calls, forms fields are manipulated via our manipulators objects — our manipulators will replace periods with smiley faces, and the word “Sitecore” with “Sitecoreยฎ”.

I then had to configure WFFM to use my custom DataProvider above in /App_Config/Include/forms.config:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:x="http://www.sitecore.net/xmlconfig/">
	<sitecore>
	
		<!-- A bunch of stuff here -->
		
		<!-- SQLite -->
		<formsDataProvider type="Sitecore.Sandbox.WFFM.Data.DataProviders.FieldValueManipulationSQLiteWFMDataProvider,Sitecore.Sandbox">
			<param desc="connection string">Data Source=/data/sitecore_webforms.db;version=3;BinaryGUID=true</param>
		</formsDataProvider>

		<!-- A bunch of stuff here -->
	
	</sitecore>
</configuration>

For testing, I build a random WFFM form containing three fields, and created a page item to hold this form.

I then navigated to my form page and filled it in:

random-form-before-sumbit

I then clicked the Submit button:

random-form-after-submit

I then opened up the Form Reports for my form:

random-form-reports

As you can see, it all gelled together nicely. ๐Ÿ™‚

Rip Out Sitecore Web Forms for Marketers Field Values During a Custom Save Action

A couple of weeks ago, I architected a Web Forms for Marketers (WFFM) solution that sends credit card information to a third-party credit card processor — via a custom form verification step — and blanks out sensitive credit card information before saving into the WFFM database.

Out of the box, WFFM will save credit card information in plain text to its database — yes, the data is hidden in the form report, although is saved to the database as plain text. This information being saved as plain text is a security risk.

One could create a solution similar to what I had done in my article discussing data encryption — albeit it might be in your best interest to avoid any headaches and potential security issues around saving and holding onto credit card information.

I can’t share the solution I built a couple of weeks ago — I built it for my current employer. I will, however, share a similar solution where I blank out the value of a field containing a social security number — a unique identifier of a citizen of the United States — of a person posing a question to the Internal Revenue Service (IRS) of the United States (yes, I chose this topic since tax season is upon us here in the US, and I loathe doing taxes :)).

First, I created a custom Web Forms for Marketers field for social security numbers. It is just a composite control containing three textboxes separated by labels containing hyphens:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Text;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore.Form.Core.Attributes;
using Sitecore.Form.Core.Controls.Data;
using Sitecore.Form.Core.Visual;
using Sitecore.Form.Web.UI.Controls;

namespace Sitecore.Sandbox.WFFM.Controls
{
    public class SocialSecurityNumber : InputControl
    {
        private const string Hyphen = "-";
        private static readonly string baseCssClassName = "scfSingleLineTextBorder";
        protected TextBox firstPart;
        protected System.Web.UI.WebControls.Label firstMiddleSeparator;
        protected TextBox middlePart;
        protected System.Web.UI.WebControls.Label middleLastSeparator;
        protected TextBox lastPart;

        public SocialSecurityNumber() : this(HtmlTextWriterTag.Div)
        {
            firstPart = new TextBox();
            firstMiddleSeparator = new System.Web.UI.WebControls.Label { Text = Hyphen };
            middlePart = new TextBox();
            middleLastSeparator = new System.Web.UI.WebControls.Label { Text = Hyphen };
            lastPart = new TextBox();
        }

        public SocialSecurityNumber(HtmlTextWriterTag tag)
            : base(tag)
        {
            this.CssClass = baseCssClassName;
        }

        protected override void OnInit(EventArgs e)
        {
            SetCssClasses();
            SetTextBoxeWidths();
            SetMaxLengths();
            SetTextBoxModes();
            AddChildControls();
        }

        private void SetCssClasses()
        {
            help.CssClass = "scfSingleLineTextUsefulInfo";
            title.CssClass = "scfSingleLineTextLabel";
            generalPanel.CssClass = "scfSingleLineGeneralPanel";
        }

        private void SetTextBoxeWidths()
        {
            firstPart.Style.Add("width", "40px");
            middlePart.Style.Add("width", "35px");
            lastPart.Style.Add("width", "50px");
        }

        private void SetMaxLengths()
        {
            firstPart.MaxLength = 3;
            middlePart.MaxLength = 2;
            lastPart.MaxLength = 4;
        }

        private void SetTextBoxModes()
        {
            firstPart.TextMode = TextBoxMode.SingleLine;
            middlePart.TextMode = TextBoxMode.SingleLine;
            lastPart.TextMode = TextBoxMode.SingleLine;
        }

        private void AddChildControls()
        {
            Controls.AddAt(0, generalPanel);
            Controls.AddAt(0, title);
            
            generalPanel.Controls.Add(firstPart);
            generalPanel.Controls.Add(firstMiddleSeparator);
            generalPanel.Controls.Add(middlePart);
            generalPanel.Controls.Add(middleLastSeparator);
            generalPanel.Controls.Add(lastPart);
            generalPanel.Controls.Add(help);
        }

        public override string ID
        {
            get
            {
                return firstPart.ID;
            }
            set
            {
                title.ID = string.Concat(value, "_text");
                firstPart.ID = value;
                base.ID = string.Concat(value, "_scope");
                title.AssociatedControlID = firstPart.ID;
            }
        }

        public override ControlResult Result
        {
            get
            {
                return GetNewControlResult();
            }
        }

        private ControlResult GetNewControlResult()
        {
            TrimAllTextBoxes();
            return new ControlResult(ControlName, GetValue(), string.Empty);
        }

        private string GetValue()
        {
            bool hasValue = !string.IsNullOrEmpty(firstPart.Text) 
                            && !string.IsNullOrEmpty(middlePart.Text) 
                            && !string.IsNullOrEmpty(lastPart.Text);

            if (hasValue)
            {
                return string.Concat(firstPart.Text, firstMiddleSeparator.Text, middlePart.Text, middleLastSeparator.Text, lastPart.Text);
            }

            return string.Empty;
        }

        private void TrimAllTextBoxes()
        {
            firstPart.Text = firstPart.Text.Trim();
            middlePart.Text = middlePart.Text.Trim();
            lastPart.Text = lastPart.Text.Trim();
        }
    }
}

I then registered this custom field in Sitecore. I added my custom field item under /sitecore/system/Modules/Web Forms for Marketers/Settings/Field Types/Custom:

registered-ssn-field

Next, I decided to create a generic interface that defines objects that will excise or rip out values from a given object of a certain type, and returns a new object of another type — although both types could be the same:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Excisors.Base
{
    public interface IExcisor<T, U>
    {
        U Excise(T source);
    }
}

My WFFM excisor will take in a collection of WFFM fields and return a new collection where the targeted field values — field values that I don’t want saved into the WFFM database — are set to the empty string.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Form.Core.Controls.Data;
using Sitecore.Form.Core.Client.Data.Submit;

namespace Sitecore.Sandbox.Utilities.Excisors.Base
{
    public interface IWFFMFieldValuesExcisor : IExcisor<AdaptedResultList, AdaptedResultList>
    {
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;
using Sitecore.Form.Core.Client.Data.Submit;
using Sitecore.Form.Core.Controls.Data;

using Sitecore.Sandbox.Utilities.Excisors.Base;

namespace Sitecore.Sandbox.Utilities.Excisors
{
    public class WFFMFieldValuesExcisor : IWFFMFieldValuesExcisor
    {
        private IEnumerable<string> FieldNamesForExtraction { get; set; }

        private WFFMFieldValuesExcisor(IEnumerable<string> fieldNamesForExtraction)
        {
            SetFieldNamesForExtraction(fieldNamesForExtraction);
        }

        private void SetFieldNamesForExtraction(IEnumerable<string> fieldNamesForExtraction)
        {
            Assert.ArgumentNotNull(fieldNamesForExtraction, "fieldNamesForExtraction");
            FieldNamesForExtraction = fieldNamesForExtraction;
        }

        public AdaptedResultList Excise(AdaptedResultList fields)
        {
            if(fields == null || fields.Count() < 1)
            {
                return fields;
            }

            List<AdaptedControlResult> adaptedControlResults = new List<AdaptedControlResult>();

            foreach(AdaptedControlResult field in fields)
            {
                adaptedControlResults.Add(GetExtractValueFieldIfApplicable(field));
            }

            return adaptedControlResults;
        }

        private AdaptedControlResult GetExtractValueFieldIfApplicable(AdaptedControlResult field)
        {
            if(ShouldExtractFieldValue(field))
            {
                return GetExtractedValueField(field);
            }

            return field;
        }

        private bool ShouldExtractFieldValue(ControlResult field)
        {
            return FieldNamesForExtraction.Contains(field.FieldName);
        }

        private static AdaptedControlResult GetExtractedValueField(ControlResult field)
        {
            return new AdaptedControlResult(CreateNewControlResultWithEmptyValue(field), true);
        }

        private static ControlResult CreateNewControlResultWithEmptyValue(ControlResult field)
        {
            ControlResult controlResult = new ControlResult(field.FieldName, string.Empty, field.Parameters);
            controlResult.FieldID = field.FieldID;
            return controlResult;
        }

        public static IWFFMFieldValuesExcisor CreateNewWFFMFieldValuesExcisor(IEnumerable<string> fieldNamesForExtraction)
        {
            return new WFFMFieldValuesExcisor(fieldNamesForExtraction);
        }
    }
}

For scalability purposes and to avoid hard-coding field name values, I put my targeted fields — I only have one at the moment — into a config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
	<sitecore>
		<settings>
			<setting name="ExciseFields.SocialSecurityNumberField" value="Social Security Number" />
		</settings>
	</sitecore>
</configuration>

I then created a custom save to database action where I call my utility class to rip out my specified targeted fields, and pass that through onto the base save to database action class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Form.Core.Client.Data.Submit;
using Sitecore.Form.Submit;

using Sitecore.Sandbox.Utilities.Excisors;
using Sitecore.Sandbox.Utilities.Excisors.Base;

namespace Sitecore.Sandbox.WFFM.Actions
{
    public class ExciseFieldValuesThenSaveToDatabase : SaveToDatabase
    {
        private static readonly IEnumerable<string> FieldsToExcise = new string[] { Settings.GetSetting("ExciseFields.SocialSecurityNumberField") };
        
        public override void Execute(ID formId, AdaptedResultList fields, object[] data)
        {
            base.Execute(formId, ExciseFields(fields), data);
        }

        private AdaptedResultList ExciseFields(AdaptedResultList fields)
        {
            IWFFMFieldValuesExcisor excisor = WFFMFieldValuesExcisor.CreateNewWFFMFieldValuesExcisor(FieldsToExcise);
            return excisor.Excise(fields);
        }
    }
}

I created a new item to register my custom save to database action under /sitecore/system/Modules/Web Forms for Marketers/Settings/Actions/Save Actions:
registered-save-db-action

I then created my form:

built-irs-form

I set this form on a new page item, and published all of my changes above.

Now, it’s time to test this out. I navigated to my form, and filled it in as an irate tax payer might:

filled-in-irs-form

Looking at my form’s report, I see that the social security number was blanked out before being saved to the database:

irs-form-report

A similar solution could also be used to add new field values into the collection of fields — values of fields that wouldn’t be available on the form, but should be displayed in the form report. An example might include a transaction ID or approval ID returned from a third-party credit card processor. Instead of ripping values, values would be inserted into the collection of fields, and then passed along to the base save to database action class.

Seek and Destroy: Root Out Newlines and Carriage Returns in Multi-Line Text Fields

A couple of days ago, Sitecore MVP Brian Pedersen wrote an article discussing how newlines and carriage returns in Multi-Line Text fields can intrusively launch Sitecore’s “Do you want to save the changes to the item?” dialog box when clicking away from an item — even when you’ve made no changes to the item. Brian then offered an extension method on the String class as a way to remedy this annoyance.

However, Brian’s extension method cannot serve a solution on its own. It has to be invoked from somewhere to stamp out the substring malefactors — newlines (“\n”), carriage returns (“\r”), tabs (“\t”), and non-breaking spaces (“\xA0”).

This article gives one possible solution for uprooting these from Multi-Line Text fields within the Sitecore client by removing them upon item save.

First, I created a utility class that removes specified substrings from a string passed to it.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public interface ISubstringAnnihilator
    {
        string AnnihilateSubstrings(string input);
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class SubstringAnnihilator : ISubstringAnnihilator
    {
        private const string ReplacementString = " ";
        private static readonly IEnumerable<string> SubstringsToAnnihilate = new string[] { "\r\n", "\n", "\r", "\t", "\xA0"};

        private SubstringAnnihilator()
        {
        }

        public string AnnihilateSubstrings(string input)
        {
            foreach (string substringToAnnihilate in SubstringsToAnnihilate)
            {
                input = input.Replace(substringToAnnihilate, ReplacementString);
            }

            return input;
        }

        public static ISubstringAnnihilator CreateNewSubstringAnnihilator()
        {
            return new SubstringAnnihilator();
        }
    }
}

It would probably be ideal to move the target substrings defined within the SubstringsToAnnihilate string array into a configuration file or even into Sitecore itself. I decided not introduce that complexity here for the sake of brevity.

Next, I created a Save pipeline. I used .NET Reflector to see how other Save pipelines in /configuration/sitecore/processors/saveUI/ in the Web.config were built — I used these as a model for creating my own — and used my SubstringAnnihilator utility class to seek and destroy the target substrings (well, just replace them with a space :)).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Data;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.Save;

using Sitecore.Sandbox.Utilities.StringUtilities;
using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Pipelines.SaveUI
{
    public class FixMultiLineTextFields
    {
        private static readonly ISubstringAnnihilator Annihilator = SubstringAnnihilator.CreateNewSubstringAnnihilator();

        public void Process(SaveArgs saveArgs)
        {
            FixAllItemFieldsWhereApplicable(saveArgs);
        }

        private static void FixAllItemFieldsWhereApplicable(SaveArgs saveArgs)
        {
            AssertSaveArgs(saveArgs);
            foreach (SaveArgs.SaveItem saveItem in saveArgs.Items)
            {
                FixSaveItemFieldsWhereApplicable(saveItem);
            }
        }

        private static void AssertSaveArgs(SaveArgs saveArgs)
        {
            Assert.ArgumentNotNull(saveArgs, "saveArgs");
            Assert.IsNotNull(saveArgs.Items, "saveArgs.Items");
        }

        private static void FixSaveItemFieldsWhereApplicable(SaveArgs.SaveItem saveItem)
        {
            Item item = GetItem(saveItem);
            foreach (SaveArgs.SaveField saveField in saveItem.Fields)
            {
                FixSaveItemFieldIfApplicable(item, saveField);
            }
        }

        private static Item GetItem(SaveArgs.SaveItem saveItem)
        {
            if (saveItem != null)
            {
                return Client.ContentDatabase.Items[saveItem.ID, saveItem.Language, saveItem.Version];
            }

            return null;
        }

        private static void FixSaveItemFieldIfApplicable(Item item, SaveArgs.SaveField saveField)
        {
            if (ShouldEnsureFieldValue(item, saveField))
            {
                saveField.Value = Annihilator.AnnihilateSubstrings(saveField.Value);
            }
        }

        private static bool ShouldEnsureFieldValue(Item item, SaveArgs.SaveField saveField)
        {
            Field field = item.Fields[saveField.ID];
            return ShouldEnsureFieldValue(field);
        }

        private static bool ShouldEnsureFieldValue(Field field)
        {
            return field.TypeKey == "memo"
                    || field.TypeKey == "multi-line text";
        }
    }
}

Now, it’s time to insert my new Save pipeline within the SaveUI pipeline stack. I’ve done this with my /App_Config/Include/FixMultiLineTextFields.config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
	<sitecore>
		<processors>
			<saveUI>
				<processor mode="on" type="Sitecore.Sandbox.Pipelines.SaveUI.FixMultiLineTextFields, Sitecore.Sandbox" patch:after="processor[@type='Sitecore.Pipelines.Save.TightenRelativeImageLinks, Sitecore.Kernel']" />
			</saveUI>
		</processors>
	</sitecore>
</configuration>

Let’s take the above for a spin. I’ve inserted a sentence with newlines after every word within it into my Blurb Multi-Line Text field:

newlines-galore

Now, I’ve clicked save, and all newlines within my Blurb Multi-Line Text field have been annihilated:

newlines-annihilated

I would like to thank to Brian Pedersen for writing his article the other day — it served as the bedrock for this one, and kindled something in me to write the code above.

Keep smiling when coding!

Honey, I Shrunk the Content: Experiments with a Custom Sitecore Cache and Compression

A few days ago, I pondered whether there would be any utility in creating a custom Sitecore cache that compresses data before it’s stored and decompresses data upon retrieval. I wondered whether having such a cache would facilitate in conserving memory resources, thus curtailing the need to beef up servers from a memory perspective.

You’re probably thinking “Mike, who cares? Memory is cheaper today than ever before!” That thought is definitely valid.

However, I would argue we owe it to our clients and to ourselves as developers to push the envelope as much as possible by architecting our solutions to be as efficient and resource conscious as possible.

Plus, I was curious over how expensive compress/decompress operations would be for real-time requests.

The following code showcases my ventures into trying to answer these questions.

First, I defined an interface for compressors. Compressors must define methods to compress and decompress data (duh :)). I also added an additional Decompress method to cast the data object to a particular type — all for the purpose of saving client code the trouble of having to do their own type casting (yeah right, I really did it to be fancy).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Base
{
    public interface ICompressor
    {
        string Name { get; }

        byte[] Compress(object uncompressedData);

        T Decompress<T>(byte[] compressedData) where T : class;

        object Decompress(byte[] compressedData);

        long GetDataSize(object data);
    }
}

I decided to use two compression algorithms available in the System.IO.Compression namespace — Deflate and GZip. Since both algorithms within this namespace implement System.IO.Stream, I found an opportunity to use the Template method pattern.

In the spirit of this design pattern, I created an abstract class — see the CompressionStreamCompressor class below — which contains shared logic for compressing/decompressing data using methods defined by the Stream class. Subclasses only have to “fill in the blanks” by implementing the abstract method CreateNewCompressionStream — a method that returns a new instance of the compression stream represented by the subclass.

CompressionStreamCompressor:

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Base
{
    public abstract class CompressionStreamCompressor : ICompressor
    {
        protected CompressionStreamCompressor()
        {
        }

        public virtual byte[] Compress(object uncompressedData)
        {
            Assert.ArgumentNotNull(uncompressedData, "uncompressedData");
            
            byte[] uncompressedBytes = ConvertObjectToBytes(uncompressedData);
            return Compress(uncompressedBytes);
        }

        private byte[] Compress(byte[] uncompressedData)
        {
            Assert.ArgumentNotNull(uncompressedData, "uncompressedData");

            using (MemoryStream memoryStream = new MemoryStream())
            {
                using (Stream compressionStream = CreateNewCompressionStream(memoryStream, CompressionMode.Compress))
                {
                    compressionStream.Write(uncompressedData, 0, uncompressedData.Length);
                }

                return memoryStream.ToArray();
            }
        }

        public virtual T Decompress<T>(byte[] compressedData) where T : class
        {
            object decompressedData = Decompress(compressedData);
            return decompressedData as T;
        }

        public virtual object Decompress(byte[] compressedBytes)
        {
            Assert.ArgumentNotNull(compressedBytes, "compressedBytes");

            using (MemoryStream inputMemoryStream = new MemoryStream(compressedBytes))
            {
                using (Stream compressionStream = CreateNewCompressionStream(inputMemoryStream, CompressionMode.Decompress))
                {
                    using (MemoryStream outputMemoryStream = new MemoryStream())
                    {
                        compressionStream.CopyTo(outputMemoryStream);
                        return ConvertBytesToObject(outputMemoryStream.ToArray());
                    }
                }
            }
        }

        protected abstract Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode);

        public long GetDataSize(object data)
        {
            if (data == null)
                return 0;

            IFormatter formatter = new BinaryFormatter();
            long size = 0;

            using (MemoryStream memoryStream = new MemoryStream())
            {
                formatter.Serialize(memoryStream, data);
                size = memoryStream.Length;
            }

            return size;
        }

        protected static byte[] ConvertObjectToBytes(object data)
        {
            if (data == null)
                return null;

            byte[] bytes = null;
            IFormatter formatter = new BinaryFormatter();

            using (MemoryStream memoryStream = new MemoryStream())
            {
                formatter.Serialize(memoryStream, data);
                bytes = memoryStream.ToArray();
            }
            
            return bytes;
        }

        protected static object ConvertBytesToObject(byte[] bytes)
        {
            if (bytes == null)
                return null;

            object deserialized = null;
            
            using (MemoryStream memoryStream = new MemoryStream(bytes))
            {
                IFormatter formatter = new BinaryFormatter();
                memoryStream.Position = 0;
                deserialized = formatter.Deserialize(memoryStream);
            }

            return deserialized;
        }
    }
}

DeflateCompressor:

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Compression
{
    public class DeflateCompressor : CompressionStreamCompressor
    {
        private DeflateCompressor()
        {
        }

        protected override Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode)
        {
            return new DeflateStream(stream, compressionMode, false);
        }

        public static ICompressor CreateNewCompressor()
        {
            return new DeflateCompressor();
        }
    }
}

GZipCompressor:

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Compression
{
    public class GZipCompressor : CompressionStreamCompressor
    {
        private GZipCompressor()
        {
        }

        protected override Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode)
        {
            return new GZipStream(stream, compressionMode, false);
        }

        public static ICompressor CreateNewCompressor()
        {
            return new GZipCompressor();
        }
    }
}

If Microsoft ever decides to augment their arsenal of compression streams in System.IO.Compression, we could easily add new Compressor classes for these via the template method paradigm above — as long as these new compression streams implement System.IO.Stream.

After implementing the classes above, I decided I needed a “dummy” Compressor — a compressor that does not execute any compression algorithm but implements the ICompressor interface. My reasoning for doing so is to have a default Compressor be returned via a compressor factory (you will see that I created one further down), and also for ascertaining baseline benchmarks.

Plus, I figured it would be nice to have an object that closely follows the Null Object pattern — albeit in our case, we aren’t truly using this design pattern since our “Null” class is actually executing logic — so client code can avoid having null checks all over the place.

I had to go back and change my Compress and Decompress methods to be virtual in my abstract class so that I can override them within my “Null” Compressor class. The methods just take in the expected parameters and return expected types with no compression or decompression actions in the mix.

using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors
{
    class NullCompressor : CompressionStreamCompressor
    {
        private NullCompressor()
        {
        }

        public override byte[] Compress(object uncompressedData)
        {
            Assert.ArgumentNotNull(uncompressedData, "uncompressedData");
            return ConvertObjectToBytes(uncompressedData);
        }

        public override object Decompress(byte[] compressedBytes)
        {
            Assert.ArgumentNotNull(compressedBytes, "compressedBytes");
            return ConvertBytesToObject(compressedBytes);
        }

        protected override Stream CreateNewCompressionStream(Stream stream, CompressionMode compressionMode)
        {
            return null;
        }

        public static ICompressor CreateNewCompressor()
        {
            return new NullCompressor();
        }
    }
}

Next, I defined my custom Sitecore cache with its interface and settings Data transfer object.

An extremely important thing to keep in mind when creating a custom Sitecore cache is knowing you must subclass CustomCache in Sitecore.Caching — most methods that add or get from cache are protected methods, and you won’t have access to these unless you subclass this abstract class (I wasn’t paying attention when I built my cache for the first time, and had to go back to the drawing board when i discovered my code would not compile due to restricted access rights).

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Caching;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Caching.DTO
{
    public class SquashedCacheSettings
    {
        public string Name { get; set; }
        public long MaxSize { get; set; }
        public ICompressor Compressor { get; set; }
        public bool CompressionEnabled { get; set; }
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Caching.Base
{
    public interface ISquashedCache
    {
        ICompressor Compressor { get; }

        void AddToCache(object key, object value);

        T GetFromCache<T>(object key) where T : class;

        object GetFromCache(object key);
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Caching;
using Sitecore.Data;
using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.Caching.Base;
using Sitecore.Sandbox.Utilities.Caching.DTO;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;

namespace Sitecore.Sandbox.Utilities.Caching
{
    public class SquashedCache : CustomCache, ISquashedCache
    {
        private SquashedCacheSettings SquashedCacheSettings { get; set; }

        public ICompressor Compressor
        {
            get
            {
                return SquashedCacheSettings.Compressor;
            }
        }

        private SquashedCache(SquashedCacheSettings squashedCacheSettings)
            : base(squashedCacheSettings.Name, squashedCacheSettings.MaxSize)
        {
            SetSquashedCacheSettings(squashedCacheSettings);
        }

        private void SetSquashedCacheSettings(SquashedCacheSettings squashedCacheSettings)
        {
            SquashedCacheSettings = squashedCacheSettings;
        }

        public void AddToCache(object key, object value)
        {
            DataInformation dataInformation = GetCompressedDataInformation(value);
            SetObject(key, dataInformation.Data, dataInformation.Size);
        }

        private DataInformation GetCompressedDataInformation(object data)
        {
            long size = SquashedCacheSettings.Compressor.GetDataSize(data);
            return GetCompressedDataInformation(data, size);
        }

        private DataInformation GetCompressedDataInformation(object data, long size)
        {
            if (SquashedCacheSettings.CompressionEnabled)
            {
                data = SquashedCacheSettings.Compressor.Compress(data);
                size = SquashedCacheSettings.Compressor.GetDataSize(data);
            }

            return new DataInformation(data, size);
        }

        public T GetFromCache<T>(object key) where T : class
        {
            object value = GetFromCache(key);
            return value as T;
        }

        public object GetFromCache(object key)
        {
            byte[] value = (byte[])GetObject(key);
            return SquashedCacheSettings.Compressor.Decompress(value);
        }

        private T GetDecompressedData<T>(byte[] data) where T : class
        {
            if (SquashedCacheSettings.CompressionEnabled)
            {
                return SquashedCacheSettings.Compressor.Decompress<T>(data);
            }

            return data as T;
        }

        private object GetDecompressedData(byte[] data)
        {
            if (SquashedCacheSettings.CompressionEnabled)
            {
                return SquashedCacheSettings.Compressor.Decompress(data);
            }

            return data;
        }

        private struct DataInformation
        {
            public object Data;
            public long Size;

            public DataInformation(object data, long size)
            {
                Data = data;
                Size = size;
            }
        }

        public static ISquashedCache CreateNewSquashedCache(SquashedCacheSettings squashedCacheSettings)
        {
            AssertSquashedCacheSettings(squashedCacheSettings);
            return new SquashedCache(squashedCacheSettings);
        }

        private static void AssertSquashedCacheSettings(SquashedCacheSettings squashedCacheSettings)
        {
            Assert.ArgumentNotNull(squashedCacheSettings, "squashedCacheSettings");
            Assert.ArgumentNotNullOrEmpty(squashedCacheSettings.Name, "squashedCacheSettings.Name");
            Assert.ArgumentCondition(squashedCacheSettings.MaxSize > 0, "squashedCacheSettings.MaxSize", "MaxSize must be greater than zero.");
            Assert.ArgumentNotNull(squashedCacheSettings.Compressor, "squashedCacheSettings.Compressor");
        }
    }
}

You’re probably thinking “Mike, what’s up with the name SquashedCache”? Well, truth be told, I was thinking about Thanksgiving here in the United States — it’s just around the corner — and how squash is a usually found on the table for Thanksgiving dinner. The name SquashedCache just fit in perfectly in the spirit of our Thanksgiving holiday.

However, the following class names were considered.

public class SirSquishALot
{
}

public class MiniMeCache
{
}

// this became part of this post’s title instead ๐Ÿ™‚
public class HoneyIShrunkTheContent
{
}

I figured having a Factory class for compressor objects would offer a clean and central place for creating them.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Enums
{
    public enum CompressorType
    {
        Deflate,
        GZip,
        Null
    } 
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Enums;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors.Base
{
    public interface ICompressorFactory
    {
        ICompressor CreateNewCompressor(CompressorType compressorType);
    }
}
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Enums;

namespace Sitecore.Sandbox.Utilities.Compression.Compressors
{
    public class CompressorFactory : ICompressorFactory
    {
        private CompressorFactory()
        {
        }

        public ICompressor CreateNewCompressor(CompressorType compressorType)
        {
            if (compressorType == CompressorType.Deflate)
            {
                return DeflateCompressor.CreateNewCompressor();
            }
            else if (compressorType == CompressorType.GZip)
            {
                return GZipCompressor.CreateNewCompressor();
            }

            return NullCompressor.CreateNewCompressor();
        }

        public static ICompressorFactory CreateNewCompressorFactory()
        {
            return new CompressorFactory();
        }
    }
}

Now, it’s time to test everything above and look at some statistics. I basically created a sublayout containing some repeaters to highlight how each compressor performed against the others — including the “Null” compressor which serves as the baseline.

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="Squash Test.ascx.cs" Inherits="Sitecore650rev120706.layouts.sublayouts.Squash_Test" %>

<div>
    <h2><u>Compression Test</u></h2>
    Uncompressed Size: <asp:Literal ID="litUncompressedSize" runat="server" /> bytes
    <asp:Repeater ID="rptCompressionTest" runat="server">
        <HeaderTemplate>
            <br /><br />
        </HeaderTemplate>
        <ItemTemplate>
                <div>
                    Compressed Size using <%# Eval("TestName")%>: <%# Eval("CompressedSize")%> bytes <br />
                    Compression Ratio using <%# Eval("TestName")%>: <%# Eval("CompressionRatio","{0:p}") %> of original size
                </div>
        </ItemTemplate>
        <SeparatorTemplate>
            <br />
        </SeparatorTemplate>
    </asp:Repeater>
</div>

<asp:Repeater ID="rptAddToCacheTest" runat="server">
    <HeaderTemplate>
        <div>
            <h2><u>AddToCache() Test</u></h2>
    </HeaderTemplate>
    <ItemTemplate>
            <div>
                AddToCache() Elasped Time for <%# Eval("TestName")%>: <%# Eval("ElapsedMilliseconds")%> ms
            </div>
    </ItemTemplate>
    <FooterTemplate>
        </div>
    </FooterTemplate>
</asp:Repeater>

<asp:Repeater ID="rptGetFromCacheTest" runat="server">
    <HeaderTemplate>
        <div>
            <h2><u>GetFromCache() Test</u></h2>
    </HeaderTemplate>
    <ItemTemplate>
            <div>
                GetFromCache() Elasped Time for <%# Eval("TestName")%>: <%# Eval("ElapsedMilliseconds")%> ms
            </div>
    </ItemTemplate>
    <FooterTemplate>
        </div>
    </FooterTemplate>
</asp:Repeater>

<asp:Repeater ID="rptDataIntegrityTest" runat="server">
    <HeaderTemplate>
        <div>
            <h2><u>Data Integrity Test</u></h2>
    </HeaderTemplate>
    <ItemTemplate>
            <div>
                Data Retrieved From <%# Eval("TestName")%> Equals Original: <%# Eval("AreEqual")%>
            </div>
    </ItemTemplate>
    <FooterTemplate>
        </div>
    </FooterTemplate>
</asp:Repeater>

In my code-behind, I’m grabbing the full text of the book War and Peace by Leo Tolstoy for testing purposes. The full text of this copy of the book is over 3.25 MB.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Net;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;

using Sitecore;

using Sitecore.Sandbox.Utilities.Caching;
using Sitecore.Sandbox.Utilities.Caching.Base;
using Sitecore.Sandbox.Utilities.Caching.DTO;

using Sitecore.Sandbox.Utilities.Compression.Compressors;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Base;
using Sitecore.Sandbox.Utilities.Compression.Compressors.Enums;

namespace Sitecore650rev120706.layouts.sublayouts
{
    public partial class Squash_Test : System.Web.UI.UserControl
    {
        private const string CacheKey = "War and Peace";
        private const string WarAndPeaceUrl = "http://www.gutenberg.org/cache/epub/2600/pg2600.txt";
        private const string MaxSize = "50MB";

        private ICompressorFactory _Factory;
        private ICompressorFactory Factory
        {
            get
            {
                if(_Factory == null)
                    _Factory = CompressorFactory.CreateNewCompressorFactory();

                return _Factory;
            }
        }

        private IEnumerable<ISquashedCache> _SquashedCaches;
        private IEnumerable<ISquashedCache> SquashedCaches
        {
            get
            {
                if(_SquashedCaches == null)
                    _SquashedCaches = CreateAllSquashedCaches();

                return _SquashedCaches;
            }
        }

        private string _WarAndPeaceText;
        private string WarAndPeaceText
        {
            get
            {
                if (string.IsNullOrEmpty(_WarAndPeaceText))
                    _WarAndPeaceText = GetWarAndPeaceText();

                return _WarAndPeaceText;
            }
        }

        private long _UncompressedSize;
        private long UncompressedSize
        {
            get
            {
                if(_UncompressedSize == 0)
                    _UncompressedSize = GetUncompressedSize();

                return _UncompressedSize;
            }
        }


        private Stopwatch _Stopwatch;
        private Stopwatch Stopwatch
        {
            get
            {
                if (_Stopwatch == null)
                    _Stopwatch = Stopwatch.StartNew();

                return _Stopwatch;
            }
        }

        protected void Page_Load(object sender, EventArgs e)
        {
            SetUncomopressedSizeLiteral();
            BindAllRepeaters();
        }

        private void SetUncomopressedSizeLiteral()
        {
            litUncompressedSize.Text = UncompressedSize.ToString();
        }

        private void BindAllRepeaters()
        {
            BindCompressionTestRepeater();
            BindAddToCacheTestRepeater();
            BindGetFromCacheTestRepeater();
            BindDataIntegrityTestRepeater();
        }

        private void BindCompressionTestRepeater()
        {
            rptCompressionTest.DataSource = GetCompressionTestData();
            rptCompressionTest.DataBind();
        }

        private IEnumerable<CompressionRatioAtom> GetCompressionTestData()
        {
            IList<CompressionRatioAtom> compressionRatioAtoms = new List<CompressionRatioAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                byte[] compressed = squashedCache.Compressor.Compress(WarAndPeaceText);
                long compressedSize = squashedCache.Compressor.GetDataSize(compressed);

                CompressionRatioAtom compressionRatioAtom = new CompressionRatioAtom
                {
                    TestName = squashedCache.Name,
                    CompressedSize = compressedSize,
                    CompressionRatio = ((decimal)compressedSize / UncompressedSize)
                };

                compressionRatioAtoms.Add(compressionRatioAtom);
            }

            return compressionRatioAtoms;
        }

        private void BindAddToCacheTestRepeater()
        {
            rptAddToCacheTest.DataSource = GetAddToCacheTestData();
            rptAddToCacheTest.DataBind();
        }

        private IEnumerable<TimeTestAtom> GetAddToCacheTestData()
        {
            IList<TimeTestAtom> timeTestAtoms = new List<TimeTestAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                Stopwatch.Start();
                squashedCache.AddToCache(CacheKey, WarAndPeaceText);
                Stopwatch.Stop();

                TimeTestAtom timeTestAtom = new TimeTestAtom
                {
                    TestName = squashedCache.Name,
                    ElapsedMilliseconds = Stopwatch.Elapsed.TotalMilliseconds
                };

                timeTestAtoms.Add(timeTestAtom);
            }

            return timeTestAtoms;
        }

        private void BindGetFromCacheTestRepeater()
        {
            rptGetFromCacheTest.DataSource = GetGetFromCacheTestData();
            rptGetFromCacheTest.DataBind();
        }

        private IEnumerable<TimeTestAtom> GetGetFromCacheTestData()
        {
            IList<TimeTestAtom> timeTestAtoms = new List<TimeTestAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                Stopwatch.Start();
                squashedCache.GetFromCache<string>(CacheKey);
                Stopwatch.Stop();

                TimeTestAtom timeTestAtom = new TimeTestAtom
                {
                    TestName = squashedCache.Name,
                    ElapsedMilliseconds = Stopwatch.Elapsed.TotalMilliseconds
                };

                timeTestAtoms.Add(timeTestAtom);
            }

            return timeTestAtoms;
        }

        private void BindDataIntegrityTestRepeater()
        {
            rptDataIntegrityTest.DataSource = GetDataIntegrityTestData();
            rptDataIntegrityTest.DataBind();
        }

        private IEnumerable<DataIntegrityTestAtom> GetDataIntegrityTestData()
        {
            IList<DataIntegrityTestAtom> dataIntegrityTestAtoms = new List<DataIntegrityTestAtom>();

            foreach (ISquashedCache squashedCache in SquashedCaches)
            {
                string cachedContent = squashedCache.GetFromCache<string>(CacheKey);

                DataIntegrityTestAtom dataIntegrityTestAtom = new DataIntegrityTestAtom
                {
                    TestName = squashedCache.Name,
                    AreEqual = cachedContent == WarAndPeaceText
                };

                dataIntegrityTestAtoms.Add(dataIntegrityTestAtom);
            }

            return dataIntegrityTestAtoms;
            
        }

        private IEnumerable<ISquashedCache> CreateAllSquashedCaches()
        {
            IList<ISquashedCache> squashedCaches = new List<ISquashedCache>();
            squashedCaches.Add(CreateNewNullSquashedCache());
            squashedCaches.Add(CreateNewDeflateSquashedCache());
            squashedCaches.Add(CreateNewGZipSquashedCache());
            return squashedCaches;
        }

        private ISquashedCache CreateNewNullSquashedCache()
        {
            return CreateNewSquashedCache("Null Cache", MaxSize, CompressorType.Null);
        }

        private ISquashedCache CreateNewDeflateSquashedCache()
        {
            return CreateNewSquashedCache("Deflate Cache", MaxSize, CompressorType.Deflate);
        }

        private ISquashedCache CreateNewGZipSquashedCache()
        {
            return CreateNewSquashedCache("GZip Cache", MaxSize, CompressorType.GZip);
        }

        private ISquashedCache CreateNewSquashedCache(string cacheName, string maxSize, CompressorType compressorType)
        {
            SquashedCacheSettings squashedCacheSettings = CreateNewSquashedCacheSettings(cacheName, maxSize, compressorType);
            return SquashedCache.CreateNewSquashedCache(squashedCacheSettings);
        }

        private SquashedCacheSettings CreateNewSquashedCacheSettings(string cacheName, string maxSize, CompressorType compressorType)
        {
            return new SquashedCacheSettings
            {
                Name = cacheName,
                MaxSize = StringUtil.ParseSizeString(maxSize),
                Compressor = Factory.CreateNewCompressor(compressorType),
                CompressionEnabled = true
            };
        }

        private static string GetWarAndPeaceText()
        {
            WebRequest webRequest = (HttpWebRequest)WebRequest.Create(WarAndPeaceUrl);
            HttpWebResponse httpWebResponse = (HttpWebResponse)webRequest.GetResponse();
            
            string warAndPeaceText = string.Empty;

            using (StreamReader streamReader = new StreamReader(httpWebResponse.GetResponseStream()))
            {
                warAndPeaceText = streamReader.ReadToEnd();
            }

            return warAndPeaceText;
        }

        private long GetUncompressedSize()
        {
            return SquashedCaches.FirstOrDefault().Compressor.GetDataSize(WarAndPeaceText);
        }

        private class CompressionRatioAtom
        {
            public string TestName { get; set; }
            public long CompressedSize { get; set; }
            public decimal CompressionRatio { get; set; }
        }

        private class TimeTestAtom
        {
            public string TestName { get; set; }
            public double ElapsedMilliseconds { get; set; }
        }

        private class DataIntegrityTestAtom
        {
            public string TestName { get; set; }
            public bool AreEqual { get; set; }
        }
    }
}

From my screenshot below, we can see that both compression algorithms compress War and Peace down to virtually the same ratio of the original size.

Plus, the add operations are quite expensive for the true compressors over the “Null” compressor — GZip yielding the worst performance next to the others.

However, the get operations don’t appear to be that far off from each other — albeit I cannot truly conclude this I am only showing one test outcome here. It would be best to make such assertions after performing load testing to truly ascertain the performance of these algorithms. My ventures here were only to acquire a rough sense of how these algorithms perform.

In the future, I may want to explore how other compression algorithms stack up next to the two above. LZF — a real-time compression algorithm that promises good performance — is one algorithm I’m considering.

I will let you know my results if I take this algorithm for a dry run.

Gobble Gobble!