Have you ever seen a Standard Values token — $name is an example of a Standard Values token — in an item’s field, and ask yourself “how the world did that get there”, or alternatively, “what can be done to replace it with what should be there?”

This can occur when tokens are added to an item’s template’s Standard Values node after the item was created — tokens are expanded once an item is created, not after the fact.

John West wrote a blog article highlighting one solution for eradicating this issue by using the Sitecore Rules Engine.

In this post, I am proposing a completely different solution — one that empowers content authors to expand unexpanded tokens by clicking a link in a custom content editor warning box.

First, I created a series of utility objects that ascertain whether “Source” objects contain substrings that we are interested in. All implement the following interface:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public interface ISubstringsChecker<T>
    {
        T Source { get; set; }
        
        IEnumerable<string> Substrings { get; set; }

        bool ContainsSubstrings();
    }
}

All of our “checkers” will have a “Source” object, a collection of substrings to look for, and a method to convey to calling code whether a substring in the collection of substrings had been found in the “Source” object.

I decided to create a base abstract class with some abstract methods along with one method to serve as a hook for asserting the Source object — albeit I did not use this method anywhere in my solution (come on Mike, you’re forgetting YAGNI — let’s get with the program).

Utimately, I am employing the template method pattern — all subclasses of this abstract base class will just fill in the defined abstract stubs, and the parent class will take care of the rest:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Sitecore.Sandbox.Utilities.StringUtilities.Base
{
    public abstract class SubstringsChecker<T> : ISubstringsChecker<T>
    {
        public T Source { get; set; }

        public virtual IEnumerable<string> Substrings { get; set; }

        protected SubstringsChecker(T source)
        {
            SetSource(source);
        }

        private void SetSource(T source)
        {
            AssertSource(source);
            Source = source;
        }

        protected virtual void AssertSource(T source)
        {
            // a hook for subclasses to assert Source objects
        }

        public bool ContainsSubstrings()
        {
            if (CanDoCheck())
            {
                return DoCheck();
            }

            return false;
        }

        protected abstract bool CanDoCheck();
        protected abstract bool DoCheck();
    }
}

The first “checker” I created will find substrings in a string. It made sense for me to start here, especially when we are ultimately checking strings at the most atomic level in our series of “checker” utility objects.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class StringSubstringsChecker : SubstringsChecker<string>
    {
        private StringSubstringsChecker(IEnumerable<string> substrings)
            : this(null, substrings)
        {
        }

        private StringSubstringsChecker(string source, IEnumerable<string> substrings)
            : base(source)
        {
            SetSubstrings(substrings);
        }

        private void SetSubstrings(IEnumerable<string> substrings)
        {
            AssertSubstrings(substrings);
            Substrings = substrings;
        }

        private static void AssertSubstrings(IEnumerable<string> substrings)
        {
            Assert.ArgumentNotNull(substrings, "substrings");
            Assert.ArgumentCondition(substrings.Any(), "substrings", "substrings must contain as at least one string!");
        }

        protected override bool CanDoCheck()
        {
            return !string.IsNullOrEmpty(Source);
        }

        protected override bool DoCheck()
        {
            Assert.ArgumentNotNullOrEmpty(Source, "Source");

            foreach (string substring in Substrings)
            {
                if (Source.Contains(substring))
                {
                    return true;
                }
            }

            return false;
        }

        public static ISubstringsChecker<string> CreateNewStringSubstringsContainer(IEnumerable<string> substrings)
        {
            return new StringSubstringsChecker(substrings);
        }

        public static ISubstringsChecker<string> CreateNewStringSubstringsContainer(string source, IEnumerable<string> substrings)
        {
            return new StringSubstringsChecker(substrings);
        }
    }
}

The next level up from strings would naturally be Fields. I designed this “checker” to consume an instance of the string “checker” defined above — all for the purposes of reuse. The field “checker” delegates calls to its string “checker” instance.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Data.Fields;
using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class FieldSubstringsChecker : SubstringsChecker<Field>
    {
        private ISubstringsChecker<string> StringSubstringsChecker { get; set; }

        public override IEnumerable<string> Substrings 
        {
            get
            {
                return StringSubstringsChecker.Substrings;
            }
            set
            {
                StringSubstringsChecker.Substrings = value;
            }
        }

        private FieldSubstringsChecker(ISubstringsChecker<string> stringSubstringsChecker)
            : this(null, stringSubstringsChecker)
        {
        }

        private FieldSubstringsChecker(Field source, ISubstringsChecker<string> stringSubstringsChecker)
            : base(source)
        {
            SetStringSubstringsChecker(stringSubstringsChecker);
        }

        private void SetStringSubstringsChecker(ISubstringsChecker<string> stringSubstringsChecker)
        {
            Assert.ArgumentNotNull(stringSubstringsChecker, "stringSubstringsChecker");
            StringSubstringsChecker = stringSubstringsChecker;
        }

        protected override bool CanDoCheck()
        {
            return Source != null;
        }

        protected override bool DoCheck()
        {
            Assert.ArgumentNotNull(Source, "Source");
            StringSubstringsChecker.Source = Source.Value;
            return StringSubstringsChecker.ContainsSubstrings();
        }

        public static ISubstringsChecker<Field> CreateNewFieldSubstringsChecker(ISubstringsChecker<string> stringSubstringsChecker)
        {
            return new FieldSubstringsChecker(stringSubstringsChecker);
        }

        public static ISubstringsChecker<Field> CreateNewFieldSubstringsChecker(Field source, ISubstringsChecker<string> stringSubstringsChecker)
        {
            return new FieldSubstringsChecker(source, stringSubstringsChecker);
        }
    }
}

If I were to ask you what the next level up in our series of “checkers” would be, I hope your answer would be Items. Below is a “checker” that uses an Item as its “Source” object, and delegates calls to an instance of a field “checker”.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;

namespace Sitecore.Sandbox.Utilities.StringUtilities
{
    public class ItemSubstringsChecker : SubstringsChecker<Item>
    {
        private ISubstringsChecker<Field> FieldSubstringsChecker { get; set; }

        public override IEnumerable<string> Substrings 
        {
            get
            {
                return FieldSubstringsChecker.Substrings;
            }
            set
            {
                FieldSubstringsChecker.Substrings = value;
            }
        }

        private ItemSubstringsChecker(ISubstringsChecker<Field> fieldSubstringsChecker)
            : this(null, fieldSubstringsChecker)
        {
        }

        private ItemSubstringsChecker(Item source, ISubstringsChecker<Field> fieldSubstringsChecker)
            : base(source)
        {
            SetFieldSubstringsChecker(fieldSubstringsChecker);
        }

        private void SetFieldSubstringsChecker(ISubstringsChecker<Field> fieldSubstringsChecker)
        {
            Assert.ArgumentNotNull(fieldSubstringsChecker, "fieldSubstringsChecker");
            FieldSubstringsChecker = fieldSubstringsChecker;
        }

        protected override bool CanDoCheck()
        {
            return Source != null && Source.Fields != null && Source.Fields.Any();
        }

        protected override bool DoCheck()
        {
            Assert.ArgumentNotNull(Source, "Source");
            bool containsSubstrings = false;

            for (int i = 0; !containsSubstrings && i < Source.Fields.Count; i++)
            {
                containsSubstrings = containsSubstrings || DoesFieldContainSubstrings(Source.Fields[i]);
            }

            return containsSubstrings;
        }

        private bool DoesFieldContainSubstrings(Field field)
        {
            FieldSubstringsChecker.Source = field;
            return FieldSubstringsChecker.ContainsSubstrings();
        }

        public static ISubstringsChecker<Item> CreateNewItemSubstringsChecker(ISubstringsChecker<Field> fieldSubstringsChecker)
        {
            return new ItemSubstringsChecker(fieldSubstringsChecker);
        }

        public static ISubstringsChecker<Item> CreateNewItemSubstringsChecker(Item source, ISubstringsChecker<Field> fieldSubstringsChecker)
        {
            return new ItemSubstringsChecker(source, fieldSubstringsChecker);
        }
    }
}

Next, I wrote code for a content editor warning box. I’m am completely indebted to .NET Reflector in helping out on this front — I looked at other content editor warning pipelines in Sitecore.Pipelines.GetContentEditorWarnings within Sitecore.Kernel to see how this is done:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Configuration;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Pipelines.GetContentEditorWarnings;

using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities;

namespace Sitecore.Sandbox.Pipelines.GetContentEditorWarnings
{
    public class HasUnexpandedTokens
    {
        private static readonly ISubstringsChecker<Item> SubstringsChecker = CreateNewItemSubstringsChecker();

        public void Process(GetContentEditorWarningsArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if (CanExpandTokens(args.Item))
            {
                AddHasUnexpandedTokensWarning(args);
            }
        }

        private static bool CanExpandTokens(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            return !IsStandardValues(item) && DoesItemContainUnexpandedTokens(item);
        }

        private static bool DoesItemContainUnexpandedTokens(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            SubstringsChecker.Source = item;
            return SubstringsChecker.ContainsSubstrings();
        }

        private static bool IsStandardValues(Item item)
        {
            return item.Template.StandardValues.ID == item.ID;
        }

        private static void AddHasUnexpandedTokensWarning(GetContentEditorWarningsArgs args)
        {
            GetContentEditorWarningsArgs.ContentEditorWarning warning = args.Add();
            warning.Title = Translate.Text("Some fields contain unexpanded tokens.");
            warning.Text = Translate.Text("To expand tokens, click Expand Tokens.");
            warning.AddOption(Translate.Text("Expand Tokens"), "item:expandtokens");
        }

        private static ISubstringsChecker<Item> CreateNewItemSubstringsChecker()
        {
            return ItemSubstringsChecker.CreateNewItemSubstringsChecker(CreateNewFieldSubstringsChecker());
        }

        private static ISubstringsChecker<Field> CreateNewFieldSubstringsChecker()
        {
            return FieldSubstringsChecker.CreateNewFieldSubstringsChecker(CreateNewStringSubstringsChecker());
        }

        private static ISubstringsChecker<string> CreateNewStringSubstringsChecker()
        {
            return StringSubstringsChecker.CreateNewStringSubstringsContainer(GetTokens());
        }

        private static IEnumerable<string> GetTokens()
        {
            return Factory.GetStringSet("tokens/token");
        }
    }
}

In my pipeline above, I am pulling a collection of token names from Sitecore configuration — these are going to be defined in a patch include file below. I had to go down this road since these tokens are not publically exposed in the Sitecore API.

Plus, we should only allow for the expansion of tokens when not on the Standard values item — it wouldn’t make much sense to expand these tokens here.

Since I’m referencing a new command in the above pipeline — a command that I’ve named “item:expandtokens” — it’s now time to create that new command:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Shell.Framework.Commands;

using Sitecore.Sandbox.Utilities.StringUtilities;
using Sitecore.Sandbox.Utilities.StringUtilities.Base;
using Sitecore.Sandbox.Utilities.StringUtilities.DTO;

namespace Sitecore.Sandbox.Commands
{
    public class ExpandTokens : Command
    {
        public override void Execute(CommandContext commandContext)
        {
            if (!DoesCommandContextContainOneItem(commandContext))
            {
                return;
            }

            ExpandTokensInItem(GetCommandContextItem(commandContext));
        }

        private static void ExpandTokensInItem(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            item.Fields.ReadAll();
            item.Editing.BeginEdit();
            ExpandTokensViaMasterVariablesReplacer(item);
            item.Editing.EndEdit();
        }

        private static void ExpandTokensViaMasterVariablesReplacer(Item item)
        {
            MasterVariablesReplacer masterVariablesReplacer = Factory.GetMasterVariablesReplacer();
            masterVariablesReplacer.ReplaceItem(item);
        }

        public override CommandState QueryState(CommandContext commandContext)
        {
            if (ShouldHideCommand(commandContext))
            {
                return CommandState.Hidden;
            }
            
            if (ShouldDisableCommand(commandContext))
            {
                return CommandState.Disabled;
            }

            return base.QueryState(commandContext);
        }

        private bool ShouldHideCommand(CommandContext commandContext)
        {
            Assert.ArgumentNotNull(commandContext, "commandContext");
            return !DoesCommandContextContainOneItem(commandContext) 
                    || !HasField(GetCommandContextItem(commandContext), FieldIDs.ReadOnly);
        }

        private static bool ShouldDisableCommand(CommandContext commandContext)
        {
            AssertCommandContextItems(commandContext);
            Item item = GetCommandContextItem(commandContext);

            return !item.Access.CanWrite()
                    || Command.IsLockedByOther(item)
                    || !Command.CanWriteField(item, FieldIDs.ReadOnly);
        }

        private static Item GetCommandContextItem(CommandContext commandContext)
        {
            Assert.ArgumentNotNull(commandContext, "commandContext");
            Assert.ArgumentNotNull(commandContext.Items, "commandContext.Items");
            return commandContext.Items.FirstOrDefault();
        }

        private static bool DoesCommandContextContainOneItem(CommandContext commandContext)
        {
            AssertCommandContextItems(commandContext);
            return commandContext.Items.Length == 1;
        }

        private static void AssertCommandContextItems(CommandContext commandContext)
        {
            Assert.ArgumentNotNull(commandContext, "commandContext");
            Assert.ArgumentNotNull(commandContext.Items, "commandContext.Items");
        }
    }
}

This new command just uses an instance of Sitecore.Data.MasterVariablesReplacer to expand Standard Values tokens on our item.

I had to register this command in the /App_Config/Commands.config file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
	<! -- A bunch of commands defined here -->
	
	<command name="item:expandtokens" type="Sitecore.Sandbox.Commands.ExpandTokens,Sitecore.Sandbox" />
	
	<! -- A bunch more defined here too -->
</configuration>

I then put all the pieces together using a patch include config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <getContentEditorWarnings>
        <processor type="Sitecore.Sandbox.Pipelines.GetContentEditorWarnings.HasUnexpandedTokens, Sitecore.Sandbox"/>
      </getContentEditorWarnings>
    </pipelines>
    <tokens>
      <token>$name</token>
      <token>$id</token>
      <token>$parentid</token>
      <token>$parentname</token>
      <token>$date</token>
      <token>$time</token>
      <token>$now</token>
    </tokens>
  </sitecore>
</configuration>

Let’s create an item for testing:

As you can see, the item’s Title field was populated automatically during item creation — the $name token lives on this item’s template’s Standard Values item in Sitecore and was expanded when we created our test item.

Let’s create the problem we are trying to solve by adding new tokens to our test item’s template’s Standard Values node:

As you can see, these new tokens appear in fields in our test item. However, don’t fret — we now have a way to fix this issue. 🙂

I clicked on the ‘Expand Tokens’ link, and saw the following:

Hopefully, this post has given you another weapon to add to your arsenal for solving the unexpanded tokens issue on existing items in Sitecore.

If you find another solution, please drop me a line — I would love to hear about it.