Home » Processors (Page 3)

Category Archives: Processors

Restrict Certain Files from Being Uploaded Through Web Forms for Marketers Forms in Sitecore: an Alternative Approach

This past weekend I noticed the <formUploadFile> pipeline in Web Forms for Marketers (WFFM), and wondered whether I could create an alternative solution to the one I had shared in this post — I had built a custom WFFM field type to prevent certain files from being uploaded through WFFM forms.

After some tinkering, I came up the following class to serve as a <formUploadFile> pipeline processor:

using System;
using System.Collections.Generic;
using System.Web;

using Sitecore.Diagnostics;

using Sitecore.Form.Core.Pipelines.FormUploadFile;

namespace Sitecore.Sandbox.Form.Pipelines.FormUploadFile
{
    public class CheckMimeTypesNotAllowed
    {
        static CheckMimeTypesNotAllowed()
        {
            MimeTypesNotAllowed = new List<string>();
        }

        public void Process(FormUploadFileArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            string mimeType = GetMimeType(args.File.FileName);
            if (IsMimeTypeAllowed(mimeType))
            {
                return;
            }

            throw new Exception(string.Format("Uploading a file with MIME type {0} is not allowed", mimeType));
        }

        protected virtual string GetMimeType(string fileName)
        {
            Assert.ArgumentNotNullOrEmpty(fileName, "fileName");
            return MimeMapping.GetMimeMapping(fileName);
        }

        protected virtual bool IsMimeTypeAllowed(string mimeType)
        {
            foreach (string mimeTypeNotAllowed in MimeTypesNotAllowed)
            {
                if (AreEqualIgnoreCase(mimeTypeNotAllowed, mimeType))
                {
                    return false;
                }    
            }

            return true;
        }

        private static bool AreEqualIgnoreCase(string stringOne, string stringTwo)
        {
            return string.Equals(stringOne, stringTwo, StringComparison.CurrentCultureIgnoreCase);
        }

        private static void AddMimeTypeNotAllowed(string mimeType)
        {
            if (string.IsNullOrWhiteSpace(mimeType) || MimeTypesNotAllowed.Contains(mimeType))
            {
                return;
            }

            MimeTypesNotAllowed.Add(mimeType);
        }

        private static IList<string> MimeTypesNotAllowed { get; set; }
    }
}

The class above adds restricted MIME types to a list — these MIME types are defined in a configuration file shown below — and checks to see if the uploaded file’s MIME type is in the restricted list. If it is restricted, an exception is thrown with some information — throwing an exception in WFFM prevents the form from being submitted.

MIME types are inferred using System.Web.MimeMapping.GetMimeMapping(string fileName), a method that is new in .NET 4.5 (this solution will not work in older versions of .NET).

I then registered the class above as a <formUploadFile> pipeline processor via the following configuration file:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:x="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <formUploadFile>
        <processor patch:before="processor[@type='Sitecore.Form.Core.Pipelines.FormUploadFile.Save, Sitecore.Forms.Core']"
                   type="Sitecore.Sandbox.Form.Pipelines.FormUploadFile.CheckMimeTypesNotAllowed">
          <mimeTypesNotAllowed hint="list:AddMimeTypeNotAllowed">
            <mimeType>application/octet-stream</mimeType>
          </mimeTypesNotAllowed >
        </processor>  
      </formUploadFile>
    </pipelines>
  </sitecore>
</configuration>

In case you are wondering, the MIME type being passed to the processor in the configuration file is for executables.

Let’s see how we did. 😉

I whipped up a test form, and pulled it up in a browser:

form-for-testing

I then selected an executable:

select-executable

I saw this after an attempt to submit the form:

form-upload-exception

And my log file expressed why:

exception-in-log

I then copied a jpeg into my test folder, and selected it to be uploaded:

select-jpg

After clicking the submit button, I was given a nice confirmation message:

success-upload

There is one problem with the solution above that I would like to point out: it does not address the issue of file extensions being changed. I could not solve this problem since the MIME type of the file being uploaded cannot be determined from the Sitecore.Form.Core.Media.PostedFile instance set in the arguments object: there is no property for it. 😦

If you know of another way to determine MIME types on Sitecore.Form.Core.Media.PostedFile instances, or have other ideas for restricting certain files from being uploaded through WFFM, please share in a comment.

Add ‘Has Content In Language’ Property to Sitecore Item Web API Responses

The other day I had read a forum thread on SDN where the poster had asked whether one could determine if content returned from the Sitecore Item Web API for an Item was the actual content for the Item in the requested language.

I was intrigued by this question because I would have assumed that no results would be returned for the Item when it does not have content in the requested language but that is not the case: I had replicated what the poster had seen.

As a workaround, I built the following class to serve as an <itemWebApiGetProperties> pipeline processor which sets a property in the response indicating whether the Item has content in the requested language (check out my previous post on adding additional properties to Sitecore Item Web API responses for more information on this topic):

using System.Collections.Generic;
using System.Linq;

using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.ItemWebApi.Pipelines.GetProperties;

namespace Sitecore.Sandbox.ItemWebApi.Pipelines.GetProperties
{
    public class SetHasContentInLanguageProperty : GetPropertiesProcessor
    {
        public override void Process(GetPropertiesArgs arguments)
        {
            Assert.ArgumentNotNull(arguments, "arguments");
            Assert.ArgumentNotNull(arguments.Item, "arguments.Item");
            arguments.Properties.Add("HasContentInLanguage", IsLanguageInCollection(arguments.Item.Languages, arguments.Item.Language));
        }

        private static bool IsLanguageInCollection(IEnumerable<Language> languages, Language language)
        {
            Assert.ArgumentNotNull(languages, "languages");
            Assert.ArgumentNotNull(language, "language");
            return languages.Any(lang => lang == language);
        }
    }
}

The code in the above class checks to see if the Item has content in the requested language — the latter is set in the Language property of the Item instance, and the Languages property contains a list of all languages it has content for.

I then added the above pipeline processor via the following configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <itemWebApiGetProperties>
        <processor patch:after="processor[@type='Sitecore.ItemWebApi.Pipelines.GetProperties.GetProperties, Sitecore.ItemWebApi']"
            type="Sitecore.Sandbox.ItemWebApi.Pipelines.GetProperties.SetHasContentInLanguageProperty, Sitecore.Sandbox" />
      </itemWebApiGetProperties>
    </pipelines>
  </sitecore>
</configuration>

Let’s see how this works!

I first created an Item for testing:

content-in-language-test

This Item only has content in English:

content-in-language-test-en

I then toggled my Sitecore Item Web API configuration to allow for anonymous access so that I can make requests in my browser, and made a request for the test Item in English:

english-has-content

The Item does have content in English, and this is denoted by the ‘HasContentInLanguage’ property.

I then made a request for the Item in French:

french-does-not-have-content

As expected, the ‘HasContentInLanguage’ is false since the Item does not have content in French.

If you have any questions or thoughts on this, please drop a comment.

Export to CSV in the Form Reports of Sitecore’s Web Forms for Marketers

The other day I was poking around Sitecore.Forms.Core.dll — this is one of the assemblies that comes with Web Forms for Marketers (what, you don’t randomly look at code in the Sitecore assemblies? 😉 ) — and decided to check out how the export functionality of the Form Reports work.

Once I felt I understood how the export code functions, I decided to take a stab at building my own custom export: functionality to export to CSV, and built the following class to serve as a pipeline processor to wedge Form Reports data into CSV format:

using System;
using System.Collections.Generic;
using System.Linq;

using Sitecore.Diagnostics;
using Sitecore.Form.Core.Configuration;
using Sitecore.Form.Core.Pipelines.Export;
using Sitecore.Forms.Data;
using Sitecore.Jobs;

namespace Sitecore.Sandbox.Form.Core.Pipelines.Export.Csv
{
    public class ExportToCsv
    {
        public void Process(ExportArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            LogInfo();
            args.Result = GenerateCsv(args.Packet.Entries);
        }

        protected virtual void LogInfo()
        {
            Job job = Context.Job;
            if (job != null)
            {
                job.Status.LogInfo(ResourceManager.Localize("EXPORTING_DATA"));
            }
        }

        private string GenerateCsv(IEnumerable<IForm> forms)
        {
            return string.Join(Environment.NewLine, GenerateAllCsvRows(forms));
        }

        protected virtual IEnumerable<string> GenerateAllCsvRows(IEnumerable<IForm> forms)
        {
            Assert.ArgumentNotNull(forms, "forms");
            IList<string> rows = new List<string>();
            rows.Add(GenerateCsvHeader(forms.FirstOrDefault()));
            foreach (IForm form in forms)
            {
                string row = GenerateCsvRow(form);
                if (!string.IsNullOrWhiteSpace(row))
                {
                    rows.Add(row);
                }
            }

            return rows;
        }

        protected virtual string GenerateCsvHeader(IForm form)
        {
            Assert.ArgumentNotNull(form, "form");
            return string.Join(",", form.Field.Select(field => field.FieldName));
        }

        protected virtual string GenerateCsvRow(IForm form)
        {
            Assert.ArgumentNotNull(form, "form");
            return string.Join(",", form.Field.Select(field => field.Value));
        }
    }
}

There really isn’t anything magical happening in the code above. The code creates a string of comma-separated values for each row of entries in args.Packet.Entries, and puts these plus a CSV header into a collection of strings.

Once all rows have been placed into a collection of strings, they are munged together on the newline character ultimately creating a multi-row CSV string. This CSV string is then set on the Result property of the ExportArgs instance.

Now we need a way to invoke a pipeline that contains the above class as a processor, and the following command does just that:

using System.Collections.Specialized;

using Sitecore.Diagnostics;
using Sitecore.Forms.Core.Commands.Export;
using Sitecore.Form.Core.Configuration;
using Sitecore.Shell.Framework.Commands;

namespace Sitecore.Sandbox.Forms.Core.Commands.Export
{
    public class Export : ExportToXml
    {
        protected override void AddParameters(NameValueCollection parameters)
        {
            parameters["filename"] = FileName;
            parameters["contentType"] = MimeType;
        }

        public override void Execute(CommandContext context)
        {
            SetProperties(context);
            base.Execute(context);
        }

        private void SetProperties(CommandContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            Assert.ArgumentNotNull(context.Parameters, "context.Parameters");
            Assert.ArgumentNotNullOrEmpty(context.Parameters["fileName"], "context.Parameters[\"fileName\"]");
            Assert.ArgumentNotNullOrEmpty(context.Parameters["mimeType"], "context.Parameters[\"mimeType\"]");
            Assert.ArgumentNotNullOrEmpty(context.Parameters["exportPipeline"], "context.Parameters[\"exportPipeline\"]");
            Assert.ArgumentNotNullOrEmpty(context.Parameters["progressDialogTitle"], "context.Parameters[\"progressDialogTitle\"]");
            FileName = context.Parameters["fileName"];
            MimeType = context.Parameters["mimeType"];
            ExportPipeline = context.Parameters["exportPipeline"];
            ProgressDialogTitle = context.Parameters["progressDialogTitle"];
        }

        protected override string GetName()
        {
            return ProgressDialogTitle;
        }

        protected override string GetProcessorName()
        {
            return ExportPipeline;
        }

        private string FileName { get; set; }

        private string MimeType { get; set; }

        private string ExportPipeline { get; set; }

        private string ProgressDialogTitle { get; set; }
    }
}

I modeled the above command after Sitecore.Forms.Core.Commands.Export.ExportToExcel in Sitecore.Forms.Core.dll: this command inherits some useful logic of Sitecore.Forms.Core.Commands.Export.ExportToXml but differs along the pipeline being invoked, the name of the export file, and content type of the file being created.

I decided to make the above command be generic: the name of the file, pipeline, progress dialog title — this is a heading that is displayed in a modal dialog that is launched when the data is being exported from the Form Reports — and content type of the file are passed to it from Sitecore via Sheer UI buttons (see below).

I then registered all of the above in Sitecore via the following patch configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <commands>
      <command name="forms:export" type="Sitecore.Sandbox.Forms.Core.Commands.Export.Export, Sitecore.Sandbox" />
    </commands>
    <pipelines>
      <exportToCsv>
        <processor type="Sitecore.Sandbox.Form.Core.Pipelines.Export.Csv.ExportToCsv, Sitecore.Sandbox" />
        <processor type="Sitecore.Form.Core.Pipelines.Export.SaveContent, Sitecore.Forms.Core" />
      </exportToCsv>
    </pipelines>
  </sitecore>
</configuration>

Now we must wire the command to Sheer UI buttons. This is how I wired up the export ‘All’ button (this button is available in a dropdown of the main export button in the Form Reports):

to-csv-all-core-db

I then created another export button which is used when exporting selected rows in the Form Reports:

to-csv-core-db

Let’s see this in action!

I opened up the Form Reports for a test form I had built for a previous blog post, and selected some rows (notice the ‘To CSV’ button in the ribbon):

form-reports-selected-export-csv

I clicked the ‘To CSV’ button — doing this launched a progress dialog (I wasn’t fast enough to grab a screenshot of it) — and was prompted to download the following file:

export-csv-txt

As you can see, the file looks beautiful in Excel 😉 :

export-csv-excel

If you have any thoughts on this, or ideas for other export data formats that could be incorporated into the Form Reports of Web Forms for Marketers, please share in a comment.

Until next time, have a Sitecoretastic day!

Restrict IP Access of Directories and Files in Your Sitecore Web Application Using a httpRequestBegin Pipeline Processor

Last week my friend and colleague Greg Coffman had asked me if I knew of a way to restrict IP access to directories within the Sitecore web application, and I recalled reading a post by Alex Shyba quite some time ago.

Although Alex’s solution is probably good enough in most circumstances, I decided to explore other solutions, and came up with the following <httpRequestBegin> pipeline processor as another way to accomplish this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web.Hosting;

using Sitecore.Configuration;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.HttpRequest;
using Sitecore.Web;

namespace Sitecore.Sandbox.Pipelines.HttpRequest
{
    public class FilePathRestrictor : HttpRequestProcessor 
    {
        public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if (!ShouldRedirect(args))
            {
                return;
            }

            RedirectToNoAccessUrl();
        }

        private bool ShouldRedirect(HttpRequestArgs args)
        {
            return CanProcess(args, GetFilePath(args)) 
                    && !CanAccess(args.Context.Request.UserHostAddress);
        }

        protected virtual string GetFilePath(HttpRequestArgs args)
        {
            if (string.IsNullOrWhiteSpace(Context.Page.FilePath))
            {
                return args.Url.FilePath;
            }

            return Context.Page.FilePath;
        }

        protected virtual bool CanProcess(HttpRequestArgs args, string filePath)
        {
            return !string.IsNullOrWhiteSpace(filePath)
                    && !string.IsNullOrWhiteSpace(RootFilePath)
                    && AllowedIPs != null
                    && AllowedIPs.Any()
                    && (HostingEnvironment.VirtualPathProvider.DirectoryExists(filePath)
                        || HostingEnvironment.VirtualPathProvider.FileExists(filePath))
                    && args.Url.FilePath.StartsWith(RootFilePath, StringComparison.CurrentCultureIgnoreCase)
                    && !string.IsNullOrWhiteSpace(args.Context.Request.UserHostAddress)
                    && !string.Equals(filePath, Settings.NoAccessUrl, StringComparison.CurrentCultureIgnoreCase);
        }

        protected virtual bool CanAccess(string ip)
        {
            Assert.ArgumentNotNullOrEmpty(ip, "ip");
            return AllowedIPs.Contains(ip);
        }

        protected virtual void RedirectToNoAccessUrl()
        {
            WebUtil.Redirect(Settings.NoAccessUrl);
        }

        protected virtual void AddAllowedIP(string ip)
        {
            if (string.IsNullOrWhiteSpace(ip) || AllowedIPs.Contains(ip))
            {
                return;
            }

            AllowedIPs.Add(ip);
        }

        private string RootFilePath { get; set; }

        private IList<string> _AllowedIPs;
        private IList<string> AllowedIPs 
        {
            get
            {
                if (_AllowedIPs == null)
                {
                    _AllowedIPs = new List<string>();
                }

                return _AllowedIPs;
            }
        }
    }
}

The pipeline processor above determines whether the IP making the request has access to the directory or file on the file system — a list of IP addresses that should have access are passed to the pipeline processor via a configuration file, and the code does check to see if the requested URL is a directory or a file on the file system — by matching the beginning of the URL with a configuration defined root path.

If the user does not have access to the requested path, s/he is redirected to the “No Access Url” which is specified in the Sitecore instance’s configuration.

The list of IP addresses that should have access to the directory — including everything within it — and the root path are handed to the pipeline processor via the following patch configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestBegin>
        <processor patch:before=" processor[@type='Sitecore.Pipelines.HttpRequest.FileResolver, Sitecore.Kernel']"
                   type="Sitecore.Sandbox.Pipelines.HttpRequest.FilePathRestrictor, Sitecore.Sandbox">
          <RootFilePath>/sitecore</RootFilePath>
          <AllowedIPs hint="list:AddAllowedIP">
            <IP>127.0.0.2</IP>
          </AllowedIPs>
        </processor>
      </httpRequestBegin>
    </pipelines>
  </sitecore>
</configuration>

Since my IP is 127.0.0.1, I decided to only allow 127.0.0.2 access to my Sitecore directory — this also includes everything within it — in the above configuration file for testing.

After navigating to /sitecore of my local sandbox instance, I was redirected to the “No Access Url” page defined in my Web.config:

no-access

If you have any thoughts on this, or know of other solutions, please share in a comment.

Embedded Tweets in Sitecore: A Proof of Concept

In a previous post, I showcased a “proof of concept” for shortcodes in Sitecore — this is a shorthand notation for embedding things like YouTube videos in your webpages without having to type up a bunch of HTML — and felt I should follow up with another “proof of concept” around incorporating Embedded Tweets in Sitecore.

You might be asking “what’s an Embedded Tweet?” An Embedded Tweet is basically the process of pasting a Tweet URL from Twitter into an editable content area of your website/blog/whatever (think Rich Text field in Sitecore), and let the code that builds the HTML for your site figure out how to display it.

For example, I had used an Embedded Tweet in a recent post:

tweet-url-wordpress

This is what is seen on the rendered page:

tweet-embedded

While doing some research via Google on how to do this in Sitecore, I found this page from Twitter that discusses how you could go about accomplishing this, and discovered how to get JSON containing information about a Tweet — including its HTML — using one of Twitter’s API URLs:

tweet-api-json

The JSON above drove me to build the following POCO class to represent data returned by that URL:

using System.Runtime.Serialization;

namespace Sitecore.Sandbox.Pipelines.RenderField.Tweets
{
    public class Tweet
    {
        [DataMember(Name = "cache_age")]
        public int CacheAgeMilliseconds { get; set; }

        [DataMember(Name = "url")]
        public string Url { get; set; }

        [DataMember(Name = "html")]
        public string Html { get; set; }
    }
}

I decided to omit some of the JSON properties returned by the Twitter URL from my class above — width and height are examples — since I felt I did not need to use them for this “proof of concept”.

I then leveraged the class above in the following class that will serve as a <renderField> pipeline processor to embed Tweets:

using System;
using System.IO;
using System.Net;
using System.Text.RegularExpressions;
using System.Web;

using Sitecore.Caching;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.RenderField;

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace Sitecore.Sandbox.Pipelines.RenderField.Tweets
{
    public class ExpandTweets
    {
        private string TwitterWidgetScriptTag {get ; set; }

        private string TwitterApiUrlFormat { get; set; }

        private string _TweetPattern;
        private string TweetPattern 
        {
            get
            {
                return _TweetPattern;
            }
            set
            {
                _TweetPattern = value;
                if (!string.IsNullOrWhiteSpace(_TweetPattern))
                {
                    _TweetPattern = HttpUtility.HtmlDecode(_TweetPattern);
                }
            }
        }

        private HtmlCache _HtmlCache;
        private HtmlCache HtmlCache
        {
            get
            {
                if (_HtmlCache == null)
                {
                    _HtmlCache = CacheManager.GetHtmlCache(Context.Site);
                }

                return _HtmlCache;
            }
        }

        public void Process(RenderFieldArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            AssertRequired();
            if(!ShouldFieldBeProcessed(args))
            {
                return;
            }

            args.Result.FirstPart = ExpandTweetUrls(args.Result.FirstPart);
        }

        private static bool ShouldFieldBeProcessed(RenderFieldArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            Assert.ArgumentNotNull(args.FieldTypeKey, "args.FieldTypeKey");
            string fieldTypeKey = args.FieldTypeKey.ToLower();
            return fieldTypeKey == "text"
                    || fieldTypeKey == "rich text"
                    || fieldTypeKey == "single-line text"
                    || fieldTypeKey == "multi-line text";
        }

        private void AssertRequired()
        {
            Assert.IsNotNullOrEmpty(TwitterWidgetScriptTag, "TwitterWidgetScriptTag must be set! Check your configuration!");
            Assert.IsNotNullOrEmpty(TwitterApiUrlFormat, "TwitterApiUrlFormat must be set! Check your configuration!");
            Assert.IsNotNullOrEmpty(TweetPattern, "TweetPattern must be set! Check your configuration!");
        }

        protected virtual string ExpandTweetUrls(string html)
        {
            string htmlExpanded = html;
            MatchCollection matches = Regex.Matches(htmlExpanded, TweetPattern, RegexOptions.Compiled | RegexOptions.IgnoreCase);
            foreach (Match match in matches)
            {
                string tweetHtml = GetTweetHtml(match.Groups["id"].Value);
                if (!string.IsNullOrWhiteSpace(tweetHtml))
                {
                    htmlExpanded = htmlExpanded.Replace(match.Value, tweetHtml);
                }
            }

            if (matches.Count > 0)
            {
                htmlExpanded = string.Concat(htmlExpanded, TwitterWidgetScriptTag);
            }

            return htmlExpanded;
        }

        protected virtual string GetTweetHtml(string id)
        {
            string html = GetTweetHtmlFromCache(id);
            if (!string.IsNullOrWhiteSpace(html))
            {
                return html;
            }

            Tweet tweet = GetTweetFromApi(id);
            AddTweetHtmlToCache(id, tweet);
            return tweet.Html;
        }

        private string GetTweetHtmlFromCache(string id)
        {
            return HtmlCache.GetHtml(id);
        }

        private void AddTweetHtmlToCache(string id, Tweet tweet)
        {
            if (string.IsNullOrWhiteSpace(tweet.Html))
            {
                return;
            }

            if (tweet.CacheAgeMilliseconds > 0)
            {
                HtmlCache.SetHtml(id, tweet.Html, DateTime.Now.AddMilliseconds(tweet.CacheAgeMilliseconds));
                return;
            }

            HtmlCache.SetHtml(id, tweet.Html);
        }

        protected virtual Tweet GetTweetFromApi(string id)
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(string.Format(TwitterApiUrlFormat, id));
            try
            {
                HttpWebResponse response = (HttpWebResponse)request.GetResponse();
                using (StreamReader reader = new StreamReader(response.GetResponseStream()))
                {
                    var result = reader.ReadToEnd();
                    JObject jObject = JObject.Parse(result);
                    return JsonConvert.DeserializeObject<Tweet>(jObject.ToString());
                }
            }
            catch (Exception ex)
            {
                Log.Error(this.ToString(), ex, this);
            }

            return new Tweet { Html = string.Empty };
        }
    }
}

Methods in the class above find all Tweet URLs in the Rich Text, Single-Line Text, or Multi-Line Text field being processed — the code determines if it’s a Tweet URL based on a pattern that is supplied by a configuration setting (you will see this below in this post); extract Tweets’ Twitter identifiers (these are located at the end of the Tweet URLs); and attempt to find the Tweets’ HTML in Sitecore’s HTML cache.

If the HTML is found in cache for a Tweet, we return it. Otherwise, we make a request to Twitter’s API to get it, put it in cache one we have it (it is set to expire after a specified number of milliseconds from the time it was retrieved: Twitter returns the number of milliseconds in one full year by default), and then we return it.

If the returned HTML is not empty, we replace it in the field’s value for display.

If the HTML returned is empty — this could happen when an exception is encountered during the Twitter API call (of course we log the exception in the Sitecore log when this happens 😉 ) — we don’t touch the Tweet URL in the field’s value.

Once all Tweet URLs have been processed, we append a script tag referencing Twitter’s widget.js file — this is supplied through a configuration setting, and it does the heavy lifting on making the Tweet HTML look Twitterific 😉 — to the field’s rendered HTML.

I then tied everything together using the following patch configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <renderField>
        <processor type="Sitecore.Sandbox.Pipelines.RenderField.Tweets.ExpandTweets, Sitecore.Sandbox"
					patch:after="processor[@type='Sitecore.Pipelines.RenderField.GetTextFieldValue, Sitecore.Kernel']">
          <TwitterWidgetScriptTag>&lt;script async src="//platform.twitter.com/widgets.js" charset="utf-8"&gt;&lt;/script&gt;</TwitterWidgetScriptTag>
          <TwitterApiUrlFormat>https://api.twitter.com/1/statuses/oembed.json?id={0}&amp;omit_script=true</TwitterApiUrlFormat>
          <TweetPattern>https://twitter.com/.+/status/(?&lt;id&gt;\d*)</TweetPattern>
        </processor>
      </renderField>
    </pipelines>
  </sitecore>
</configuration>

Let’s see this in action!

I created a test Item, and added some legitimate and bogus Tweet URLs into one of its Rich Text fields (please pardon the grammatical issues in the following screenshots :-/):

tweets-rich-text

This isn’t the most aesthetically pleasing HTML, but it will serve its purpose for testing:

tweets-rich-text-html

After saving and publishing, I navigated to my test Item’s page, and saw this:

tweets-front-end

If you have any suggestions on making this better, or have other ideas for embedding Tweets in Sitecore, please share in a comment.

Prevent Sitecore Dictionary Entry Keys From Appearing When Their Phrase Field is Empty

Earlier today when doing research for another blog post around on-demand language translation in Sitecore, I remembered I wanted to blog about an issue I saw a while back when using the Sitecore Dictionary, but before I dive into that issue — and a possible approach for resolving it — let me give you a little information on what the Sitecore Dictionary is, and why you might want to use it — actually you probably should use it!

The Sitecore Dictionary is a place in Sitecore where you can store multilingual content for labels or string literals in your code (this could be front-end code, or even content displayed in the Sitecore shell). I’ve created the following Dictionary entry as an example:

coffee-dictionary-entry-set

The “coffee” item above is the Dictionary entry, and its parent item “beverage types” is a Dictionary folder.

You could use a sublayout like the following to display the text stored in the Phrase field on the front-end of your website:

<%@ Control Language="C#" AutoEventWireup="true" CodeBehind="Translate Test.ascx.cs" Inherits="Sandbox.layouts.sublayouts.Translate_Test" %>
<h2>Dictionary Test</h2>
Key => <asp:Literal ID="litKey" runat="server" /><br />
Phrase => <asp:Literal ID="litTranslateTest" runat="server" />

The code-behind of the sublayout:

using System;

using Sitecore.Globalization;

namespace Sandbox.layouts.sublayouts
{
    public partial class Translate_Test : System.Web.UI.UserControl
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            string key = "beveragetypes.coffee";
            litKey.Text = key;
            litTranslateTest.Text = Translate.Text(key);
        }
    }
}

In the Page_Load method above, I’ve invoked Sitecore.Globalization.Translate.Text() to grab the value out of the Phrase field of the “coffee” Dictionary entry using its key. The Sitecore.Globalization.Translate.Text() method uses Sitecore.Context.Language to ascertain which language version of the Dictionary entry to use.

When I navigated to the page that has the sublayout above mapped to its presentation, I see the “coffee” entry’s Phrase appear:

coffee-dictionary-entry-set-front-end

Let’s see how this works using another language version of this Dictionary entry. I added a Danish version for our “coffee” entry:

coffee-dictionary-entry-set-danish

I navigated to my page again after embedding the Danish language code in its URL to get the Danish version of this Dictionary entry:

coffee-dictionary-entry-set-front-end-danish

As you can see the Danish version appeared, and I did not have to write any additional code to make this happen.

Well, this is great and all until someone forgets to include a phrase for a Dictionary entry:

coffee-dictionary-entry-not-set

When we go to the front-end, we see that the Dictionary entry’s key appears instead of its phrase:

coffee-dictionary-entry-empty-front-end-problem

As a fix for this, I created the following class to serve as a processor for the <getTranslation> pipeline (this pipeline was introduced in Sitecore 6.6):

using System.Collections.Generic;

using Sitecore.Diagnostics;
using Sitecore.Pipelines.GetTranslation;

namespace Sitecore.Sandbox.Pipelines.GetTranslation
{
    public class SetAsEmpty
    {
        private IList<string> _KeyPrefixes;
        private IList<string> KeyPrefixes 
        {
            get
            {
                if (_KeyPrefixes == null)
                {
                    _KeyPrefixes = new List<string>();
                }

                return _KeyPrefixes;
            }
        }

        public void Process(GetTranslationArgs args)
        {
            if (!ShouldSetAsEmpty(args))
            {
                return;
            }

            args.Result = string.Empty;
        }

        protected virtual bool ShouldSetAsEmpty(GetTranslationArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            return args.Result == null && HasKeyPrefix(args.Key);
        }

        protected virtual bool HasKeyPrefix(string key)
        {
            if (string.IsNullOrWhiteSpace(key))
            {
                return false;
            }

            foreach (string keyPrefix in KeyPrefixes)
            {
                if (key.StartsWith(keyPrefix))
                {
                    return true;
                }
            }

            return false;
        }

        protected virtual void AddKeyPrefix(string keyPrefix)
        {
            if(string.IsNullOrWhiteSpace(keyPrefix))
            {
                return;
            }

            KeyPrefixes.Add(keyPrefix);
        }
    }
}

The idea here is to check to see if the Dictionary entry’s key starts with a configuration defined prefix, and if it does, set the GetTranslationArgs instance’s Result property to the empty string when it’s null.

The reason why we check for a specific prefix is to ensure we don’t impact other parts of Sitecore that use methods that leverage the <getTranslation> pipeline (I learned this the hard way when virtually all labels in my instance’s Content Editor disappeared before adding the logic above to check whether a Dictionary entry’s key started with a config defined prefix).

I then wired this up in Sitecore using the following configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <getTranslation>
        <processor type="Sitecore.Sandbox.Pipelines.GetTranslation.SetAsEmpty, Sitecore.Sandbox">
          <keyPrefixes hint="list:AddKeyPrefix">
            <prefix>beveragetypes.</prefix>
          </keyPrefixes>
        </processor>
      </getTranslation>
    </pipelines>
  </sitecore>
</configuration>

When I navigated back to my page, I see that nothing appears for the Dictionary entry’s phrase since it was set to the empty string by our pipeline processor above.

coffee-dictionary-entry-empty-front-end

One thing I should note: I have only tested this in Sitecore 6.6, and I’m not aware if this Dictionary entry issue exists in Sitecore 7. If this issue was fixed in Sitecore 7, please share in a comment.

Plus, if you have any comments on this, or other ideas for solving this problem, please leave a comment.

Synchronize IDTable Entries Across Multiple Sitecore Databases Using a Custom publishItem Pipeline Processor

In a previous post I showed a solution that uses the Composite design pattern in an attempt to answer the following question by Sitecore MVP Kyle Heon:

Although I enjoyed building that solution, it isn’t ideal for synchronizing IDTable entries across multiple Sitecore databases — entries are added to all configured IDTables even when Items might not exist in all databases of those IDTables (e.g. the Sitecore Items have not been published to those databases).

I came up with another solution to avoid the aforementioned problem — one that synchronizes IDTable entries using a custom <publishItem> pipeline processor, and the following class contains code for that processor:

using System.Collections.Generic;
using System.Linq;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.IDTables;
using Sitecore.Diagnostics;
using Sitecore.Publishing.Pipelines.PublishItem;

namespace Sitecore.Sandbox.Pipelines.Publishing
{
    public class SynchronizeIDTables : PublishItemProcessor
    {
        private IEnumerable<string> _IDTablePrefixes;
        private IEnumerable<string> IDTablePrefixes
        {
            get
            {
                if (_IDTablePrefixes == null)
                {
                    _IDTablePrefixes = GetIDTablePrefixes();
                }

                return _IDTablePrefixes;
            }
        }

        private string IDTablePrefixesConfigPath { get; set; }

        public override void Process(PublishItemContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            Assert.ArgumentNotNull(context.PublishOptions, "context.PublishOptions");
            Assert.ArgumentNotNull(context.PublishOptions.SourceDatabase, "context.PublishOptions.SourceDatabase");
            Assert.ArgumentNotNull(context.PublishOptions.TargetDatabase, "context.PublishOptions.TargetDatabase");
            IDTableProvider sourceProvider = CreateNewIDTableProvider(context.PublishOptions.SourceDatabase);
            IDTableProvider targetProvider = CreateNewIDTableProvider(context.PublishOptions.TargetDatabase);
            RemoveEntries(targetProvider, GetAllEntries(targetProvider, context.ItemId));
            AddEntries(targetProvider, GetAllEntries(sourceProvider, context.ItemId));
        }

        protected virtual IDTableProvider CreateNewIDTableProvider(Database database)
        {
            Assert.ArgumentNotNull(database, "database");
            return Factory.CreateObject(string.Format("IDTable[@id='{0}']", database.Name), true) as IDTableProvider;
        }

        protected virtual IEnumerable<IDTableEntry> GetAllEntries(IDTableProvider provider, ID itemId)
        {
            Assert.ArgumentNotNull(provider, "provider");
            Assert.ArgumentCondition(!ID.IsNullOrEmpty(itemId), "itemId", "itemId cannot be null or empty!");
            List<IDTableEntry> entries = new List<IDTableEntry>();
            foreach(string prefix in IDTablePrefixes)
            {
                IEnumerable<IDTableEntry> entriesForPrefix = provider.GetKeys(prefix, itemId);
                if (entriesForPrefix.Any())
                {
                    entries.AddRange(entriesForPrefix);
                }
            }

            return entries;
        }

        private static void RemoveEntries(IDTableProvider provider, IEnumerable<IDTableEntry> entries)
        {
            Assert.ArgumentNotNull(provider, "provider");
            Assert.ArgumentNotNull(entries, "entries");
            foreach (IDTableEntry entry in entries)
            {
                provider.Remove(entry.Prefix, entry.Key);
            }
        }

        private static void AddEntries(IDTableProvider provider, IEnumerable<IDTableEntry> entries)
        {
            Assert.ArgumentNotNull(provider, "provider");
            Assert.ArgumentNotNull(entries, "entries");
            foreach (IDTableEntry entry in entries)
            {
                provider.Add(entry);
            }
        }

        protected virtual IEnumerable<string> GetIDTablePrefixes()
        {
            Assert.ArgumentNotNullOrEmpty(IDTablePrefixesConfigPath, "IDTablePrefixConfigPath");
            return Factory.GetStringSet(IDTablePrefixesConfigPath);
        }
    }
}

The Process method above grabs all IDTable entries for all defined IDTable prefixes — these are pulled from the configuration file that is shown later on in this post — from the source database for the Item being published, and pushes them all to the target database after deleting all preexisting entries from the target database for the Item (the code is doing a complete overwrite for the Item’s IDTable entries in the target database).

I also added the following code to serve as an item:deleted event handler (if you would like to learn more about events and their handlers, check out John West‘s post about them, and also take a look at this page on the
Sitecore Developer Network (SDN)) to remove entries for the Item when it’s being deleted:

using System;
using System.Collections.Generic;
using System.Linq;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Events;
using Sitecore.Data.IDTables;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Events;

namespace Sitecore.Sandbox.Data.IDTables
{
    public class ItemEventHandler
    {
        private IEnumerable<string> _IDTablePrefixes;
        private IEnumerable<string> IDTablePrefixes
        {
            get
            {
                if (_IDTablePrefixes == null)
                {
                    _IDTablePrefixes = GetIDTablePrefixes();
                }

                return _IDTablePrefixes;
            }
        }

        private string IDTablePrefixesConfigPath { get; set; }

        protected void OnItemDeleted(object sender, EventArgs args)
        {
            if (args == null)
            {
                return;
            }

            Item item = Event.ExtractParameter(args, 0) as Item;
            if (item == null)
            {
                return;
            }

            DeleteItemEntries(item);
        }

        private void DeleteItemEntries(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            IDTableProvider provider = CreateNewIDTableProvider(item.Database.Name);
            foreach (IDTableEntry entry in GetAllEntries(provider, item.ID))
            {
                provider.Remove(entry.Prefix, entry.Key);
            }
        }

        protected virtual IEnumerable<IDTableEntry> GetAllEntries(IDTableProvider provider, ID itemId)
        {
            Assert.ArgumentNotNull(provider, "provider");
            Assert.ArgumentCondition(!ID.IsNullOrEmpty(itemId), "itemId", "itemId cannot be null or empty!");
            List<IDTableEntry> entries = new List<IDTableEntry>();
            foreach (string prefix in IDTablePrefixes)
            {
                IEnumerable<IDTableEntry> entriesForPrefix = provider.GetKeys(prefix, itemId);
                if (entriesForPrefix.Any())
                {
                    entries.AddRange(entriesForPrefix);
                }
            }

            return entries;
        }

        private static void RemoveEntries(IDTableProvider provider, IEnumerable<IDTableEntry> entries)
        {
            Assert.ArgumentNotNull(provider, "provider");
            Assert.ArgumentNotNull(entries, "entries");
            foreach (IDTableEntry entry in entries)
            {
                provider.Remove(entry.Prefix, entry.Key);
            }
        }

        protected virtual IDTableProvider CreateNewIDTableProvider(string databaseName)
        {
            return Factory.CreateObject(string.Format("IDTable[@id='{0}']", databaseName), true) as IDTableProvider;
        }

        protected virtual IEnumerable<string> GetIDTablePrefixes()
        {
            Assert.ArgumentNotNullOrEmpty(IDTablePrefixesConfigPath, "IDTablePrefixConfigPath");
            return Factory.GetStringSet(IDTablePrefixesConfigPath);
        }
    }
}

The above code retrieves all IDTable entries for the Item being deleted — filtered by the configuration defined IDTable prefixes — from its database’s IDTable, and calls the Remove method on the IDTableProvider instance that is created for the Item’s database for each entry.

I then registered all of the above in Sitecore using the following configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <events>
      <event name="item:deleted">
        <handler type="Sitecore.Sandbox.Data.IDTables.ItemEventHandler, Sitecore.Sandbox" method="OnItemDeleted">
          <IDTablePrefixesConfigPath>IDTablePrefixes/IDTablePrefix</IDTablePrefixesConfigPath>
        </handler>
      </event>
    </events>
    <IDTable type="Sitecore.Data.$(database).$(database)IDTable, Sitecore.Kernel" singleInstance="true">
      <patch:attribute name="id">master</patch:attribute>
      <param connectionStringName="master"/>
      <param desc="cacheSize">500KB</param>
    </IDTable>
    <IDTable id="web" type="Sitecore.Data.$(database).$(database)IDTable, Sitecore.Kernel" singleInstance="true">
      <param connectionStringName="web"/>
      <param desc="cacheSize">500KB</param>
    </IDTable>
    <IDTablePrefixes>
      <IDTablePrefix>IDTableTest</IDTablePrefix>
    </IDTablePrefixes>
    <pipelines>
      <publishItem>
        <processor type="Sitecore.Sandbox.Pipelines.Publishing.SynchronizeIDTables, Sitecore.Sandbox">
          <IDTablePrefixesConfigPath>IDTablePrefixes/IDTablePrefix</IDTablePrefixesConfigPath>
        </processor>
      </publishItem>
    </pipelines>
  </sitecore>
</configuration>

For testing, I quickly whipped up a web form to add a couple of IDTable entries using an IDTableProvider for the master database — I am omitting that code for brevity — and ran a query to verify the entries were added into the IDTable in my master database (I also ran another query for the IDTable in my web database to show that it contains no entries):

idtables-before-publish

I published both items, and queried the IDTable in the master and web databases:

idtables-after-publish-both-items

As you can see, both entries were inserted into the web database’s IDTable.

I then deleted one of the items from the master database via the Sitecore Content Editor:

idtables-deleted-from-master

It was removed from the IDTable in the master database.

I then published the deleted item’s parent with subitems:

idtables-published-deletion

As you can see, it was removed from the IDTable in the web database.

If you have any suggestions for making this code better, or have another solution for synchronizing IDTable entries across multiple Sitecore databases, please share in a comment.

Restart the Sitecore Client and Server Using Custom Pipelines

Last week Michael West asked me about creating shortcuts to restart the Sitecore client and server via this tweet, and I was immediately curious myself on what was needed to accomplish this.

If you are unfamiliar with restarting the Sitecore client and server, these are options that are presented to Sitecore users — typically developers — after installing a package into Sitecore:

install-package-end-wizard

Until last week, I never understood how either of these checkboxes worked, and uncovered the following code during my research:

restart-code-in-InstallPackageForm

Michael West had conveyed how he wished these two lines of code lived in pipelines, and that prompted me to write this article — basically encapsulate the logic above into two custom pipelines: one to restart the Sitecore client, and the other to restart the Sitecore server.

I decided to define the concept of a ‘Restarter’, an object that restarts something — this could be anything — and defined the following interface for such objects:

namespace Sitecore.Sandbox.Utilities.Restarters
{
    public interface IRestarter
    {
        void Restart();
    }
}

I then created the following IRestarter for the Sitecore client:

using Sitecore;
using Sitecore.Diagnostics;
using Sitecore.Web.UI.Sheer;

namespace Sitecore.Sandbox.Utilities.Restarters
{
    public class SitecoreClientRestarter : IRestarter
    {
        private ClientResponse ClientResponse { get; set; }

        public SitecoreClientRestarter()
            : this(Context.ClientPage)
        {
        }

        public SitecoreClientRestarter(ClientPage clientPage)
        {
            SetClientResponse(clientPage);
        }

        public SitecoreClientRestarter(ClientResponse clientResponse)
        {
            SetClientResponse(clientResponse);
        }

        private void SetClientResponse(ClientPage clientPage)
        {
            Assert.ArgumentNotNull(clientPage, "clientPage");
            SetClientResponse(clientPage.ClientResponse);
        }

        private void SetClientResponse(ClientResponse clientResponse)
        {
            Assert.ArgumentNotNull(clientResponse, "clientResponse");
            ClientResponse = clientResponse;
        }

        public void Restart()
        {
            ClientResponse.Broadcast(ClientResponse.SetLocation(string.Empty), "Shell");
        }
    }
}

The class above has three constructors. One constructor takes an instance of Sitecore.Web.UI.Sheer.ClientResponse — this lives in Sitecore.Kernel.dll — and another constructor takes in an instance of
Sitecore.Web.UI.Sheer.ClientPage — this also lives in Sitecore.Kernel.dll — which contains a property instance of Sitecore.Web.UI.Sheer.ClientResponse, and this instance is set on the ClientResponse property of the SitecoreClientRestarter class.

The third constructor — which is parameterless — calls the constructor that takes in a Sitecore.Web.UI.Sheer.ClientPage instance, and passes the ClientResponse instance set in Sitecore.Context.

I followed building the above class with another IRestarter — one that restarts the Sitecore server:

using Sitecore.Install;

namespace Sitecore.Sandbox.Utilities.Restarters
{
    public class SitecoreServerRestarter : IRestarter
    {
        public SitecoreServerRestarter()
        {
        }

        public void Restart()
        {
            Installer.RestartServer();
        }
    }
}

There really isn’t much happening in the class above. It just calls the static method RestartServer() — this method changes the timestamp on the Sitecore instance’s Web.config to trigger a web application restart — on Sitecore.Install.Installer in Sitecore.Kernel.dll.

Now we need to a way to use the IRestarter classes above. I built the following class to serve as a processor of custom pipelines I define later on in this post:

using Sitecore.Diagnostics;
using Sitecore.Pipelines;

using Sitecore.Sandbox.Utilities.Restarters;

namespace Sitecore.Sandbox.Pipelines.RestartRestarter
{
    public class RestartRestarterOperations
    {
        private IRestarter Restarter { get; set; }

        public void Process(PipelineArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            Assert.ArgumentNotNull(Restarter, "Restarter");
            Restarter.Restart();
        }
    }
}

Through a pipeline processor configuration setting, we define the type of IRestarter — it’s magically created by Sitecore when the pipeline processor instance is created.

After some null checks, the Process() method invokes Restart() on the IRestarter instance, ultimately restarting whatever the IRestarter is set to restart.

I then needed a way to test the pipelines I define later on in this post. I built the following class to serve as commands that I added into the Sitecore ribbon:

using Sitecore.Diagnostics;
using Sitecore.Pipelines;
using Sitecore.Shell.Framework.Commands;

namespace Sitecore.Sandbox.Commands.Admin
{
    public class Restart : Command
    {
        public override void Execute(CommandContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            Assert.ArgumentNotNull(context.Parameters, "context.Parameters");
            Assert.ArgumentNotNullOrEmpty(context.Parameters["pipeline"], "context.Parameters[\"pipeline\"]");
            CorePipeline.Run(context.Parameters["pipeline"], new PipelineArgs());
        }
    }
}

The command above expects a pipeline name to be supplied via the Parameters NameValueCollection instance set on the CommandContext instance passed to the Execute method() — I show later on in this post how I pass the name of the pipeline to the command.

If the pipeline name is given, we invoke the pipeline.

I then glued everything together using the following configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <commands>
      <command name="admin:Restart" type="Sitecore.Sandbox.Commands.Admin.Restart, Sitecore.Sandbox"/>
    </commands>
    <pipelines>
      <restartClient>
        <processor type="Sitecore.Sandbox.Pipelines.RestartRestarter.RestartRestarterOperations, Sitecore.Sandbox">
          <Restarter type="Sitecore.Sandbox.Utilities.Restarters.SitecoreClientRestarter, Sitecore.Sandbox"/>
        </processor>  
      </restartClient>
      <restartServer>
        <processor type="Sitecore.Sandbox.Pipelines.RestartRestarter.RestartRestarterOperations, Sitecore.Sandbox">
          <Restarter type="Sitecore.Sandbox.Utilities.Restarters.SitecoreServerRestarter, Sitecore.Sandbox"/>
        </processor>
      </restartServer>
    </pipelines>
  </sitecore>
</configuration>

I’m omitting how I created custom buttons in a custom ribbon in Sitecore to test this. If you want to learn about adding buttons to the Sitecore Ribbon, please read John West’s blog post on doing so.

However, I did do the following to pass the name of the pipeline we want to invoke in the custom command class defined above:

restart-client-button

I wish I could show you some screenshots on how this works. However, there really isn’t much visual to see here.

If you have any suggestions on how I could show this in action, or improve the code above, please share in a comment.

Perform a Virus Scan on Files Uploaded into Sitecore

Last week I took notice of a SDN forum post asking whether there are any best practices for checking files uploaded into Sitecore for viruses.

Although I am unaware of there being any best practices for this, adding a processor into the <uiUpload> pipeline to perform virus scans has been on my plate for the past month. Not being able to find an open source .NET antivirus library has been my major blocker for building one.

However, after many hours of searching the web — yes, I do admit some of this time is sprinkled with idle surfing — over multiple weeks, I finally discovered nClam — a .NET client library for scanning files using a Clam Antivirus server (I setup one using instructions enumerated here).

Before I continue, I would like to caution you on using this or any antivirus solution before doing a substantial amount of research — I do not know how robust the solution I am sharing actually is, and I am by no means an antivirus expert. The purpose of this post is to show how one might go about adding antivirus capabilities into Sitecore, and I am not advocating or recommending any particular antivirus software package. Please consult with an antivirus expert before using any antivirus software/.NET client library .

The solution I came up with uses the adapter design pattern — it basically wraps and makes calls to the nClam library, and is accessible through the following antivirus scanner interface:

using System.IO;

namespace Sitecore.Sandbox.Security.Antivirus
{
    public interface IScanner
    {
        ScanResult Scan(Stream sourceStream);

        ScanResult Scan(string filePath);
    }
}

This interface defines the bare minimum methods our antivirus classes should have. Instances of these classes should offer the ability to scan a file through the file’s Stream, or scan a file located on the server via the supplied file path.

The two scan methods defined by the interface must return an instance of the following data transfer object:

namespace Sitecore.Sandbox.Security.Antivirus
{
    public class ScanResult
    {
        public string Message { get; set; }

        public ScanResultType ResultType { get; set; }
    }
}

Instances of the above class contain a detailed message coupled with a ScanResultType enumeration value which conveys whether the scanned file is clean, contains a virus, or something else went wrong during the scanning process:

namespace Sitecore.Sandbox.Security.Antivirus
{
    public enum ScanResultType
    {
        Clean,
        VirusDetected,
        Error,
        Unknown
    }
}

I used the ClamScanResults enumeration as a model for the above.

I created and used the ScanResultType enumeration instead of the ClamScanResults enumeration so that this solution can be extended for other antivirus libraries — or calls could be made to other antivirus software through the command-line — and these shouldn’t be tightly coupled to the nClam library.

I then wrapped the nClam library calls in the following ClamScanner class:

using System;
using System.IO;
using System.Linq;

using Sitecore.Diagnostics;

using nClam;

namespace Sitecore.Sandbox.Security.Antivirus
{
    public class ClamScanner : IScanner
    {
        private ClamClient Client { get; set; }

        public ClamScanner(string server, string port)
            : this(server, int.Parse(port))
        {
        }

        public ClamScanner(string server, int port)
            : this(new ClamClient(server, port))
        {
        }

        public ClamScanner(ClamClient client)
        {
            SetClient(client);
        }

        private void SetClient(ClamClient client)
        {
            Assert.ArgumentNotNull(client, "client");
            Client = client;
        }

        public ScanResult Scan(Stream sourceStream)
        {
            ScanResult result;
            try
            {
                result = CreateNewScanResult(Client.SendAndScanFile(sourceStream));
            }
            catch (Exception ex)
            {
                result = CreateNewExceptionScanResult(ex);
            }

            return result;
        }

        public ScanResult Scan(string filePath)
        {
            ScanResult result;
            try
            {
                result = CreateNewScanResult(Client.SendAndScanFile(filePath));
            }
            catch (Exception ex)
            {
                result = CreateNewExceptionScanResult(ex);
            }

            return result;
        }

        private static ScanResult CreateNewScanResult(ClamScanResult result)
        {
            Assert.ArgumentNotNull(result, "result");
            if (result.Result == ClamScanResults.Clean)
            {
                return CreateNewScanResult("Yay! No Virus found!", ScanResultType.Clean);
            }

            if (result.Result == ClamScanResults.VirusDetected)
            {
                string message = string.Format("Oh no! The {0} virus was found!", result.InfectedFiles.First().VirusName);
                return CreateNewScanResult(message, ScanResultType.VirusDetected);
            }

            if (result.Result == ClamScanResults.Error)
            {
                string message = string.Format("Something went terribly wrong somewhere. Details: {0}", result.RawResult);
                return CreateNewScanResult(message, ScanResultType.Error);
            }

            return CreateNewScanResult("I have no clue about what just happened.", ScanResultType.Unknown);
        }

        private static ScanResult CreateNewExceptionScanResult(Exception ex)
        {
            string message = string.Format("Something went terribly wrong somewhere. Details:\n{0}", ex.ToString());
            return CreateNewScanResult(ex.ToString(), ScanResultType.Error);
        }

        private static ScanResult CreateNewScanResult(string message, ScanResultType type)
        {
            return new ScanResult
            {
                Message = message,
                ResultType = type
            };
        }
    }
}

Methods in the above class make calls to the nClam library, and return a ScanResult intance containing detailed information about the file scan.

Next I developed the following <uiUpload> pipeline processor to use instances of classes that define the IScanner interface above, and these instances are set via Sitecore when instantiating this pipeline processor — the ClamScanner type is defined in the patch configuration file shown later in this post:

using System.IO;
using System.Web;

using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Pipelines.Upload;

using Sitecore.Sandbox.Security.Antivirus;

namespace Sitecore.Sandbox.Pipelines.Upload
{
    public class ScanForViruses
    {
        public void Process(UploadArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            foreach (string fileKey in args.Files)
            {
                HttpPostedFile file = args.Files[fileKey];
                ScanResult result = ScanFile(file.InputStream);
                if(result.ResultType != ScanResultType.Clean)
                {
                    args.ErrorText = Translate.Text(string.Format("The file \"{0}\" cannot be uploaded. Reason: {1}", file.FileName, result.Message));
                    Log.Warn(args.ErrorText, this);
                    args.AbortPipeline();
                }
            }
        }

        private ScanResult ScanFile(Stream fileStream)
        {
            Assert.ArgumentNotNull(fileStream, "fileStream");
            return Scanner.Scan(fileStream);
        }

        private IScanner Scanner { get; set; }
    }
}

Uploaded files are passed to the IScanner instance, and the pipeline is aborted if something isn’t quite right — this occurs when a virus is detected, or an error is reported by the IScanner instance. If a virus is discovered, or an error occurs, the message contained within the ScanResult instance is captured in the Sitecore log.

I then glued everything together using the following patch configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <processors>
      <uiUpload>
        <processor mode="on" patch:before="processor[@type='Sitecore.Pipelines.Upload.CheckSize, Sitecore.Kernel']" 
                   type="Sitecore.Sandbox.Pipelines.Upload.ScanForViruses, Sitecore.Sandbox">
          <Scanner type="Sitecore.Sandbox.Security.Antivirus.ClamScanner">
            <param desc="server">localhost</param>
            <param desc="port">3310</param>
          </Scanner>
        </processor>
      </uiUpload>
    </processors>
  </sitecore>
</configuration>

I wish I could show you this in action when a virus is discovered in an uploaded file. However, I cannot put my system at risk by testing with an infected file.

But, I can show you what happens when an error is detected. I will do this by shutting down the Clam Antivirus server on my machine:

scanner-turned-off

On upload, I see the following:

tried-to-upload-file

When I opened up the Sitecore log, I see what the problem is:

log-error

Let’s turn it back on:

scanner-turned-on

I can now upload files again:

upload-no-error

If you have any suggestions on making this better, or know of a way to test it with a virus, please share in a comment below.

Choose Template Fields to Display in the Sitecore Content Editor

The other day I was going through search terms people had used to get to my blog, and discovered a few people made their way to my blog by searching for ‘sitecore hide sections in data template’.

I had built something like this in the past, but no longer remember how I had implemented that particular solution — not that I could show you that solution since it’s owned by a previous employer — and decided I would build another solution to accomplish this.

Before I move forward, I would like to point out that Sitecore MVP Andy Uzick wrote a blog post recently showing how to hide fields and sections in the content editor, though I did not have much luck with hiding sections in the way that he had done it — sections with no fields were still displaying for me in the content editor — and I decided to build a different solution altogether to make this work.

I thought it would be a good idea to let users turn this functionality on and off via a checkbox in the ribbon, and used some code from a previous post — in this post I had build an object to keep track of the state of a checkbox in the ribbon — as a model. In the spirit of that object, I defined the following interface:

namespace Sitecore.Sandbox.Utilities.ClientSettings
{
    public interface IRegistrySettingToggle
    {
        bool IsOn();

        void TurnOn();

        void TurnOff();
    }
}

I then created the following abstract class which implements the interface above, and stores the state of the setting defined by the given key — the key of the setting and the “on” value are passed to it via a subclass — in the Sitecore registry.

using Sitecore.Diagnostics;

using Sitecore.Web.UI.HtmlControls;

namespace Sitecore.Sandbox.Utilities.ClientSettings
{
    public abstract class RegistrySettingToggle : IRegistrySettingToggle
    {
        private string RegistrySettingKey { get; set; }

        private string RegistrySettingOnValue { get; set; }

        protected RegistrySettingToggle(string registrySettingKey, string registrySettingOnValue)
        {
            SetRegistrySettingKey(registrySettingKey);
            SetRegistrySettingOnValue(registrySettingOnValue);
        }

        private void SetRegistrySettingKey(string registrySettingKey)
        {
            Assert.ArgumentNotNullOrEmpty(registrySettingKey, "registrySettingKey");
            RegistrySettingKey = registrySettingKey;
        }

        private void SetRegistrySettingOnValue(string registrySettingOnValue)
        {
            Assert.ArgumentNotNullOrEmpty(registrySettingOnValue, "registrySettingOnValue");
            RegistrySettingOnValue = registrySettingOnValue;
        }

        public bool IsOn()
        {
            return Registry.GetString(RegistrySettingKey) == RegistrySettingOnValue;
        }

        public void TurnOn()
        {
            Registry.SetString(RegistrySettingKey, RegistrySettingOnValue);
        }

        public void TurnOff()
        {
            Registry.SetString(RegistrySettingKey, string.Empty);
        }
    }
}

I then built the following class to toggle the display settings for our displayable fields:

using System;

namespace Sitecore.Sandbox.Utilities.ClientSettings
{
    public class ShowDisplayableFieldsOnly : RegistrySettingToggle
    {
        private const string RegistrySettingKey = "/Current_User/Content Editor/Show Displayable Fields Only";
        private const string RegistrySettingOnValue = "on";

        private static volatile IRegistrySettingToggle current;
        private static object lockObject = new Object();

        public static IRegistrySettingToggle Current
        {
            get
            {
                if (current == null)
                {
                    lock (lockObject)
                    {
                        if (current == null)
                        {
                            current = new ShowDisplayableFieldsOnly();
                        }
                    }
                }

                return current;
            }
        }

        private ShowDisplayableFieldsOnly()
            : base(RegistrySettingKey, RegistrySettingOnValue)
        {
        }
    }
}

It passes its Sitecore registry key and “on” state value to the RegistrySettingToggle base class, and employs the Singleton pattern — I saw no reason for there to be multiple instances of this object floating around.

In order to use a checkbox in the ribbon, we have to create a new command for it:

using System.Linq;

using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Shell.Framework.Commands;

using Sitecore.Sandbox.Utilities.ClientSettings;

namespace Sitecore.Sandbox.Commands
{
    public class ToggleDisplayableFieldsVisibility : Command
    {
        public override void Execute(CommandContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            ToggleDisplayableFields();
            Refresh(context);
        }

        private static void ToggleDisplayableFields()
        {
            IRegistrySettingToggle showDisplayableFieldsOnly = ShowDisplayableFieldsOnly.Current;
            if (!showDisplayableFieldsOnly.IsOn())
            {
                showDisplayableFieldsOnly.TurnOn();
            }
            else
            {
                showDisplayableFieldsOnly.TurnOff();
            }
        }

        public override CommandState QueryState(CommandContext context)
        {
            if (!ShowDisplayableFieldsOnly.Current.IsOn())
            {
                return CommandState.Enabled;
            }

            return CommandState.Down;
        }

        private static void Refresh(CommandContext context)
        {
            Refresh(GetItem(context));
        }

        private static void Refresh(Item item)
        {
            Assert.ArgumentNotNull(item, "item");
            Context.ClientPage.ClientResponse.Timer(string.Format("item:load(id={0})", item.ID), 1);
        }

        private static Item GetItem(CommandContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            return context.Items.FirstOrDefault();
        }
    }
}

The command above leverages the instance of the ShowDisplayableFieldsOnly class defined above to turn the displayable fields feature on and off.

I followed the creation of the command above with the definition of the ribbon checkbox in the core database:

show-displayable-fields-checkbox-defined

The command name above — which is set in the Click field — is defined in the patch configuration file towards the end of this post.

I then created the following data template with a TreelistEx field to store the displayable fields:

displayable-fields-template

The TreelistEx field above will pull in all sections and their fields into the TreelistEx dialog, but only allow the selection of template fields, as is dictated by the following parameters that I have mapped in its Source field:

DataSource=/sitecore/templates/Sample/Sample Item&IncludeTemplatesForSelection=Template field&IncludeTemplatesForDisplay=Template section,Template field&AllowMultipleSelection=no

I then set this as a base template in my sandbox solution’s Sample Item template:

set-displayable-fields-template-as-base

In order to remove fields, we need a <getContentEditorFields> pipeline processor. I built the following class for to serve as one:

using System;

using Sitecore.Configuration;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Data.Managers;
using Sitecore.Data.Templates;
using Sitecore.Diagnostics;
using Sitecore.Shell.Applications.ContentManager;
using Sitecore.Shell.Applications.ContentEditor.Pipelines.GetContentEditorFields;

using Sitecore.Sandbox.Utilities.ClientSettings;

namespace Sitecore.Sandbox.Shell.Applications.ContentEditor.Pipelines.GetContentEditorFields
{
    public class RemoveUndisplayableFields
    {
        public void Process(GetContentEditorFieldsArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            Assert.ArgumentNotNull(args.Item, "args.Item");
            Assert.ArgumentCondition(!string.IsNullOrWhiteSpace(DisplayableFieldsFieldName), "DisplayableFieldsFieldName", "DisplayableFieldsFieldName must be set in the configuration file!");
            if (!ShowDisplayableFieldsOnly.Current.IsOn())
            {
                return;
            }

            foreach (Editor.Section section in args.Sections)
            {
                AddDisplayableFields(args.Item[DisplayableFieldsFieldName], section);
            }
        }

        private void AddDisplayableFields(string displayableFieldIds, Editor.Section section)
        {
            Editor.Fields displayableFields = new Editor.Fields();
            foreach (Editor.Field field in section.Fields)
            {
                if (IsDisplayableField(displayableFieldIds, field))
                {
                    displayableFields.Add(field);
                }
            }

            section.Fields.Clear();
            section.Fields.AddRange(displayableFields);
        }

        private bool IsDisplayableField(string displayableFieldIds, Editor.Field field)
        {
            if (IsStandardValues(field.ItemField.Item))
            {
                return true;
            }

            if (IsDisplayableFieldsField(field.ItemField))
            {
                return false;
            }

            return IsStandardTemplateField(field.ItemField)
                    || string.IsNullOrWhiteSpace(displayableFieldIds)
                    || displayableFieldIds.Contains(field.ItemField.ID.ToString());
        }
        
        private bool IsDisplayableFieldsField(Field field)
        {
            return string.Equals(field.Name, DisplayableFieldsFieldName, StringComparison.CurrentCultureIgnoreCase);
        }

        private static bool IsStandardValues(Item item)
        {
            if (item.Template.StandardValues != null)
            {
                return item.Template.StandardValues.ID == item.ID;
            }

            return false;
        }

        private bool IsStandardTemplateField(Field field)
        {
            Assert.IsNotNull(StandardTemplate, "The Stardard Template could not be found.");
            return StandardTemplate.ContainsField(field.ID);
        }

        private static Template GetStandardTemplate()
        {
            return TemplateManager.GetTemplate(Settings.DefaultBaseTemplate, Context.ContentDatabase);
        }

        private Template _StandardTemplate;
        private Template StandardTemplate
        {
            get
            {
                if (_StandardTemplate == null)
                {
                    _StandardTemplate = GetStandardTemplate();
                }

                return _StandardTemplate;
            }
        }

        private string DisplayableFieldsFieldName { get; set; }
    }
}

The class above iterates over all fields for the supplied item, and adds only those that were selected in the Displayable Fields TreelistEx field, and also Standard Fields — we don’t want to remove these since they are shown/hidden by the Standard Fields feature in Sitecore — to a new Editor.Fields collection. This new collection is then set on the GetContentEditorFieldsArgs instance.

Plus, we don’t want to show the Displayable Fields TreelistEx field when the feature is turned on, and we are on an item. This field should only display when we are on the standard values item when the feature is turned on — this is how we will choose our displayable fields.

Now we have to handle sections without fields — especially after ripping them out via the pipeline processor above.

I built the following class to serve as a <renderContentEditor> pipeline processor to do this:

using System.Linq;

using Sitecore.Diagnostics;
using Sitecore.Shell.Applications.ContentManager;
using Sitecore.Shell.Applications.ContentEditor.Pipelines.RenderContentEditor;

namespace Sitecore.Sandbox.Shell.Applications.ContentEditor.Pipelines.RenderContentEditor
{
    public class FilterSectionsWithFields
    {
        public void Process(RenderContentEditorArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            args.Sections = GetSectionsWithFields(args.Sections);
        }

        private static Editor.Sections GetSectionsWithFields(Editor.Sections sections)
        {
            Assert.ArgumentNotNull(sections, "sections");
            Editor.Sections sectionsWithFields = new Editor.Sections();
            foreach (Editor.Section section in sections)
            {
                AddIfContainsFields(sectionsWithFields, section);
            }

            return sectionsWithFields;
        }

        private static void AddIfContainsFields(Editor.Sections sections, Editor.Section section)
        {
            Assert.ArgumentNotNull(sections, "sections");
            Assert.ArgumentNotNull(section, "section");
            if (!ContainsFields(section))
            {
                return;
            }

            sections.Add(section);
        }

        private static bool ContainsFields(Editor.Section section)
        {
            Assert.ArgumentNotNull(section, "section");
            return section.Fields != null && section.Fields.Any();
        }
    }
}

It basically builds a new collection of sections that contain at least one field, and sets it on the RenderContentEditorArgs instance being passed through the <renderContentEditor> pipeline.

In order for this to work, we must make this pipeline processor run before all other processors.

I tied all of the above together with the following patch configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <commands>
      <command name="contenteditor:ToggleDisplayableFieldsVisibility" type="Sitecore.Sandbox.Commands.ToggleDisplayableFieldsVisibility, Sitecore.Sandbox"/>
    </commands>
    <pipelines>
      <getContentEditorFields>
        <processor type="Sitecore.Sandbox.Shell.Applications.ContentEditor.Pipelines.GetContentEditorFields.RemoveUndisplayableFields, Sitecore.Sandbox">
          <DisplayableFieldsFieldName>Displayable Fields</DisplayableFieldsFieldName>
        </processor>  
      </getContentEditorFields>
      <renderContentEditor>
        <processor patch:before="processor[@type='Sitecore.Shell.Applications.ContentEditor.Pipelines.RenderContentEditor.RenderSkinedContentEditor, Sitecore.Client']"
                   type="Sitecore.Sandbox.Shell.Applications.ContentEditor.Pipelines.RenderContentEditor.FilterSectionsWithFields, Sitecore.Sandbox"/>
      </renderContentEditor>
    </pipelines>
  </sitecore>
</configuration>

Let’s try this out.

I navigated to the standard values item for my Sample Item data template, and selected the fields I want to display in the content editor:

selected-some-fields

I then went to an item that uses this data template, and turned on the displayable fields feature:

displayable-fields-on

As you can see, only the fields we had chosen display — along with standard fields since the Standard Fields checkbox is checked.

I then turned off the displayable fields feature:

displayable-fields-off

Now all fields for the item display.

I then turned the displayable fields feature back on, and turned off Standard Fields:

standard-fields-off-displayable-fields-on

Now only our selected fields display.

If you have any thoughts on this, or ideas around making this better, please share in a comment.