Home » Files

Category Archives: Files

Abstract Out Sitecore FileWatcher Logic Which Monitors Rendering Files on the File System

While digging through code of Sitecore.IO.FileWatcher subclasses this weekend, I noticed a lot of code similarities between the LayoutWatcher and XslWatcher classes, and thought it might be a good idea to abstract this logic out into a new base abstract class so that future FileWatchers which monitor other renderings on the file system can easily be added without having to write much logic.

Before I move forward on how I did this, let me explain what the LayoutWatcher FileWatcher does. The LayoutWatcher FileWatcher clears the html cache of all websites defined in Sitecore when it determines that a layout file (a.k.a .aspx) or sublayout file (a.k.a .ascx) has been changed, deleted, renamed or added.

Likewise, the XslWatcher FileWatcher does the same thing for XSLT renderings but also clears the XSL cache along with the html cache.

Ok, now back to the abstraction. I came up with the following class to serve as the base class for any FileWatcher that will monitor renderings that live on the file system:

using System;
using System.Collections.Generic;
using System.Linq;

using Sitecore.Configuration;
using Sitecore.Diagnostics;
using Sitecore.IO;
using Sitecore.Web;

namespace Sitecore.Sandbox.IO.Watchers
{
    public abstract class RenderingFileWatcher : FileWatcher
    {
        private string RenderingFileModifiedMessage { get; set; }

        private string RenderingFileDeletedMessage { get; set; }

        private string RenderingFileRenamedMessage { get; set; }

        private string FileWatcherErrorMessage { get; set; }

        private object Owner { get; set; }

        public RenderingFileWatcher(string configPath)
            : base(configPath)
        {
            SetMessages();
        }

        private void SetMessages()
        {
            string modifiedMessage = GetRenderingFileModifiedMessage();
            AssertNotNullOrWhiteSpace(modifiedMessage, "modifiedMessage", "GetRenderingFileModifiedMessage() cannot return null, empty or whitespace!");
            RenderingFileModifiedMessage = modifiedMessage;

            string deletedMessage = GetRenderingFileDeletedMessage();
            AssertNotNullOrWhiteSpace(deletedMessage, "deletedMessage", "GetRenderingFileDeletedMessage() cannot return null, empty or whitespace!");
            RenderingFileDeletedMessage = deletedMessage;

            string renamedMessage = GetRenderingFileRenamedMessage();
            AssertNotNullOrWhiteSpace(renamedMessage, "renamedMessage", "GetRenderingFileRenamedMessage() cannot return null, empty or whitespace!");
            RenderingFileRenamedMessage = renamedMessage;

            string errorMessage = GetFileWatcherErrorMessage();
            AssertNotNullOrWhiteSpace(errorMessage, "errorMessage", "GetFileWatcherErrorMessage() cannot return null, empty or whitespace!");
            FileWatcherErrorMessage = errorMessage;

            object owner = GetOwner();
            Assert.IsNotNull(owner, "GetOwner() cannot return null!");
            Owner = owner;
        }

        protected abstract string GetRenderingFileModifiedMessage();

        protected abstract string GetRenderingFileDeletedMessage();

        protected abstract string GetRenderingFileRenamedMessage();

        protected abstract string GetFileWatcherErrorMessage();

        protected abstract object GetOwner();

        private void AssertNotNullOrWhiteSpace(string argument, string argumentName, string errorMessage)
        {
            Assert.ArgumentCondition(!string.IsNullOrWhiteSpace(argument), argumentName, errorMessage);
        }

        protected override void Created(string fullPath)
        {
            try
            {
                Log.Info(string.Format("{0}: {1}", RenderingFileModifiedMessage, fullPath), Owner);
                ClearCaches();
            }
            catch (Exception ex)
            {
                Log.Error(FileWatcherErrorMessage, ex, Owner);
            }
        }

        protected override void Deleted(string filePath)
        {
            try
            {
                Log.Info(string.Format("{0}: {1}", RenderingFileDeletedMessage, filePath), Owner);
                ClearCaches();
            }
            catch (Exception ex)
            {
                Log.Error(FileWatcherErrorMessage, ex, Owner);
            }
        }

        protected override void Renamed(string filePath, string oldFilePath)
        {
            try
            {
                Log.Info(string.Format("{0}: {1}. Old path: {2}", RenderingFileRenamedMessage, filePath, oldFilePath), Owner);
                ClearCaches();
            }
            catch (Exception ex)
            {
                Log.Error(FileWatcherErrorMessage, ex, this);
            }
        }

        protected virtual void ClearCaches()
        {
            ClearHtmlCaches();
        }

        protected virtual void ClearHtmlCaches()
        {
            IEnumerable<SiteInfo> siteInfos = GetSiteInfos();
            if (IsNullOrEmpty(siteInfos))
            {
                return;
            }

            foreach(SiteInfo siteInfo in siteInfos)
            {
                if (siteInfo.HtmlCache != null)
                {
                    Log.Info(string.Format("Clearing Html Cache for site: {0}", siteInfo.Name), Owner);
                    siteInfo.HtmlCache.Clear();
                }
            }
        }

        protected virtual IEnumerable<SiteInfo> GetSiteInfos()
        {
            IEnumerable<string> siteNames = GetSiteNames();
            if (IsNullOrEmpty(siteNames))
            {
                return Enumerable.Empty<SiteInfo>();
            }

            IList<SiteInfo> siteInfos = new List<SiteInfo>();
            foreach(string siteName in siteNames)
            {
                SiteInfo siteInfo = Factory.GetSiteInfo(siteName);
                if(siteInfo != null)
                {
                    siteInfos.Add(siteInfo);
                }
            }

            return siteInfos;
        }

        protected virtual IEnumerable<string> GetSiteNames()
        {
            IEnumerable<string> siteNames = Factory.GetSiteNames();
            if(IsNullOrEmpty(siteNames))
            {
                return Enumerable.Empty<string>();
            }

            return siteNames;
        }

        protected virtual bool IsNullOrEmpty<T>(IEnumerable<T> collection)
        {
            if (collection == null || !collection.Any())
            {
                return true;
            }

            return false;
        }
    }
}

The above class defines five abstract methods which subclasses must implement. Data returned by these methods are used when logging information or errors in the Sitecore log. The SetMessages() method vets whether subclass returned these objects correctly, and sets them in private properties which are used in the Created(), Deleted() and Renamed() methods.

The Created(), Deleted() and Renamed() methods aren’t really doing anything different from each other — they are all clearing the html cache for each Sitecore.Web.SiteInfo instance returned by the GetSiteInfos() method, though I do want to point out that each of these methods are defined on the Sitecore.IO.FileWatcher base class and serve as event handlers for file system file actions:

  • Created() is invoked when a new file is dropped in a directory or subdirectory being monitored, or when a targeted file is changed.
  • Deleted() is invoked when a targeted file is deleted.
  • Renamed() is invoked when a targeted file is renamed.

In order to ascertain whether the code above works, I need a subclass whose instance will serve as the actual FileWatcher. I decided to build the following subclass which will monitor Razor files under the /Views directory of my Sitecore website root:

namespace Sitecore.Sandbox.IO.Watchers
{
    public class RazorViewFileWatcher : RenderingFileWatcher
    {
        public RazorViewFileWatcher()
            : base("watchers/view")
        {
        }
        
        protected override string GetRenderingFileModifiedMessage()
        {
            return "Razor View modified";
        }

        protected override string GetRenderingFileDeletedMessage()
        {
            return "Razor View deleted";
        }

        protected override string GetRenderingFileRenamedMessage()
        {
            return "Razor View renamed";
        }

        protected override string GetFileWatcherErrorMessage()
        {
            return "Error in RazorViewFileWatcher";
        }

        protected override object GetOwner()
        {
            return this;
        }
    }
}

I’m sure Sitecore has another way of clearing/replenishing the html cache for Sitecore MVC View and Controller renderings — if you know how this works in Sitecore, please share in a comment, or better yet: please share in a blog post — but I went with this just for testing.

There’s not much going on in the above class. It’s just defining the needed methods for its base class to work its magic.

I then had to add the configuration needed by the RazorViewFileWatcher above in the following patch include configuration file:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <settings>
      <setting name="ViewsFolder" value="/Views" />
    </settings>
    <watchers>
      <view>
        <folder ref="settings/setting[@name='ViewsFolder']/@value"/>
        <filter>*.cshtml</filter>
      </view>
    </watchers>
  </sitecore>
</configuration>

As I’ve discussed in this post, FileWatchers in Sitecore are Http Modules — these must be registered under <modules> of <system.webServer> of your Web.config:


<!-- lots of stuff up here -->

<system.webServer>
    <modules runAllManagedModulesForAllRequests="true">

		<!-- stuff here -->
      
		<add type="Sitecore.Sandbox.IO.Watchers.RazorViewFileWatcher, Sitecore.Sandbox" name="SitecoreRazorViewFileWatcher" />

		<!-- stuff here as well -->

	</modules>

	<!-- more stuff down here -->

</system.webServer>

<!-- even more stuff down here -->

Let’s see if this works.

I decided to choose the following Razor file which comes with Web Forms For Marketers 8.1 Update-2:

razor-view-file-watcher-cshtml-1

I first performed a copy and paste of the Razor file into a new file:

razor-view-file-watcher-cshtml-2

I then deleted the new Razor file:

razor-view-file-watcher-cshtml-3

Next, I renamed the Razor file:

razor-view-file-watcher-cshtml-4

When I looked in my Sitecore log file, I saw that all operations were executed:

razor-view-file-watcher-log

If you have any thoughts on this, please drop a comment.

Advertisement

Augment the Sitecore UploadWatcher to Delete Files from the Upload Directory After Uploading to the Media Library

In my previous post I discussed the Sitecore UploadWatcher — a Sitecore.IO.FileWatcher which monitors the /upload directory of your Sitecore instance and uploads files dropped into that directory into the Media Library.

One thing I could never find in the UploadWatcher is functionality to delete files in the /upload directory after they are uploaded into the Media Library — if you know of an “out of the box” way of doing this, please drop a comment.

After peeking into Sitecore.Resources.Media.UploadWatcher using .NET Reflector, I discovered I could add this functionality quite easily since its Created() method — this method handles the uploading of the files into the Media Library by delegating to Sitecore.Resources.Media.MediaManager.Creator.FileCreated() — is overridable. In theory, all I would need to do would be to subclass the UploadWatcher class; override the Created() method; delegate to the base class’ Created() method to handle the upload of the file to the Media Library; then delete the file once the base class’ Create() method was done executing.

After coding, experimenting and refactoring, I built the following class that subclasses Sitecore.Resources.Media.UploadWatcher:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Xml;

using Sitecore.Configuration;
using Sitecore.Diagnostics;
using Sitecore.IO;
using Sitecore.Resources.Media;
using Sitecore.Xml;

namespace Sitecore.Sandbox.Resources.Media
{
    public class CleanupUploadWatcher : UploadWatcher
    {
        private IEnumerable<string> PathsToIgnore { get; set; }

        private IEnumerable<string> FileNamePartsToIgnore { get; set; }

        private bool ShouldDeleteAfterUpload { get; set; }

        public CleanupUploadWatcher()
            : base()
        {
            Initialize();
        }

        private void Initialize()
        {
            IEnumerable<XmlNode> configNodes = GetIgnoreConfigNodes();
            PathsToIgnore = GetPathsToIgnore(configNodes);
            FileNamePartsToIgnore = GetFileNamePartsToIgnore(configNodes);
            ShouldDeleteAfterUpload = GetShouldDeleteAfterUpload();
        }

        protected virtual IEnumerable<XmlNode> GetIgnoreConfigNodes()
        {
            string configNodeRoot = GetConfigNodeRoot();
            if (string.IsNullOrWhiteSpace(configNodeRoot))
            {
                return Enumerable.Empty<XmlNode>();
            }

            XmlNode rootNode = Factory.GetConfigNode(configNodeRoot);
            if (rootNode == null)
            {
                return Enumerable.Empty<XmlNode>();
            }

            return XmlUtil.GetChildNodes(rootNode, true);
        }

        protected virtual string GetConfigNodeRoot()
        {
            return "mediaLibrary/watcher/ignoreList";
        }

        protected virtual IEnumerable<string> GetPathsToIgnore(IEnumerable<XmlNode> nodes)
        {
            if (IsEmpty(nodes))
            {
                return Enumerable.Empty<string>();
            }

            HashSet<string> pathsToIgnore = new HashSet<string>();
            foreach (XmlNode node in nodes)
            {
                string containsValue = XmlUtil.GetAttribute("contains", node);
                if (ShouldAddContainsValue(containsValue, node, "ignorepath"))
                {
                    pathsToIgnore.Add(containsValue);
                }
            }

            return pathsToIgnore;
        }

        protected virtual IEnumerable<string> GetFileNamePartsToIgnore(IEnumerable<XmlNode> nodes)
        {
            if (IsEmpty(nodes))
            {
                return Enumerable.Empty<string>();
            }

            HashSet<string> partsToIgnore = new HashSet<string>();
            foreach (XmlNode node in nodes)
            {
                string containsValue = XmlUtil.GetAttribute("contains", node);
                if (ShouldAddContainsValue(containsValue, node, "ignore"))
                {
                    partsToIgnore.Add(containsValue);
                }
            }

            return partsToIgnore;
        }

        protected virtual bool ShouldAddContainsValue(string containsValue, XmlNode node, string targetNodeName)
        {
            return !string.IsNullOrWhiteSpace(containsValue)
                    && node != null
                    && string.Equals(node.Name, targetNodeName, StringComparison.OrdinalIgnoreCase);
        }

        protected static bool IsEmpty<T>(IEnumerable<T> collection)
        {
            return collection == null || !collection.Any();
        }

        protected override void Created(string filePath)
        {
            Assert.ArgumentNotNullOrEmpty(filePath, "filePath");
            if (!ShouldIgnoreFile(filePath))
            {
                base.Created(filePath);
                DeleteFile(filePath);
            }
        }

        protected virtual bool GetShouldDeleteAfterUpload()
        {
            XmlNode node = Factory.GetConfigNode("watchers/media/additionalSettings");
            if (node == null)
            {
                return false;
            }

            string deleteAfterUploadValue = XmlUtil.GetAttribute("deleteAfterUpload", node);
            if(string.IsNullOrWhiteSpace(deleteAfterUploadValue))
            {
                return false;
            }
            
            bool shouldDelete;
            bool.TryParse(deleteAfterUploadValue, out shouldDelete);
            return shouldDelete;
        }

        protected virtual bool ShouldIgnoreFile(string filePath)
        {
            Assert.ArgumentNotNullOrEmpty(filePath, "filePath");
            foreach (string path in PathsToIgnore)
            {
                if (ContainsSubstring(filePath, path))
                {
                    return true;
                }
            }

            string fileName = Path.GetFileName(filePath);
            foreach (string part in FileNamePartsToIgnore)
            {
                if(ContainsSubstring(fileName, part))
                {
                    return true;
                }
            }

            return false;
        }

        protected virtual bool ContainsSubstring(string value, string substring)
        {
            if(string.IsNullOrWhiteSpace(value) || string.IsNullOrWhiteSpace(substring))
            {
                return false;
            }

            return value.IndexOf(substring, StringComparison.OrdinalIgnoreCase) > -1;
        }

        protected virtual void DeleteFile(string filePath)
        {
            Assert.ArgumentNotNullOrEmpty(filePath, "filePath");
            if(!ShouldDeleteAfterUpload)
            {
                return;
            }

            try
            {
                FileUtil.Delete(filePath);
            }
            catch(Exception ex)
            {
                Log.Error(ToString(), ex, this);
            }
        }
    }
}

You might thinking “Mike, there is more going on in here than just delegating to the base class’ Created() method and deleting the file. Well, you are correct — I had to do a few things further than what I thought I needed to do, and I’ll explain why.

I had to duplicate the logic — actually I wrote my own logic — to parse the Sitecore configuration which defines the substrings of file names and paths to ignore since these collections on the base UploadWatcher class are private — subclasses cannot access these collections — in order to prevent the code from deleting a file that should be ignored.

I also wedged in configuration with code to turn off the delete functionality if needed.

I then created the following patch include configuration file to hold the configuration setting to turn the delete functionality on/off:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <watchers>
      <media>
        <additionalSettings deleteAfterUpload="true" />
      </media>
    </watchers>
  </sitecore>
</configuration>

I then registered the new CleanupUploadWatcher in the Web.config — please see my previous post which explains why this is needed:

<system.webServer>
    <modules runAllManagedModulesForAllRequests="true">

      <!-- stuff here -->

      <!-- <add type="Sitecore.Resources.Media.UploadWatcher, Sitecore.Kernel" name="SitecoreUploadWatcher" /> -->
      <add type="Sitecore.Sandbox.Resources.Media.CleanupUploadWatcher, Sitecore.Sandbox" name="SitecoreUploadWatcher" />

      <!-- more stuff down here -->

    </modules>

    <!-- and more stuff down here -->
    
<system.webServer>

Let’s see this in action!

As you can see, there is no Media Library Item in the root of my Media Library (/sitecore/media library) — this is where the UploadWatcher uploads the file to:

cleanup-uploadwatcher-media-library-no-file

I then copied an image from a folder on my Desktop to the /upload directory of my Sitecore instance:

cleanup-uploadwatcher-move-file

As you can see above, the image is deleted after it is dropped.

I then went back to my Media Library and refreshed. As you can see here, the file was uploaded:

cleanup-uploadwatcher-media-library-file

If you have any thoughts on this or suggestions on making it better, please share in a comment.

Download Images and Save to the Media Library Via a Custom Content Editor Image Field in Sitecore

Yesterday evening — a Friday evening by the way (what, you don’t code on Friday evenings? 😉 ) — I wanted to have a bit of fun by building some sort of customization in Sitecore but was struggling on what to build.

After about an hour of pondering, it dawned on me: I was determined to build a custom Content Editor Image field that gives content authors the ability to download images from a supplied URL; save the image to disk; upload the image into the Media Libary; and then set it on a custom Image field of an Item.

I’m sure someone has built something like this in the past and may have even uploaded a module that does this to the Sitecore Marketplace — I didn’t really look into whether this had already been done before since I wanted to have some fun by taking on the challenge. What follows is the fruit of that endeavor.

Before I move forward, I would like to caution you on using the code that follows — I have not rigorously tested this code at all so use at your own risk.

Before I began coding, I thought about how I wanted to approach this challenge. I decided I would build a custom Sitecore pipeline to handle this code. Why? Well, quite frankly, it gives you flexibility on customization, and also native Sitecore code is hugely composed of pipelines — why deviate from the framework?

First, I needed a class whose instances would serve as the custom pipeline’s arguments object. The following class was built for that:

using Sitecore.Data;
using Sitecore.Pipelines;

namespace Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary
{
    public class DownloadImageToMediaLibraryArgs : PipelineArgs
    {
        public Database Database { get; set; }

        public string ImageFileName { get; set; }

        public string ImageFilePath { get; set; }

        public string ImageItemName { get; set; }

        public string ImageUrl { get; set; }

        public string MediaId { get; set; }

        public string MediaLibaryFolderPath { get; set; }

        public string MediaPath { get; set; }

        public bool FileBased { get; set; }

        public bool IncludeExtensionInItemName { get; set; }

        public bool OverwriteExisting { get; set; }

        public bool Versioned { get; set; }
    }
}

I didn’t just start off with all of the properties you see on this class — it was an iterative process where I had to go back, add more and even remove some that were no longer needed. You will see why I have these on it from the code below.

I decided to employ the Template method pattern in this code, and defined the following abstract base class which all processors of my custom pipeline will sub-class:

using Sitecore.Diagnostics;

namespace Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary
{
    public abstract class DownloadImageToMediaLibraryProcessor
    {
        public void Process(DownloadImageToMediaLibraryArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if(!CanProcess(args))
            {
                AbortPipeline(args);
                return;
            }

            Execute(args);
        }

        protected abstract bool CanProcess(DownloadImageToMediaLibraryArgs args);

        protected virtual void AbortPipeline(DownloadImageToMediaLibraryArgs args)
        {
            args.AbortPipeline();
        }

        protected abstract void Execute(DownloadImageToMediaLibraryArgs args);
    }
}

All processors of the custom pipeline will have to implement the CanProcess and Execute methods above, and also have the ability to redefine the AbortPipeline method if needed.

The main magic for all processors happen in the Process method above — if the processor can process the data supplied via the arguments object, then it will do so using the Execute method. Otherwise, the pipeline will be aborted via the AbortPipeline method.

The following class serves as the first processor of the custom pipeline.

using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.IO;

namespace Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary
{
    public class SetProperties : DownloadImageToMediaLibraryProcessor
    {
        private string UploadDirectory { get; set; }

        protected override bool CanProcess(DownloadImageToMediaLibraryArgs args)
        {
            Assert.IsNotNullOrEmpty(UploadDirectory, "UploadDirectory must be set in configuration!");
            Assert.IsNotNull(args.Database, "args.Database must be supplied!");
            return !string.IsNullOrWhiteSpace(args.ImageUrl)
                && !string.IsNullOrWhiteSpace(args.MediaLibaryFolderPath);
        }

        protected override void Execute(DownloadImageToMediaLibraryArgs args)
        {
            args.ImageFileName = GetFileName(args.ImageUrl);
            args.ImageItemName = GetImageItemName(args.ImageUrl);
            args.ImageFilePath = GetFilePath(args.ImageFileName);
        }

        protected virtual string GetFileName(string url)
        {
            Assert.ArgumentNotNullOrEmpty(url, "url");
            return FileUtil.GetFileName(url);
        }

        protected virtual string GetImageItemName(string url)
        {
            Assert.ArgumentNotNullOrEmpty(url, "url");
            string fileNameNoExtension = GetFileNameNoExtension(url);
            if(string.IsNullOrWhiteSpace(fileNameNoExtension))
            {
                return string.Empty;
            }

            return ItemUtil.ProposeValidItemName(fileNameNoExtension);
        }

        protected virtual string GetFileNameNoExtension(string url)
        {
            Assert.ArgumentNotNullOrEmpty(url, "url");
            return FileUtil.GetFileNameWithoutExtension(url);
        }

        protected virtual string GetFilePath(string fileName)
        {
            Assert.ArgumentNotNullOrEmpty(fileName, "fileName");
            return string.Format("{0}/{1}", FileUtil.MapPath(UploadDirectory), fileName);
        }
    }
}

Instances of the above class will only run if an upload directory is supplied via configuration (see the patch include configuration file down below); a Sitecore Database is supplied (we have to upload this image somewhere); an image URL is supplied (can’t download an image without this); and a Media Library folder is supplied (where are we storing this image?).

The Execute method then sets additional properties on the arguments object that the next processors will need in order to complete their tasks.

The following class serves as the second processor of the custom pipeline. This processor will download the image from the supplied URL:

using System.Net;

namespace Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary
{
    public class DownloadImage : DownloadImageToMediaLibraryProcessor
    {
        protected override bool CanProcess(DownloadImageToMediaLibraryArgs args)
        {
            return !string.IsNullOrWhiteSpace(args.ImageUrl)
                && !string.IsNullOrWhiteSpace(args.ImageFilePath);
        }

        protected override void Execute(DownloadImageToMediaLibraryArgs args)
        {
            using (WebClient client = new WebClient())
            {
                client.DownloadFile(args.ImageUrl, args.ImageFilePath);
            }
        }
    }
}

The processor instance of the above class will only execute when an image URL is supplied and a location on the file system is given — this is the location on the file system where the image will live before being uploaded into the Media Library.

If all checks out, the image is downloaded from the given URL into the specified location on the file system.

The next class serves as the third processor of the custom pipeline. This processor will upload the image on disk to the Media Library:

using System.IO;

using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Resources.Media;
using Sitecore.Sites;

namespace Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary
{
    public class UploadImageToMediaLibrary : DownloadImageToMediaLibraryProcessor
    {
        private string Site { get; set; }

        protected override bool CanProcess(DownloadImageToMediaLibraryArgs args)
        {
            Assert.IsNotNullOrEmpty(Site, "Site must be set in configuration!");
            return !string.IsNullOrWhiteSpace(args.MediaLibaryFolderPath)
                && !string.IsNullOrWhiteSpace(args.ImageItemName)
                && !string.IsNullOrWhiteSpace(args.ImageFilePath)
                && args.Database != null;
        }

        protected override void Execute(DownloadImageToMediaLibraryArgs args)
        {
            MediaCreatorOptions options = new MediaCreatorOptions
            {
                Destination = GetMediaLibraryDestinationPath(args),
                FileBased = args.FileBased,
                IncludeExtensionInItemName = args.IncludeExtensionInItemName,
                OverwriteExisting = args.OverwriteExisting,
                Versioned = args.Versioned,
                Database = args.Database
            };
            
            MediaCreator creator = new MediaCreator();
            MediaItem mediaItem;
            using (SiteContextSwitcher switcher = new SiteContextSwitcher(GetSiteContext()))
            {
                using (FileStream fileStream = File.OpenRead(args.ImageFilePath))
                {
                    mediaItem = creator.CreateFromStream(fileStream, args.ImageFilePath, options);
                }
            }
            
            if (mediaItem == null)
            {
                AbortPipeline(args);
                return;
            }
            
            args.MediaId = mediaItem.ID.ToString();
            args.MediaPath = mediaItem.MediaPath;
        }

        protected virtual SiteContext GetSiteContext()
        {
            SiteContext siteContext = SiteContextFactory.GetSiteContext(Site);
            Assert.IsNotNull(siteContext, string.Format("The site: {0} does not exist!", Site));
            return siteContext;
        }

        protected virtual string GetMediaLibraryDestinationPath(DownloadImageToMediaLibraryArgs args)
        {
            return string.Format("{0}/{1}", args.MediaLibaryFolderPath, args.ImageItemName);
        }
    }
}

The processor instance of the class above will only run when we have a Media Library folder location; an Item name for the image; a file system path for the image; and a Database to upload the image to. I also ensure a “site” is supplied via configuration so that I can switch the site context — when using the default of “shell”, I was being brought to the image Item in the Media Library after it was uploaded which was causing the image not to be set on the custom Image field on the Item.

If everything checks out, we upload the image to the Media Library in the specified location.

The next class serves as the last processor of the custom pipeline. This processor just deletes the image from the file system (why keep it around since we are done with it?):

using Sitecore.IO;

namespace Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary
{
    public class DeleteImageFromFileSystem : DownloadImageToMediaLibraryProcessor
    {
        protected override bool CanProcess(DownloadImageToMediaLibraryArgs args)
        {
            return !string.IsNullOrWhiteSpace(args.ImageFilePath)
                && FileUtil.FileExists(args.ImageFilePath);
        }

        protected override void Execute(DownloadImageToMediaLibraryArgs args)
        {
            FileUtil.Delete(args.ImageFilePath);
        }
    }
}

The processor instance of the class above can only delete the image if its path is supplied and the file exists.

If all checks out, the image is deleted.

The next class is the class that serves as the custom Image field:

using System;
using System.Net;

using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Pipelines;
using Sitecore.Shell.Framework;
using Sitecore.Web.UI.Sheer;

using Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary;

namespace Sitecore.Sandbox.Shell.Applications.ContentEditor
{
    public class Image : Sitecore.Shell.Applications.ContentEditor.Image
    {
        public Image()
            : base()
        {
        }

        public override void HandleMessage(Message message)
        {
            Assert.ArgumentNotNull(message, "message");
            if (string.Equals(message.Name, "contentimage:download", StringComparison.CurrentCultureIgnoreCase))
            {
                GetInputFromUser();
                return;
            }

            base.HandleMessage(message);
        }

        protected void GetInputFromUser()
        {
            RunProcessor("GetImageUrl", new ClientPipelineArgs());
        }

        protected virtual void GetImageUrl(ClientPipelineArgs args)
        {
            if (!args.IsPostBack)
            {
                SheerResponse.Input("Enter the url of the image to download:", string.Empty);
                args.WaitForPostBack();
            }
            else if (args.HasResult && IsValidUrl(args.Result))
            {
                args.Parameters["imageUrl"] = args.Result;
                args.IsPostBack = false;
                RunProcessor("ChooseMediaLibraryFolder", args);
            }
            else
            {
                CancelOperation(args);
            }
        }

        protected virtual bool IsValidUrl(string url)
        {
            if (string.IsNullOrWhiteSpace(url))
            {
                return false;
            }

            try
            {
                HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
                request.Method = "HEAD";
                request.GetResponse();
            }
            catch (Exception ex)
            {
                SheerResponse.Alert("The specified url is not valid. Please try again.");
                return false;
            }

            return true;
        }

        protected virtual void RunProcessor(string processor, ClientPipelineArgs args)
        {
            Assert.ArgumentNotNullOrEmpty(processor, "processor");
            Sitecore.Context.ClientPage.Start(this, processor, args);
        }

        public void ChooseMediaLibraryFolder(ClientPipelineArgs args)
        {
            if (!args.IsPostBack)
            {
                Dialogs.BrowseItem
                (
                    "Select A Media Library Folder",
                    "Please select a media library folder to store this image.",
                    "Applications/32x32/folder_into.png",
                    "OK",
                    "/sitecore/media library", 
                    string.Empty
                );

                args.WaitForPostBack();
            }
            else if (args.HasResult)
            {
                Item folder = Client.ContentDatabase.Items[args.Result];
                args.Parameters["mediaLibaryFolderPath"] = folder.Paths.FullPath;
                RunProcessor("DownloadImage", args);
            }
            else
            {
                CancelOperation(args);
            }
        }

        protected virtual void DownloadImage(ClientPipelineArgs args)
        {
            DownloadImageToMediaLibraryArgs downloadArgs = new DownloadImageToMediaLibraryArgs
            {
                Database = Client.ContentDatabase,
                ImageUrl = args.Parameters["imageUrl"],
                MediaLibaryFolderPath = args.Parameters["mediaLibaryFolderPath"]
            };

            CorePipeline.Run("downloadImageToMediaLibrary", downloadArgs);
            SetMediaItemInField(downloadArgs);
        }

        protected virtual void SetMediaItemInField(DownloadImageToMediaLibraryArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if(string.IsNullOrWhiteSpace(args.MediaId) || string.IsNullOrWhiteSpace(args.MediaPath))
            {
                return;
            }

            XmlValue.SetAttribute("mediaid", args.MediaId);
            Value = args.MediaPath;
            Update();
            SetModified();
        }

        protected virtual void CancelOperation(ClientPipelineArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            args.AbortPipeline();
        }
    }
}

The class above subclasses the Sitecore.Shell.Applications.ContentEditor.Image class — this lives in Sitecore.Kernel.dll — which is the “out of the box” Content Editor Image field. The Sitecore.Shell.Applications.ContentEditor.Image class provides hooks that we can override in order to augment functionality which I am doing above.

The magic of this class starts in the HandleMessage method — I intercept the message for a Menu item option that I define below for downloading an image from a URL.

If we are to download an image from a URL, we first prompt the user for a URL via the GetImageUrl method using a Sheer UI api call (note: I am running these methods as one-off client pipeline processors as this is the only way you can get Sheer UI to run properly).

If we have a valid URL, we then prompt the user for a Media Library location via another Sheer UI dialog (this is seen in the ChooseMediaLibraryFolder method).

If the user chooses a location in the Media Library, we then call the DownloadImage method as a client pipeline processor — I had to do this since I was seeing some weird behavior on when the image was being saved into the Media Library — which invokes the custom pipeline for downloading the image to the file system; uploading it into the Media Library; and then removing it from disk.

I then duct-taped everything together using the following patch include configuration file:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <controlSources>
      <source mode="on" namespace="Sitecore.Sandbox.Shell.Applications.ContentEditor" assembly="Sitecore.Sandbox" prefix="sandbox-content"/>
    </controlSources>
    <overrideDialogs>
      <override dialogUrl="/sitecore/shell/Applications/Item%20browser.aspx" with="/sitecore/client/applications/dialogs/InsertSitecoreItemViaTreeDialog">
        <patch:delete/>
      </override>
    </overrideDialogs>
    <pipelines>
      <downloadImageToMediaLibrary>
        <processor type="Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary.SetProperties, Sitecore.Sandbox">
          <UploadDirectory>/upload</UploadDirectory>
        </processor>  
        <processor type="Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary.DownloadImage, Sitecore.Sandbox" />
        <processor type="Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary.UploadImageToMediaLibrary, Sitecore.Sandbox">
          <Site>website</Site>
        </processor>
        <processor type="Sitecore.Sandbox.Pipelines.DownloadImageToMediaLibrary.DeleteImageFromFileSystem, Sitecore.Sandbox" />
      </downloadImageToMediaLibrary>
    </pipelines>
  </sitecore>
</configuration>

One thing to note in the above file: I’ve disabled the SPEAK dialog for the Item Browser — you can see this in the <overrideDialogs> xml element — as I wasn’t able to set messaging text on it but could do so using the older Sheer UI dialog.

Now that all code is in place, we need to tell Sitecore that we have a new Image field. I do so by defining it in the Core database:

external-image-core-1

We also need a new Menu item for the “Download Image” link:

external-image-core-2

Let’s take this for a spin!

I added a new field using the custom Image field type to my Sample item template:

template-new-field-external-image

As you can see, we have this new Image field on my “out of the box” home item. Let’s click the “Download Image” link:

home-external-image-1

I was then prompted with a dialog to supply an image URL. I pasted one I found on the internet:

home-external-image-2

After clicking “OK”, I was prompted with another dialog to choose a Media Library location for storing the image. I chose some random folder:

home-external-image-3

After clicking “OK” on that dialog, the image was magically downloaded from the internet; uploaded into the Media Library; and set in the custom Image field on my home item:

home-external-image-4

If you have any comments, thoughts or suggestions on this, please drop a comment.

Addendum:
It’s not a good idea to use the /upload directory for temporarily storing download images — see this post for more details.

If you decide to use this solution — by the way, use this solution at your own risk 😉 — you will have to change the /upload directory in the patch include configuration file above.

Restart the Sitecore Server Using a Custom FileWatcher

For a few months now, I’ve been contemplating potential uses for a custom Sitecore.IO.FileWatcher — this lives in Sitecore.Kernel.dll, and defines abstract methods to handle changes to files on the file system within your Sitecore web application — and finally came up with something: how about a FileWatcher that restarts the Sitecore server when a certain file is uploaded to a specific directory?

You might be thinking “why would I ever want use such a thing?” Well, suppose you need to restart the Sitecore server on one of your Content Delivery Servers immediately, but you do not have direct access to it, and the person who does has left for the week. What do you do?

The following FileWatcher might be one option for the scenario above (another option might be to make frantic phone calls to get the server restarted):

using System;

using Sitecore.Diagnostics;
using Sitecore.Install;
using Sitecore.IO;

namespace Sitecore.Sandbox.IO
{
    public class RestartServerWatcher : FileWatcher
    {
        public RestartServerWatcher()
            : base("watchers/restartServer")
        {
        }

        protected override void Created(string fullPath)
        {
            try
            {
                Log.Info(string.Format("Restart server file detected: {0}. Restarting the server.", fullPath), this);
                FileUtil.Delete(fullPath);
                Installer.RestartServer();
            }
            catch (Exception exception)
            {
                Log.Error("Error in RestartServerWatcher", exception, typeof(RestartServerWatcher));
            }
        }
        
        protected override void Deleted(string filePath)
        {
            return;
        }

        protected override void Renamed(string filePath, string oldFilePath)
        {
            return;
        }
    }
}

All of the magic occurs in the Created() method above — we do not care if the file is renamed or deleted. If the file is detected, the code in the Created() method logs information to the Sitecore log, deletes the file, and then initiates a Sitecore server restart.

I created the following patch configuration file for the RestartServerWatcher class above:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <watchers>
      <restartServer>
        <folder>/restart</folder>
        <filter>restart-server.txt</filter>
      </restartServer>
    </watchers>
  </sitecore>
</configuration>

Since FileWatchers are HttpModules, I had to register the RestartServerWatcher in the <system.webServer> section of my Web.config (this configuration element lives outside of the <sitecore> configuration element, and cannot be mapped via a Sitecore patch configuration file):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
<!-- Lots of stuff up here -->
<system.webServer>
	<!-- Some stuff here -->
	<add type="Sitecore.Sandbox.IO.RestartServerWatcher, Sitecore.Sandbox" name="SitecoreRestartServerWatcher"/>
</system.webServer>
<!-- More stuff down here -->
</configuration>

For testing, I uploaded my target file into the target location via the Sitecore File Explorer to trigger a Sitecore server restart:

file-explorer-wizard-upload

I then opened up my Sitecore log, and saw the following entries:

restart-server-log-file

If you have any thoughts on this, or have other ideas for custom FileWatchers, please share in a comment.

Restrict Certain Files from Being Uploaded Through Web Forms for Marketers Forms in Sitecore: an Alternative Approach

This past weekend I noticed the <formUploadFile> pipeline in Web Forms for Marketers (WFFM), and wondered whether I could create an alternative solution to the one I had shared in this post — I had built a custom WFFM field type to prevent certain files from being uploaded through WFFM forms.

After some tinkering, I came up the following class to serve as a <formUploadFile> pipeline processor:

using System;
using System.Collections.Generic;
using System.Web;

using Sitecore.Diagnostics;

using Sitecore.Form.Core.Pipelines.FormUploadFile;

namespace Sitecore.Sandbox.Form.Pipelines.FormUploadFile
{
    public class CheckMimeTypesNotAllowed
    {
        static CheckMimeTypesNotAllowed()
        {
            MimeTypesNotAllowed = new List<string>();
        }

        public void Process(FormUploadFileArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            string mimeType = GetMimeType(args.File.FileName);
            if (IsMimeTypeAllowed(mimeType))
            {
                return;
            }

            throw new Exception(string.Format("Uploading a file with MIME type {0} is not allowed", mimeType));
        }

        protected virtual string GetMimeType(string fileName)
        {
            Assert.ArgumentNotNullOrEmpty(fileName, "fileName");
            return MimeMapping.GetMimeMapping(fileName);
        }

        protected virtual bool IsMimeTypeAllowed(string mimeType)
        {
            foreach (string mimeTypeNotAllowed in MimeTypesNotAllowed)
            {
                if (AreEqualIgnoreCase(mimeTypeNotAllowed, mimeType))
                {
                    return false;
                }    
            }

            return true;
        }

        private static bool AreEqualIgnoreCase(string stringOne, string stringTwo)
        {
            return string.Equals(stringOne, stringTwo, StringComparison.CurrentCultureIgnoreCase);
        }

        private static void AddMimeTypeNotAllowed(string mimeType)
        {
            if (string.IsNullOrWhiteSpace(mimeType) || MimeTypesNotAllowed.Contains(mimeType))
            {
                return;
            }

            MimeTypesNotAllowed.Add(mimeType);
        }

        private static IList<string> MimeTypesNotAllowed { get; set; }
    }
}

The class above adds restricted MIME types to a list — these MIME types are defined in a configuration file shown below — and checks to see if the uploaded file’s MIME type is in the restricted list. If it is restricted, an exception is thrown with some information — throwing an exception in WFFM prevents the form from being submitted.

MIME types are inferred using System.Web.MimeMapping.GetMimeMapping(string fileName), a method that is new in .NET 4.5 (this solution will not work in older versions of .NET).

I then registered the class above as a <formUploadFile> pipeline processor via the following configuration file:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:x="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <formUploadFile>
        <processor patch:before="processor[@type='Sitecore.Form.Core.Pipelines.FormUploadFile.Save, Sitecore.Forms.Core']"
                   type="Sitecore.Sandbox.Form.Pipelines.FormUploadFile.CheckMimeTypesNotAllowed">
          <mimeTypesNotAllowed hint="list:AddMimeTypeNotAllowed">
            <mimeType>application/octet-stream</mimeType>
          </mimeTypesNotAllowed >
        </processor>  
      </formUploadFile>
    </pipelines>
  </sitecore>
</configuration>

In case you are wondering, the MIME type being passed to the processor in the configuration file is for executables.

Let’s see how we did. 😉

I whipped up a test form, and pulled it up in a browser:

form-for-testing

I then selected an executable:

select-executable

I saw this after an attempt to submit the form:

form-upload-exception

And my log file expressed why:

exception-in-log

I then copied a jpeg into my test folder, and selected it to be uploaded:

select-jpg

After clicking the submit button, I was given a nice confirmation message:

success-upload

There is one problem with the solution above that I would like to point out: it does not address the issue of file extensions being changed. I could not solve this problem since the MIME type of the file being uploaded cannot be determined from the Sitecore.Form.Core.Media.PostedFile instance set in the arguments object: there is no property for it. 😦

If you know of another way to determine MIME types on Sitecore.Form.Core.Media.PostedFile instances, or have other ideas for restricting certain files from being uploaded through WFFM, please share in a comment.

Restrict IP Access of Directories and Files in Your Sitecore Web Application Using a httpRequestBegin Pipeline Processor

Last week my friend and colleague Greg Coffman had asked me if I knew of a way to restrict IP access to directories within the Sitecore web application, and I recalled reading a post by Alex Shyba quite some time ago.

Although Alex’s solution is probably good enough in most circumstances, I decided to explore other solutions, and came up with the following <httpRequestBegin> pipeline processor as another way to accomplish this:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web.Hosting;

using Sitecore.Configuration;
using Sitecore.Diagnostics;
using Sitecore.Pipelines.HttpRequest;
using Sitecore.Web;

namespace Sitecore.Sandbox.Pipelines.HttpRequest
{
    public class FilePathRestrictor : HttpRequestProcessor 
    {
        public override void Process(HttpRequestArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            if (!ShouldRedirect(args))
            {
                return;
            }

            RedirectToNoAccessUrl();
        }

        private bool ShouldRedirect(HttpRequestArgs args)
        {
            return CanProcess(args, GetFilePath(args)) 
                    && !CanAccess(args.Context.Request.UserHostAddress);
        }

        protected virtual string GetFilePath(HttpRequestArgs args)
        {
            if (string.IsNullOrWhiteSpace(Context.Page.FilePath))
            {
                return args.Url.FilePath;
            }

            return Context.Page.FilePath;
        }

        protected virtual bool CanProcess(HttpRequestArgs args, string filePath)
        {
            return !string.IsNullOrWhiteSpace(filePath)
                    && !string.IsNullOrWhiteSpace(RootFilePath)
                    && AllowedIPs != null
                    && AllowedIPs.Any()
                    && (HostingEnvironment.VirtualPathProvider.DirectoryExists(filePath)
                        || HostingEnvironment.VirtualPathProvider.FileExists(filePath))
                    && args.Url.FilePath.StartsWith(RootFilePath, StringComparison.CurrentCultureIgnoreCase)
                    && !string.IsNullOrWhiteSpace(args.Context.Request.UserHostAddress)
                    && !string.Equals(filePath, Settings.NoAccessUrl, StringComparison.CurrentCultureIgnoreCase);
        }

        protected virtual bool CanAccess(string ip)
        {
            Assert.ArgumentNotNullOrEmpty(ip, "ip");
            return AllowedIPs.Contains(ip);
        }

        protected virtual void RedirectToNoAccessUrl()
        {
            WebUtil.Redirect(Settings.NoAccessUrl);
        }

        protected virtual void AddAllowedIP(string ip)
        {
            if (string.IsNullOrWhiteSpace(ip) || AllowedIPs.Contains(ip))
            {
                return;
            }

            AllowedIPs.Add(ip);
        }

        private string RootFilePath { get; set; }

        private IList<string> _AllowedIPs;
        private IList<string> AllowedIPs 
        {
            get
            {
                if (_AllowedIPs == null)
                {
                    _AllowedIPs = new List<string>();
                }

                return _AllowedIPs;
            }
        }
    }
}

The pipeline processor above determines whether the IP making the request has access to the directory or file on the file system — a list of IP addresses that should have access are passed to the pipeline processor via a configuration file, and the code does check to see if the requested URL is a directory or a file on the file system — by matching the beginning of the URL with a configuration defined root path.

If the user does not have access to the requested path, s/he is redirected to the “No Access Url” which is specified in the Sitecore instance’s configuration.

The list of IP addresses that should have access to the directory — including everything within it — and the root path are handed to the pipeline processor via the following patch configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <httpRequestBegin>
        <processor patch:before=" processor[@type='Sitecore.Pipelines.HttpRequest.FileResolver, Sitecore.Kernel']"
                   type="Sitecore.Sandbox.Pipelines.HttpRequest.FilePathRestrictor, Sitecore.Sandbox">
          <RootFilePath>/sitecore</RootFilePath>
          <AllowedIPs hint="list:AddAllowedIP">
            <IP>127.0.0.2</IP>
          </AllowedIPs>
        </processor>
      </httpRequestBegin>
    </pipelines>
  </sitecore>
</configuration>

Since my IP is 127.0.0.1, I decided to only allow 127.0.0.2 access to my Sitecore directory — this also includes everything within it — in the above configuration file for testing.

After navigating to /sitecore of my local sandbox instance, I was redirected to the “No Access Url” page defined in my Web.config:

no-access

If you have any thoughts on this, or know of other solutions, please share in a comment.

Restrict Certain Files from Being Attached to Web Forms for Marketers Forms in Sitecore

Last week I was given a task to research how to prevent certain files from being attached to Web Forms for Marketers (WFFM) forms: basically files that have certain extensions, or files that exceed a specified size.

I have not seen this done before in WFFM, so I did what comes naturally to me: I Googled! 🙂

After a few unsuccessful searches on the internet — if you have some examples on how others have accomplished this in WFFM, please share in a comment — I decided to dig into the WFFM assemblies to see what would be needed to accomplish this, and felt using custom WFFM field validators would be the way to go.

I thought having a custom validator to check the attached file’s MIME type would be a better solution over one that checks the file’s extension — thanks to Sitecore MVP Yogesh Patel for giving me the idea from his post on restricting certain files from being uploading into Sitecore by checking their MIME types — since a malefactor could attach a restricted file with a different extension to bypass the extension validation step.

That thought lead me to build the following custom WFFM field validator:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web.UI.WebControls;

using Sitecore.Form.Core.Validators;
using Sitecore.Form.Web.UI.Controls;

namespace Sitecore.Sandbox.Form.Core.Validators
{
    public class RestrictedFileTypes : FormCustomValidator
    {
        public string MimeTypesNotAllowed
        {
            get
            {
                if (string.IsNullOrWhiteSpace(base.classAttributes["mimeTypesNotAllowed"]))
                {
                    return string.Empty;
                }

                return base.classAttributes["mimeTypesNotAllowed"];
            }
            set
            {
                base.classAttributes["mimeTypesNotAllowed"] = value;
            }
        }

        public RestrictedFileTypes()
        {
        }

        protected override bool OnServerValidate(string value)
        {
            IEnumerable<string> mimeTypesNotAllowed = GetMimeTypesNotAllowed();
            FileUpload fileUpload = FindControl(ControlToValidate) as FileUpload;
            bool canProcess = mimeTypesNotAllowed.Any() && fileUpload != null && fileUpload.HasFile;
            if (!canProcess)
            {
                return true;
            }

            foreach(string mimeType in mimeTypesNotAllowed)
            {
                if (AreEqualIgnoreCase(mimeType, fileUpload.PostedFile.ContentType))
                {
                    return false;
                }
            }

            return true;
        }

        private IEnumerable<string> GetMimeTypesNotAllowed()
        {
            if (string.IsNullOrWhiteSpace(MimeTypesNotAllowed))
            {
                return new List<string>();
            }

            return MimeTypesNotAllowed.Split(new []{',', ';'}, StringSplitOptions.RemoveEmptyEntries);
        }

        private static bool AreEqualIgnoreCase(string stringOne, string stringTwo)
        {
            return string.Equals(stringOne, stringTwo, StringComparison.CurrentCultureIgnoreCase);
        }
    }
}

Restricted MIME types are passed to the custom validator through a parameter named MimeTypesNotAllowed, and these are injected into a property of the same name.

The MIME types can be separated by commas or semicolons, and the code above splits the string along these delimiters into a collection, checks to see if the uploaded file — we get the uploaded file via the FileUpload control on the form for the field we are validating — has a restricted MIME type by iterating over the collection of restricted MIME types, and comparing each entry to its MIME type. If there is a match, we return “false”: basically the field is invalid.

If no MIME types were set for the validator, or no file was uploaded, we return “true”: the field is valid. We do this for the case where the field is not required, or there is a required field validator set for it, and we don’t want to interfere with its validation check.

I then mapped the above validator in Sitecore:

restricted-file-types-validator

After saving the above validator Item in Sitecore, I built the following validator class to check to see if a file exceeds a certain size:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web.UI.WebControls;

using Sitecore.Form.Core.Validators;
using Sitecore.Form.Web.UI.Controls;

namespace Sitecore.Sandbox.Form.Core.Validators
{
    public class RestrictedFileSize : FormCustomValidator
    {
        public int MaxFileSize
        {
            get
            {
                int maxSize;
                if (int.TryParse(base.classAttributes["mimeTypesNotAllowed"], out maxSize))
                {
                    return maxSize;
                }

                return 0;
            }
            set
            {
                base.classAttributes["mimeTypesNotAllowed"] = value.ToString();
            }
        }

        public RestrictedFileSize()
        {
        }

        protected override bool OnServerValidate(string value)
        {
            FileUpload fileUpload = FindControl(ControlToValidate) as FileUpload;
            if (!fileUpload.HasFile)
            {
                return true;
            }

            return fileUpload.PostedFile.ContentLength <= MaxFileSize;
        }
    }
}

Just as we had done in the other validator, we grab the FileUpload from the form, and see if there is a file attached. If there is no file attached, we return “true”: we don’t want to say the field is invalid when the field is not required.

We then return whether the uploaded file is less than or equal to the specified maximum size in bytes — this is set on a parameter named MaxFileSize which gets injected into the MaxFileSize property of the validator instance.

I then registered the above validator in Sitecore:

restricted-file-size-validator

I then decided to create a custom WFFM field type for the purposes of mapping our validators above, so that we don’t enforce these restrictions on the “out of the box” WFFM “File Upload” field type:

restricted-file-upload-field-type

I then set the new field type on a file field I added to a test WFFM form:

set-custom-field-type

Let’s see how we did!

Let’s try to upload an executable that exceeds the maximum upload size limit:

big-exe-wffm-upload

As you can see both validators were triggered for this file:

big-exe-wffm-upload-errormessage

How about a JavaScript file? Let’s try to attach one:

js-wffm-upload

Nope, we can’t attach that file either:

js-wffm-upload-errormessage

How about an image that is larger than the size limit? Let’s try one:

big-image-wffm-upload

Nope, we can’t upload that either:

big-image-wffm-upload-errormessage

Let’s try an image that is under 100KB:

allowed-image-wffm-upload

Yes, we can attach that file since it’s not restricted, and the form did submit:

allowed-image-wffm-upload-no-errormessage

If you have any thoughts on this, or other ideas around preventing certain files from being attached to WFFM form submissions, please share in a comment.

Perform a Virus Scan on Files Uploaded into Sitecore

Last week I took notice of a SDN forum post asking whether there are any best practices for checking files uploaded into Sitecore for viruses.

Although I am unaware of there being any best practices for this, adding a processor into the <uiUpload> pipeline to perform virus scans has been on my plate for the past month. Not being able to find an open source .NET antivirus library has been my major blocker for building one.

However, after many hours of searching the web — yes, I do admit some of this time is sprinkled with idle surfing — over multiple weeks, I finally discovered nClam — a .NET client library for scanning files using a Clam Antivirus server (I setup one using instructions enumerated here).

Before I continue, I would like to caution you on using this or any antivirus solution before doing a substantial amount of research — I do not know how robust the solution I am sharing actually is, and I am by no means an antivirus expert. The purpose of this post is to show how one might go about adding antivirus capabilities into Sitecore, and I am not advocating or recommending any particular antivirus software package. Please consult with an antivirus expert before using any antivirus software/.NET client library .

The solution I came up with uses the adapter design pattern — it basically wraps and makes calls to the nClam library, and is accessible through the following antivirus scanner interface:

using System.IO;

namespace Sitecore.Sandbox.Security.Antivirus
{
    public interface IScanner
    {
        ScanResult Scan(Stream sourceStream);

        ScanResult Scan(string filePath);
    }
}

This interface defines the bare minimum methods our antivirus classes should have. Instances of these classes should offer the ability to scan a file through the file’s Stream, or scan a file located on the server via the supplied file path.

The two scan methods defined by the interface must return an instance of the following data transfer object:

namespace Sitecore.Sandbox.Security.Antivirus
{
    public class ScanResult
    {
        public string Message { get; set; }

        public ScanResultType ResultType { get; set; }
    }
}

Instances of the above class contain a detailed message coupled with a ScanResultType enumeration value which conveys whether the scanned file is clean, contains a virus, or something else went wrong during the scanning process:

namespace Sitecore.Sandbox.Security.Antivirus
{
    public enum ScanResultType
    {
        Clean,
        VirusDetected,
        Error,
        Unknown
    }
}

I used the ClamScanResults enumeration as a model for the above.

I created and used the ScanResultType enumeration instead of the ClamScanResults enumeration so that this solution can be extended for other antivirus libraries — or calls could be made to other antivirus software through the command-line — and these shouldn’t be tightly coupled to the nClam library.

I then wrapped the nClam library calls in the following ClamScanner class:

using System;
using System.IO;
using System.Linq;

using Sitecore.Diagnostics;

using nClam;

namespace Sitecore.Sandbox.Security.Antivirus
{
    public class ClamScanner : IScanner
    {
        private ClamClient Client { get; set; }

        public ClamScanner(string server, string port)
            : this(server, int.Parse(port))
        {
        }

        public ClamScanner(string server, int port)
            : this(new ClamClient(server, port))
        {
        }

        public ClamScanner(ClamClient client)
        {
            SetClient(client);
        }

        private void SetClient(ClamClient client)
        {
            Assert.ArgumentNotNull(client, "client");
            Client = client;
        }

        public ScanResult Scan(Stream sourceStream)
        {
            ScanResult result;
            try
            {
                result = CreateNewScanResult(Client.SendAndScanFile(sourceStream));
            }
            catch (Exception ex)
            {
                result = CreateNewExceptionScanResult(ex);
            }

            return result;
        }

        public ScanResult Scan(string filePath)
        {
            ScanResult result;
            try
            {
                result = CreateNewScanResult(Client.SendAndScanFile(filePath));
            }
            catch (Exception ex)
            {
                result = CreateNewExceptionScanResult(ex);
            }

            return result;
        }

        private static ScanResult CreateNewScanResult(ClamScanResult result)
        {
            Assert.ArgumentNotNull(result, "result");
            if (result.Result == ClamScanResults.Clean)
            {
                return CreateNewScanResult("Yay! No Virus found!", ScanResultType.Clean);
            }

            if (result.Result == ClamScanResults.VirusDetected)
            {
                string message = string.Format("Oh no! The {0} virus was found!", result.InfectedFiles.First().VirusName);
                return CreateNewScanResult(message, ScanResultType.VirusDetected);
            }

            if (result.Result == ClamScanResults.Error)
            {
                string message = string.Format("Something went terribly wrong somewhere. Details: {0}", result.RawResult);
                return CreateNewScanResult(message, ScanResultType.Error);
            }

            return CreateNewScanResult("I have no clue about what just happened.", ScanResultType.Unknown);
        }

        private static ScanResult CreateNewExceptionScanResult(Exception ex)
        {
            string message = string.Format("Something went terribly wrong somewhere. Details:\n{0}", ex.ToString());
            return CreateNewScanResult(ex.ToString(), ScanResultType.Error);
        }

        private static ScanResult CreateNewScanResult(string message, ScanResultType type)
        {
            return new ScanResult
            {
                Message = message,
                ResultType = type
            };
        }
    }
}

Methods in the above class make calls to the nClam library, and return a ScanResult intance containing detailed information about the file scan.

Next I developed the following <uiUpload> pipeline processor to use instances of classes that define the IScanner interface above, and these instances are set via Sitecore when instantiating this pipeline processor — the ClamScanner type is defined in the patch configuration file shown later in this post:

using System.IO;
using System.Web;

using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Pipelines.Upload;

using Sitecore.Sandbox.Security.Antivirus;

namespace Sitecore.Sandbox.Pipelines.Upload
{
    public class ScanForViruses
    {
        public void Process(UploadArgs args)
        {
            Assert.ArgumentNotNull(args, "args");
            foreach (string fileKey in args.Files)
            {
                HttpPostedFile file = args.Files[fileKey];
                ScanResult result = ScanFile(file.InputStream);
                if(result.ResultType != ScanResultType.Clean)
                {
                    args.ErrorText = Translate.Text(string.Format("The file \"{0}\" cannot be uploaded. Reason: {1}", file.FileName, result.Message));
                    Log.Warn(args.ErrorText, this);
                    args.AbortPipeline();
                }
            }
        }

        private ScanResult ScanFile(Stream fileStream)
        {
            Assert.ArgumentNotNull(fileStream, "fileStream");
            return Scanner.Scan(fileStream);
        }

        private IScanner Scanner { get; set; }
    }
}

Uploaded files are passed to the IScanner instance, and the pipeline is aborted if something isn’t quite right — this occurs when a virus is detected, or an error is reported by the IScanner instance. If a virus is discovered, or an error occurs, the message contained within the ScanResult instance is captured in the Sitecore log.

I then glued everything together using the following patch configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <processors>
      <uiUpload>
        <processor mode="on" patch:before="processor[@type='Sitecore.Pipelines.Upload.CheckSize, Sitecore.Kernel']" 
                   type="Sitecore.Sandbox.Pipelines.Upload.ScanForViruses, Sitecore.Sandbox">
          <Scanner type="Sitecore.Sandbox.Security.Antivirus.ClamScanner">
            <param desc="server">localhost</param>
            <param desc="port">3310</param>
          </Scanner>
        </processor>
      </uiUpload>
    </processors>
  </sitecore>
</configuration>

I wish I could show you this in action when a virus is discovered in an uploaded file. However, I cannot put my system at risk by testing with an infected file.

But, I can show you what happens when an error is detected. I will do this by shutting down the Clam Antivirus server on my machine:

scanner-turned-off

On upload, I see the following:

tried-to-upload-file

When I opened up the Sitecore log, I see what the problem is:

log-error

Let’s turn it back on:

scanner-turned-on

I can now upload files again:

upload-no-error

If you have any suggestions on making this better, or know of a way to test it with a virus, please share in a comment below.

Restrict Certain Types of Files From Being Uploaded in Sitecore

Tonight I was struck by a thought while conducting research for a future blog post: should we prevent users from uploading certain types of files in Sitecore for security purposes?

You might thinking “Prevent users from uploading files? Mike, what on Earth are you talking about?”

What I’m suggesting is we might want to consider restricting certain types of files — specifically executables (these have an extension of .exe) and DOS command files (these have an extension of .com) — from being uploaded into Sitecore, especially when files can be easily downloaded from the media library.

Why should we do this?

Doing this will curtail the probability of viruses being spread among our Sitecore users — such could happen if one user uploads an executable that harbors a virus, and other users of our Sitecore system download that tainted executable, and run it on their machines.

As a “proof of concept”, I built the following uiUpload pipeline processor — the uiUpload pipeline lives in /configuration/sitecore/processors/uiUpload in your Sitecore instance’s Web.config — to restrict certain types of files from being uploaded into Sitecore:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web;
using System.Xml;

using Sitecore.Diagnostics;
using Sitecore.Globalization;
using Sitecore.Pipelines.Upload;

namespace Sitecore.Sandbox.Pipelines.Upload
{
    public class CheckForRestrictedFiles : UploadProcessor
    {
        private List<string> _RestrictedExtensions;
        private List<string> RestrictedExtensions
        {
            get
            {
                if (_RestrictedExtensions == null)
                {
                    _RestrictedExtensions = new List<string>();
                }

                return _RestrictedExtensions;
            }
        }

        public void Process(UploadArgs args)
        {
            foreach(string fileKey in args.Files)
            {
                string fileName = GetFileName(args.Files, fileKey);
                string extension = Path.GetExtension(fileName);
                if (IsRestrictedExtension(extension))
                {
                    args.ErrorText = Translate.Text(string.Format("The file \"{0}\" cannot be uploaded. Files with an extension of {1} are not allowed.", fileName, extension));
                    Log.Warn(args.ErrorText, this);
                    args.AbortPipeline();
                }
            }
        }

        private static string GetFileName(HttpFileCollection files, string fileKey)
        {
            Assert.ArgumentNotNull(files, "files");
            Assert.ArgumentNotNullOrEmpty(fileKey, "fileKey");
            return files[fileKey].FileName;
        }

        private bool IsRestrictedExtension(string extension)
        {
            return RestrictedExtensions.Exists(restrictedExtension => string.Equals(restrictedExtension, extension, StringComparison.CurrentCultureIgnoreCase));
        }

        protected virtual void AddRestrictedExtension(XmlNode configNode)
        {
            if (configNode == null || string.IsNullOrWhiteSpace(configNode.InnerText))
            {
                return;
            }

            RestrictedExtensions.Add(configNode.InnerText);
        }
    }
}

The class above ascertains whether each uploaded file has an extension that is restricted — restricted extensions are defined in the configuration file below, and are added to a list of strings via the AddRestrictedExtensions method — and logs an error message when a file possessing a restricted extension is encountered.

I then tied everything together using the following patch configuration file including specifying some file extensions to restrict:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <processors>
      <uiUpload>
        <processor mode="on" type="Sitecore.Sandbox.Pipelines.Upload.CheckForRestrictedFiles, Sitecore.Sandbox" patch:before="processor[@type='Sitecore.Pipelines.Upload.CheckSize, Sitecore.Kernel']">
          <restrictedExtensions hint="raw:AddRestrictedExtension">
            <!-- Be sure to prefix with a dot -->
            <extension>.exe</extension>
            <extension>.com</extension>
          </restrictedExtensions>
        </processor>
      </uiUpload>
    </processors>
  </sitecore>
</configuration>

Let’s try this out.

I went to the media library, and attempted to upload an executable:

upload-exe-dialog_001

After clicking the “Open” button, I was presented with the following:

upload-exe-error

An error in my Sitecore instance’s latest log file conveys why I could not upload the chosen file:

upload-exe-error-log

If you have thoughts on this, or have ideas for other processors that should be added to uiUpload pipeline, please share in a comment.

Delete Associated Files on the Filesystem of Sitecore Items Deleted From the Recycle Bin

Last week a question was asked in one of the SDN forums on how one should go about deleting files on the filesystem that are associated with Items that are permanently deleted from the Recycle Bin — I wasn’t quite clear on what the original poster meant by files being linked to Items inside of Sitecore, but I assumed this relationship would be defined somewhere, or somehow.

After doing some research, I reckoned one could create a new command based on Sitecore.Shell.Framework.Commands.Archives.Delete in Sitecore.Kernel.dll to accomplish this:

Sitecore.Shell.Framework.Commands.Archives.Delete

However, I wasn’t completely satisfied with this approach, especially when it would require a substantial amount of copying and pasting of code — a practice that I vehemently abhor — and decided to seek out a different, if not better, way of doing this.

From my research, I discovered that one could just create his/her own Archive class — it would have to ultimately derive from Sitecore.Data.Archiving.Archive in Sitecore.Kernel — which would delete a file on the filesystem associated with a Sitecore Item:

using System;
using System.IO;
using System.Linq;
using System.Web;

using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Archiving;
using Sitecore.Data.DataProviders.Sql;
using Sitecore.Diagnostics;

namespace Sitecore.Sandbox.Data.Archiving
{
    public class FileSystemHookSqlArchive : SqlArchive
    {
        private static readonly string FolderPath = GetFolderPath();

        public FileSystemHookSqlArchive(string name, Database database)
            : base(name, database)
        {
        }

        public override void RemoveEntries(ArchiveQuery query)
        {
            DeleteFromFileSystem(query);
            base.RemoveEntries(query);
        }

        protected virtual void DeleteFromFileSystem(ArchiveQuery query)
        {
            if (query.ArchivalId == Guid.Empty)
            {
                return;
            }

            Guid itemId = GetItemId(query.ArchivalId);
            if (itemId == Guid.Empty)
            {
                return;
            }

            string filePath = GetFilePath(itemId.ToString());
            if (string.IsNullOrWhiteSpace(filePath))
            {
                return;
            }

            TryDeleteFile(filePath);
        }

        private void TryDeleteFile(string filePath)
        {
            try
            {
                if (File.Exists(filePath))
                {
                    File.Delete(filePath);
                }
            }
            catch (Exception ex)
            {
                Log.Error(this.ToString(), ex, this);
            }
        }

        public virtual Guid GetItemId(Guid archivalId)
        {
            if (archivalId == Guid.Empty)
            {
                return Guid.Empty;
            }
            
            ArchiveQuery query = new ArchiveQuery
            {
                ArchivalId = archivalId
            };

            SqlStatement selectStatement = GetSelectStatement(query, "{0}ItemId{1}");
            if (selectStatement == null)
            {
                return Guid.Empty;
            }
            return GetGuid(selectStatement.Sql, selectStatement.GetParameters(), Guid.Empty);
        }

        private Guid GetGuid(string sql, object[] parameters, Guid defaultValue)
        {
            using (DataProviderReader reader = Api.CreateReader(sql, parameters))
            {
                if (!reader.Read())
                {
                    return defaultValue;
                }
                return Api.GetGuid(0, reader);
            }
        }

        private static string GetFilePath(string fileName)
        {
            string filePath = Directory.GetFiles(FolderPath, string.Concat(fileName, "*.*")).FirstOrDefault();
            if (!string.IsNullOrWhiteSpace(filePath))
            {
                return filePath;    
            }

            return string.Empty;
        }

        private static string GetFolderPath()
        {
            return HttpContext.Current.Server.MapPath(Settings.GetSetting("FileSystemHookSqlArchive.Folder"));
        }
    }
}

In the subclass of Sitecore.Data.Archiving.SqlArchive above — I’m using Sitecore.Data.Archiving.SqlArchive since I’m using SqlServer for my Sitecore instance — I try to find a file that is named after its associated Item’s ID — minus the curly braces — in a folder that I’ve mapped in a configuration include file (see below).

I first have to get the Item’s ID from the database using the supplied ArchivalId — this is all the calling code gives us, so we have to make do with what we have.

If the file exists, we try to delete it — we do this before letting the base class delete the Item from Recycle Bin so that we can retrieve the Item’s ID from the database before it’s removed from the Archive database table — and log any errors we encounter upon exception.

I then hooked in an instance of the above Archive class in a custom Sitecore.Data.Archiving.ArchiveProvider class:

using System.Xml;

using Sitecore.Data;
using Sitecore.Data.Archiving;
using Sitecore.Xml;

namespace Sitecore.Sandbox.Data.Archiving
{
    public class FileSystemHookSqlArchiveProvider : SqlArchiveProvider
    {
        protected override Archive GetArchive(XmlNode configNode, Database database)
        {
            string attribute = XmlUtil.GetAttribute("name", configNode);
            if (string.IsNullOrEmpty(attribute))
            {
                return null;
            }

            return new FileSystemHookSqlArchive(attribute, database);
        }
    }
}

The above class — which derives from Sitecore.Data.Archiving.SqlArchiveProvider since I’m using SqlServer — only overrides its base class’s GetArchive factory method. We instantiate an instance of our Archive class instead of the “out of the box” Sitecore.Data.Archiving.SqlArchive class within it.

I then had to replace the “out of the box” Sitecore.Data.Archiving.ArchiveProvider reference, and define the location of our files in the following configuration file:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <archives defaultProvider="sql" enabled="true">
      <providers>
        <add name="sql" patch:instead="add[@type='Sitecore.Data.Archiving.SqlArchiveProvider, Sitecore.Kernel']" type="Sitecore.Sandbox.Data.Archiving.FileSystemHookSqlArchiveProvider, Sitecore.Sandbox" database="*"/>
      </providers>
    </archives>
    <settings>
      <setting name="FileSystemHookSqlArchive.Folder" value="/test/" />
    </settings>
  </sitecore>
</configuration>

Let’s test this out.

I first created a test Item to delete:

test-item-to-delete

I then had to create a test file on the filesystem in my test folder — the test folder lives in my Sitecore instance’s website root:

test-folder-with-test-file

I deleted the test Item from the content tree, opened up the Recycle Bin, selected the test Item, and got an itchy trigger finger — I want to delete the Item forever 🙂 :

delete-file-forever

After clicking the Delete button, I saw that the file on the filesystem was deleted as well:

file-was-deleted

If you have any thoughts on this, or recommendations around making it better, please leave a comment.