SMTP Connection Test Script with ASP.NET

Slightly off topic today, one of the few posts not about Sitecore CMS.
A small challenge I had to deal with – troubleshoot a number of SMTP servers, trying to figure out which one can actually relay.
As basic as this sounds, I have not found a ready to go ASP.NET script on the web that can help me. So I decided to build my own.

The idea is simple – use standard SmtpClient class, wrap the Send() method in try/catch and output the exception.

SmtpClient client = new SmtpClient(server, int.Parse(port));
client.Credentials = new System.Net.NetworkCredential(username, password);
MailMessage mailMessage = new MailMessage(from, to);
mailMessage.Subject = "Test Email";
mailMessage.Body = "Hello, this is a test email from BayNET. Please ignore.";
try
{
   // trying to send...
   client.Send(mailMessage);
   Response.Write("Success!!!");
}
// catching SmtpException 
catch (SmtpException exception)
{
   Response.Write(String.Format("Cannot send mail. Status Code {0}. Details:{1}", exception.StatusCode, exception.Message));
}

The files can be downloaded from here.

Ideas were taken pretty much from the folks at stackoverflow.com.

Happy SMTP relaying!

Sitecore Logging Part 3. Adding custom parameters to the log.

Summer…I am on a blog posting spree :-)

This is a continuation of the sage about SQL logging for Sitecore CMS. As I previously blogged, you can easily have Sitecore log to a SQL database instead of a flat text file.

Now what if we take it one step forward and have Sitecore output more information than we had before, including Sitecore Context User, Sitecore Context Item Id and raw server URL?

Well, after some digging, here is the solution for you.

First of all, we need to add those columns to the actual table that we are going to use for storage.
Here is how your “Log” table could look like:

CREATE TABLE [dbo].[Log](
	[ID] [int] IDENTITY(1,1) NOT NULL,
	[Date] [datetime] NOT NULL,
	[Thread] [varchar](255) NOT NULL,
	[Level] [varchar](20) NOT NULL,
	[Logger] [varchar](255) NOT NULL,
	[Message] [varchar](4000) NOT NULL,
	[Exception] [varchar](2000) NULL,
	[SCUser] [varchar](255) NULL,
	[SCItemId] [varchar](38) NULL,
	[RawUrl] [varchar](255) NULL
) ON [PRIMARY]

Secondly, we need a custom version of the log4net.Appender.ADONetAppender class where we simply add the values of those 3 into the Properties collection.
I decided to do it only if my Context Site is resolved properly and the logging levels indicate either ERROR or FATAL levels:

using log4net.Appender;
using log4net.spi;
namespace Project.Shell.System
{
   public class SitecoreDatabaseLogAppender : ADONetAppender
   {
      protected override void Append(LoggingEvent loggingEvent)
      {
         if (Sitecore.Context.Site != null &&
            (loggingEvent.Level == Level.FATAL || loggingEvent.Level == Level.ERROR))
         {
            var properties = loggingEvent.Properties;
            if (Sitecore.Context.User != null)
            {
               properties["scuser"] = Sitecore.Context.User.Name;
            }
            if (Sitecore.Context.Item != null)
            {
               properties["scitemid"] = Sitecore.Context.Item.ID.ToString();
            }
            properties["rawurl"] = Sitecore.Web.WebUtil.GetServerUrl() + Sitecore.Web.WebUtil.GetRawUrl();
         }
         base.Append(loggingEvent);
      }
   }
}

Pretty simple, huh?

Afterwards, we just need to tweak our configuration a bit:

1. Specify the reference to the new custom “SitecoreDatabaseAppender” class:
<appender name="ADONetAppender_SqlServer" type="Project.Shell.System.SitecoreDatabaseLogAppender, Project.Shell" >

2. Extend CommandText value to include 3 new parameter we are adding:
<param name="CommandText" value="INSERT INTO Log ([Date],[Thread],[Level],[Logger],[Message],[Exception], [SCUser], [SCItemId], [RawUrl]) VALUES (@log_date, @thread, @log_level, @logger, @message, @exception, @scuser, @scitemid, @rawurl)" />

3. Define the way you want log4net parsing engine to process these parameters:
Same story here. Make sure that the value of the size parameter matches the database column size.
<param name="Parameter">
               <param name="ParameterName" value="@scuser" />
               <dbType value="String" />
               <size value="255" />
               <layout type="Project.Shell.System.PropertyLayout,Project.Shell">
                  <param name="PropertyName" value="scuser" />
               </layout>
</param>
<param name="Parameter">
               <param name="ParameterName" value="@scitemid" />
               <dbType value="String" />
               <size value="38" />
               <layout type="Project.Shell.System.PropertyLayout,Project.Shell">
                  <param name="PropertyName" value="scitemid" />
               </layout>
</param>
<param name="Parameter">
               <param name="ParameterName" value="@rawurl" />
               <dbType value="String" />
               <size value="255" />
               <layout type="Project.Shell.System.PropertyLayout,Project.Shell">
                  <param name="PropertyName" value="rawurl" />
               </layout>
</param>

You may also notice that I am using a custom type for the layout parameter. The reason for that is that I could not find a proper way of extracting the properties appended by SitecoreDatabaseLogAppender with default layouts. So I created a custom one which is able to extract those values with ease.

4. This means that you should also compile the following class:

using System;
using System.Text;
using log4net.Layout;
using log4net.spi;
namespace BayNET.Shell.System
{
   public class PropertyLayout : LayoutSkeleton
   {
      // Fields
      protected const int BUF_SIZE = 0x100;
      private StringBuilder m_sbuf = new StringBuilder(0x100);
      protected const int MAX_CAPACITY = 0x400;
      private string m_propertyName;
      public string PropertyName
      {
         get
         {
            return m_propertyName;
         }
         set
         {
            m_propertyName = value;
         }
      }
      public override bool IgnoresException
      {
         get
         {
            return false;
         }
      }
      public override void ActivateOptions()
      {
      }
      public override string Format(LoggingEvent loggingEvent)
      {
         if (loggingEvent == null)
         {
            throw new ArgumentNullException("loggingEvent");
         }
         if (m_sbuf.Capacity > 0x400)
         {
            m_sbuf = new StringBuilder(0x100);
         }
         else
         {
            m_sbuf.Length = 0;
         }
         if (!String.IsNullOrEmpty(PropertyName))
         {
            m_sbuf.Append(loggingEvent.Properties[PropertyName]);
         }
         return m_sbuf.ToString();
      }
   }
}

That should be it. Now you can happily observe all these 3 parameters being written in the log when en error happens. Isn’t that sweet?

Sitecore Error in Package Designer/Installation Wizard

I just got the following exception in the Package Designer on my fairly clean Sitecore 6.1 install.

Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

Exception Details: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[ReflectionTypeLoadException: Unable to load one or more of the requested types.
Retrieve the LoaderExceptions property for more information.]
   System.Reflection.Module._GetTypesInternal(StackCrawlMark& stackMark) +0
   System.Reflection.Assembly.GetTypes() +96
   Sitecore.Shell.Applications.Installer.Commands.Commands.Init() +48
   Sitecore.Shell.Applications.Installer.Commands.Commands..cctor() +6

[TypeInitializationException: The type initializer for 
   'Sitecore.Shell.Applications.Installer.Commands.Commands' threw an exception.]
   Sitecore.Shell.Applications.Installer.Commands.Commands.Init() +36
   Sitecore.Shell.Applications.Installer.Designer..ctor() +10

Based on the findings from our support portal, I found out that quite a few customers had the same issue and that is it also popping up in Installation Wizard.
So I decided to publish a quick post about it.

The stack trace indicates that the Installer app is trying to load some commands via reflection and obviously fails during this process.
By merging the bin directory of my Sitecore installation with the clean distributive, I found out that a few assemblies were somehow different, specifically Sitecore.Kernel.dll and Interop.Shell32.dll. So I copied them over from the clean distributive…and that resolved the issue!
It begs the question how it happened specifically, and my hunch is that somehow the DLLs got overwritten during the build process by Visual Studio, but let’s keep that as mystery.

Happy troubleshooting!

Sitecore Logging Part 2. Dealing with the Exceptions

As as follow up to the initial post about SQL based logging in Sitecore, here is another quick tip about how to include exception stack trace into your log database.

It is quite simple actually:

1. Make sure to add the Exception column to your table.

CREATE TABLE [dbo].[Log](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [Date] [datetime] NOT NULL,
    [Thread] [varchar](255) NOT NULL,
    [Level] [varchar](20) NOT NULL,
    [Logger] [varchar](255) NOT NULL,
    [Message] [varchar](4000) NOT NULL,
    [Exception] [varchar](2000) NULL
) ON [PRIMARY]

2. Extend the CommandText value to include the Exception parameter:

<param name="CommandText" 
value="INSERT INTO Log ([Date],[Thread],[Level],[Logger],[Message],[Exception])
       VALUES (@log_date, @thread, @log_level, @logger, @message, @exception)" />

3. Finally, add the following parameter to the log4net configuration. Note that special “Exception” layout is used.
The value of the sizeparameter should match the size of the SQL column on #1.

<param name="Parameter">
   <param name="ParameterName" value="@exception" />
   <dbType value="String" />
   <size value="2000" />
   <layout type="log4net.Layout.ExceptionLayout" />
</param>

Now you are all set, and should be able to see exceptions in the database:

Cheers!

Sitecore Logging. Quick update.

Remember in the last post about SQL based logging I mentioned that there is an internal buffer that log4net has before it dumps all into the database.
While it seems to be quite useful when running in production, in development environment you would want to see immediate messages in the logs, especially when troubleshooting.
In order to do that, simply add the “bufferSize” section for your ADONetAppender and set the value to “1”:

<appender name="ADONetAppender_SqlServer" type="log4net.Appender.ADONetAppender" >
<bufferSize value="1" />

If the buffer is not explicitly set, it will be defaulted to 512 :-)

Happy logging!

File Download Dialog Box for Sitecore Media Item

It is not a secret that Sitecore CMS can deliver media content in a dynamic fashion.
You can do pretty sick things with our MediaRequestHandler (as an example of that, check out what Alistair Deneys done with it in the past).
Today, however, I wanted to cover a fairly easy aspect of this functionality.
Specifically, the way you can force Media handler to force download of a requested resource, rather than having it being opened in the browser.
I cannot describe the technical challenge any better than Microsoft KB article explains it.
Basically, it all boils down to sending a specific HTTP header, for example:
Content-disposition: attachment; filename=fname.ext

Now good news is that with Sitecore you are able to control it via configuration and on MIME type basis. What you do is go to web.config and set “forceDownload” parameter to true on specific media type, for example, PDF:

<mediaType name="PDF file" extensions="pdf">
   <mimeType>application/pdf</mimeType>
   <forceDownload>true</forceDownload>
   <sharedTemplate>system/media/unversioned/pdf</sharedTemplate>
   <versionedTemplate>system/media/versioned/pdf</versionedTemplate>
</mediaType>

While this is nice and extremely convenient, what if you need more granularity? For example, have certain JPEGs being forced to download?
Sitecore’s answer – no problem!

Here is what you can do:

1. Create a custom MediaRequestHandler based on default one:

using System.Web;
using Sitecore.Resources.Media;
namespace Sitecore.StarterKit.Media
{
    public class ForceDownloadMediaHandler : MediaRequestHandler
    {
        public override void ProcessRequest(HttpContext context)
        {
            ProcessForceDownload(context);
            base.ProcessRequest(context);
        }
        private static void ProcessForceDownload(HttpContext context)
        {
            if (context.Request.QueryString["force"] == "1")
            {
                var request = MediaManager.ParseMediaRequest(context.Request);
                if (request != null)
                {
                    var media = MediaManager.GetMedia(request.MediaUri);
                    if (media != null)
                    {
                        context.Response.AddHeader("Content-Disposition", "attachment; filename=\"" + 
                                         media.MediaData.MediaItem.Name + "." + media.MediaData.Extension + "\"");
                    }
                }
            }
        }
    }
}

2. Compile, place the assembly in /bin folder as usual.

3. Replace the default MediaRequestHandler with your custom one in web.config:
- Depending on the configuration,  under <system.webServer>/<handlers>:
<add verb="*" path="sitecore_media.ashx" type="Sitecore.StarterKit.Media.ForceDownloadMediaHandler, Sitecore.Starterkit" name="Sitecore.MediaRequestHandler" />
- And/or under <system.web>/<httpHandlers>:
<add verb="*" path="sitecore_media.ashx" type="Sitecore.StarterKit.Media.ForceDownloadMediaHandler, Sitecore.Starterkit" />

4. When you render a link to a resource, append the “force=1” query string.

5. Use it:
http://www.site.com/~/media/docs/installguide.ashx?forcedownload=1

That’s all, folks!
Thanks goes to Alexey for the source code :-)

Hidden feature of Sitecore 6.2

Too many times I’ve seen this question, it is not even funny. My images are not showing up when I publish an item. While it can easily be explained from the technical perspective and addressed, it does not always make sense from user’s point of view.

Today I’ve learnt that Sitecore 6.2 actually introduced a small feature, an additional processor for the publishItem pipeline, intended to process related media items linked from the FileDropArea field, which was also another addition in 6.2.

You may have already guessed. While it is only processing the links from FDA field, that does not mean we cannot teach it to process links from other fields as well.

using System.Collections.Generic;
using System.Linq;
using Sitecore.Data.Items;
using Sitecore.Diagnostics;
using Sitecore.Publishing;
using Sitecore.Publishing.Pipelines.PublishItem;
namespace Project.Shell.Pipelines.PublishItem
{
   public class AddAllItemReferences : Sitecore.Publishing.Pipelines.PublishItem.AddItemReferences
   {
      protected override List<Item> GetItemReferences(PublishItemContext context)
      {
         Assert.ArgumentNotNull(context, "context");
         var list = new List<Item>();
         // calling base method which processes links from FileDropArea field
         list.AddRange(base.GetItemReferences(context));
         // adding our "own" related items
         list.AddRange(GetRelatedReferences(context));
         return list;
      }
      protected virtual List<Item> GetRelatedReferences(PublishItemContext context)
      {
         Assert.ArgumentNotNull(context, "context");
         var relatedReferenceList = new List<Item>();
         if (context.PublishOptions.Mode == PublishMode.SingleItem)
         {
            var sourceItem = context.PublishHelper.GetSourceItem(context.ItemId);
            if (sourceItem.Paths.IsContentItem)
            {
               var itemLinks = sourceItem.Links.GetValidLinks();
               // adding only valid related items
               relatedReferenceList.AddRange(
                   itemLinks.Select(link => link.GetTargetItem()).Where(relatedItem => relatedItem != null));
            }
         }
         return relatedReferenceList;
      }
   }
}

Now all you need is replace the default AddItemReferences processor with the custom one:

<!--<processor type="Sitecore.Publishing.Pipelines.PublishItem.AddItemReferences, Sitecore.Kernel"/>—>
<processor type="Project.Shell.Pipelines.PublishItem.AddAllItemReferences, Project.Shell"/>

Too many blogs and articles were already posted about how to do it, but I still believe this is the easiest and most elegant way of solving this challenge.
http://sitecorejohn.spaces.live.com/Blog/cns!960125F1D4A59952!829.entry?wa=wsignin1.0&sa=4023005
http://sitecoreblog.alexshyba.com/2007/10/publish-related-media-items.html
http://sdn.sitecore.net/FAQ/Media/Publish%20related%20media%20items.aspx

Enjoy!

Sitecore Logging. Write it to SQL!

Today I have been trying to make Sitecore CMS log to the database instead of file system. After hitting a few roadblocks, I’ve managed to find an answer via our helpful support portal.

Now it is not a secret that we are using an open source logging framework called log4net and that it is possible to configure it to use SQL database for storage instead of the file system.

This could come extremely handy due to a couple of reasons.

1. By default, file system logging means that Sitecore frequently creates a new file during system restart with a new timestamp, which may not be really what you expect, but this is by design.

2. Parsing of text files is an extremely creative and involving process. There are tools and solutions to facilitate in that but, I prefer writing SQL queries to get my data.

There are definitely more, but you catch my drift…

So how to get it working? As I mentioned, our support folks, or I should say, solution consultants, came up with 3 easy steps.

1. Create a table to store your logs. I’d recommend creating a separate database for that called “SitecoreSystem” as you don’t want to add a custom table to your Sitecore database, believe me!

CREATE TABLE [dbo].[Log] ( 
[ID] [int] IDENTITY (1, 1) NOT NULL ,
[Date] [datetime] NOT NULL ,
[Thread] [varchar] (255) NOT NULL ,
[Level] [varchar] (20) NOT NULL ,
[Logger] [varchar] (255) NOT NULL ,
[Message] [varchar] (4000) NOT NULL 
) ON [PRIMARY]

2. Define your own “SQL based appender” in <log4net> section of web.config.

<appender name="ADONetAppender_SqlServer" type="log4net.Appender.ADONetAppender, Sitecore.Logging" >
<param name="ConnectionType" value="System.Data.SqlClient.SqlConnection, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
<param name="ConnectionString" value="user id=sa;password=sitecorerocks;Data Source=.\\sql2008;Database=SitecoreSystem" />
<param name="CommandText" value="INSERT INTO Log ([Date],[Thread],[Level],[Logger],[Message]) VALUES (@log_date, @thread, @log_level, @logger, @message)" />
<param name="Parameter">
<param name="ParameterName" value="@log_date" />
<param name="DbType" value="DateTime" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%d{yyyy&apos;-&apos;MM&apos;-&apos;dd HH&apos;:&apos;mm&apos;:&apos;ss&apos;.&apos;fff}" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@thread" />
<param name="DbType" value="String" />
<param name="Size" value="255" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%t" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@log_level" />
<param name="DbType" value="String" />
<param name="Size" value="50" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%p" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@logger" />
<param name="DbType" value="String" />
<param name="Size" value="255" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%c" />
</param>
</param>
<param name="Parameter">
<param name="ParameterName" value="@message" />
<param name="DbType" value="String" />
<param name="Size" value="4000" />
<param name="Layout" type="log4net.Layout.PatternLayout">
<param name="ConversionPattern" value="%m" />
</param>
</param>
</appender>

3. Reference it now in <root> section, comment out the standard file based appender:

<root>
<priority value="INFO"/>
<appender-ref ref="ADONetAppender_SqlServer" />
<!--<appender-ref ref="LogFileAppender"/>-->
</root>

Troubleshooting

1. Make sure to double back slash the SQL name, e.g.  (.\\sqlexpress) in ConnectionString.

2. If you don’t see any entries in the Log database straight after the change, don’t panic. There is a threshold which should be exceeded before anything is written to the database. To test, launch tree serialization process – as this creates a lot of log entries, you will definitely see those unless something is wrong.

3. If something is wrong, add the following to your web.config. And if you don’t see anything written to the database, check the log file specified below.

<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="D:\logs\error.log" />
<remove name="Default" />
</listeners>
</trace>
</system.diagnostics>

 Open questions

- Maintainability
Now the most interesting part. You will probably need to create a scheduled agent that cleans up the Log table periodically for obvious reasons.

What next?

You can take it further and add custom parameters to your logs. Let’s leave that till next time.

Share if you think it is useful. Cheers!