Sitecore Event Queue – How to clean it – and why

Yesterday the dreaded Sitecore Event Queue almost killed me again – well it certainly almost killed my CM server. The server went from being busy but stable to being unresponsive. CPU and memory load skyrocketed:

A fully loaded CM server

A fully loaded CM server

Obviously it happened after a system update, so after a panic debug and rollback the system owner pointed out: “Hey, the event queue table is quite big?“.
Of course, the system updated flooded the event with 1.5 million events, and the problem did not go away because I keep 1 day of events in the queue.

SO WHAT TO DO ABOUT IT?

First we need to stabilize the system, then we need to update the configuration.

STEP 1: CLEAN OUT EVENT QUEUE, HISTORY TABLE, PUBLISH QUEUE

The following SQL statement will clean out the history table, publish queue and event queue, leaving only 12 hours of history and publish data and 4 hours of events. Replace YOURDATABASE with the name of your database:

/****** History ******/
delete FROM [YOURDATABASE_Core].[dbo].[History] where Created < DATEADD(HOUR, -12, GETDATE())
delete FROM [YOURDATABASE_Master].[dbo].[History] where Created < DATEADD(HOUR, -12, GETDATE())
delete FROM [YOURDATABASE_Web].[dbo].[History] where Created < DATEADD(HOUR, -12, GETDATE())
 
/****** Publishqueue ******/
delete FROM [YOURDATABASE_Core].[dbo].[PublishQueue] where Date < DATEADD(HOUR, -12, GETDATE());    
delete FROM [YOURDATABASE_Master].[dbo].[PublishQueue] where Date < DATEADD(HOUR, -12, GETDATE());
delete FROM [YOURDATABASE_Web].[dbo].[PublishQueue] where Date < DATEADD(HOUR, -12, GETDATE());
    
/****** EventQueue ******/
delete FROM [YOURDATABASE_Master].[dbo].[EventQueue] where [Created] < DATEADD(HOUR, -4, GETDATE())
delete FROM [YOURDATABASE_Core].[dbo].[EventQueue] where [Created] < DATEADD(HOUR, -4, GETDATE())
delete FROM [YOURDATABASE_Web].[dbo].[EventQueue] where [Created] < DATEADD(HOUR, -4, GETDATE())

STEP 2: CONFIGURE THE SYSTEM TO CLEAN THE TABLES MORE OFTEN

With the system stabilized, we need to take more care of the table sizes.

HISTORY TABLE:

Sitecore is already configured to clean the tables so they only contain 12 hours of data. 12 hours of data is usually what any SQL server will handle, and you will have up to 10.000 rows in the table.

<Engines.HistoryEngine.Storage>
  <obj type="Sitecore.Data.$(database).$(database)HistoryStorage, Sitecore.Kernel">
    <param connectionStringName="$(id)" />
    <EntryLifeTime>00.12:00:00</EntryLifeTime>
  </obj>
</Engines.HistoryEngine.Storage>

PUBLISH QUEUE:

Sitecore keeps 30 days of publish queue. If you insert and update items often, you should lower this number. For each item change (including any change that the system does) is stored here.

<agent type="Sitecore.Tasks.CleanupPublishQueue, Sitecore.Kernel" method="Run" interval="04:00:00">
  <DaysToKeep>2</DaysToKeep>
</agent>

EVENT QUEUE:

The event queue is the most important table to keep small. In a distributed environment, each server will read the contents of the table every 5 seconds, using a time stamp stored in the Properties table as key. Any row before the time stamp will not be read.

You therefore need enough history to cater that a server will be offline for a while, but at the same time so little contents that any read and write will be amazingly fast.

If you can keep the number of rows below 7.000, most SQL server should be able to handle that amount of data. Even smaller numbers are preferred as well.

Before Sitecore 8.1, Sitecore would only allow you to clean events older that 1 day. This is way too much, especially if you publish often. The new IntervalToKeep will allow you to determine the hours to keep as well:

<agent type="Sitecore.Tasks.CleanupEventQueue, Sitecore.Kernel" method="Run" interval="04:00:00">
  <IntervalToKeep>04:00:00</IntervalToKeep>
  <DaysToKeep>1</DaysToKeep>
</agent>

 

THE EFFECT ON THE CLEANUP

After these changes, my server is back to normal, busy but responsive:

Normal Load

Normal Load

MORE TO READ:

 

Posted in Sitecore 8 | Tagged , , , , | 2 Comments

Sitecore Measure Pipeline performance and profile processors

In Sitecore, Pipelines are by far my favorite dependency injection pattern, and I have used them since Sitecore 5. One of the secret features of pipelines are the built in profiling. Oh, yes, by flipping 2 switches you can measure the performance of your pipeline processors.

STEP 1: SWITCH ON PIPELINE PROFILING

Enable the \App_Config\Include\Sitecore.PipelineProfiling.config.disabled by removing the .disabled extension.

Or add the following 2 settings to your /App_Config/sitecore.config:

<setting name="Pipelines.Profiling.Enabled" value="true" />
<setting name="Pipelines.Profiling.MeasureCpuTime" value="true" />

The Pipelines.Profiling.MeasureCpuTime is not enabled by default in the Sitecore.PipelineProfiling.config file as it introduces a performance overhead, so only use this setting on your test environment.

STEP 2: FIND THE PROFILE PAGE:

Sitecore comes with a profiling page (it has been there since Sitecore 7 i guess):

http://yourwebsite/sitecore/admin/pipelines.aspx

Pipeline Profiler

Pipeline Profiler

The page displays all pipelines executed since you pressed the refresh button, how many times it has been executed, and the average wall time (wall time = real-world time from processor started to it finished, as opposed to CPU time which is the time spend by the CPU executing the processor) per execution.

MORE TO READ:

 

Posted in Sitecore 7, Sitecore 8 | Tagged , , , | Leave a comment

Sitecore Advanced Configuration in .config files

Sitecore allows you to create configuration sections in /configuration/sitecore/ that consists of hierarchies of configurations including lists, and it will even help you serialize the configuration to object instances.

Observe the following real life example of a configuration I created to configure lists of IP adresses and IP ranges to ignore in my web application:

<pt_restrictions>
  <restriction type="Restrictions.Model.Configuration, Restrictions">
    <ipraddresses hint="raw:AddIpAddresses" >
<address ip="127.0.0.1" />
<address ip="127.0.0.2" />
    </ipraddresses>
    <ipranges hint="raw:AddIpRanges">
      <range from="192.168.0.1" to="192.168.0.255"/>
      <range from="192.169.0.1" to="192.169.0.255"/>
    </ipranges>
  </restriction>
</pt_restrictions>

The configuration section is added to a Sitecore .config file under /configuration/sitecore/.

Please notice the “type” property “Restrictions.Model.Configuration, Restrictions” and the 2 “hint” properties. These allows Sitecore to perform some reflection magic that will map the configuration section to classes.

So let’s code. First I will create an interface that outlines my configuration, a list of Ip Addresses and a list of Ip Ranges

namespace Restrictions.Model
{
  public interface IConfiguration
  {
    IEnumerable<IpAddress> IpAddresses { get; }
    IEnumerable<IpRange> IpRanges { get; }
  }
}

I need implementations of IpAddress and IpRange. The implementations contains the XML attributes as properties:

namespace Restrictions.Model
{
  public class IpAddress
  {
    public string Ip { get; set; }
  }

  public class IpRange
  {
    public string From { get; set; }
    public string To { get; set; }
  }
}

For the final touch,  I will assemble it all in a Configuration class. And here is where the magic lies. Remember the “hint=”raw:AddIpAddresses” and “hint=”raw.AddIpRanges”” attributes? These are method names in my class. Sitecore calls these for each Xml node:

using System.Collections.Generic;
using System.Xml;

namespace Restrictions.Model
{
  public class Configuration : IConfiguration
  {
    private readonly List<IpAddress> _ipAddresses = new List<IpAddress>();
    public IEnumerable<IpAddress> IpAddresses { get { return _ipAddresses; } }

    private readonly List<IpRange> _ipRanges = new List<IpRange>();
    public IEnumerable<IpRange> IpRanges  { get { return _ipRanges; } }

    protected void AddIpAddresses(XmlNode node)
    {
      if (node == null)
        return;
      if (node.Attributes == null)
        return;
      if (node.Attributes["ip"] == null)
        return;
      if (node.Attributes["ip"].Value == null)
        return;

      _ipAddresses.Add(new IpAddress() { Ip = node.Attributes["ip"].Value });
    }

    protected void AddIpRanges(XmlNode node)
    {
      if (node == null)
        return;
      if (node.Attributes == null)
        return;
      if (node.Attributes["from"] == null)
        return;
      if (node.Attributes["from"].Value == null)
        return;
      if (node.Attributes["to"] == null)
        return;
      if (node.Attributes["to"].Value == null)
        return;

      _ipRanges.Add(new IpRange() { From = node.Attributes["from"].Value, To = node.Attributes["to"].Value });
    }
  }
}

The final class is the repository that will use reflection magic to convert an XmlNode to a class implementation:

using System.Xml;

namespace Restrictions.Model.Repositories
{
  public class ConfigurationRepository
  {
    public IConfiguration GetConfiguration(string restrictionName)
    {
      XmlNode xmlNode = Sitecore.Configuration.Factory.GetConfigNode("pt_restrictions/restriction");
      return Sitecore.Configuration.Factory.CreateObject<IConfiguration>(xmlNode);
    }
  }
}

I use the class like this:

ConfigurationRepository rep = new ConfigurationRepository();
IConfiguration config = rep.GetConfiguration();

foreach (var ipaddress in config.IpAddresses)
  // do stuff;
  
foreach (var iprange in config.IpRanges)
  // do stuff

It looks like a big job (and it probably is) but once you get it running, it is easy to extend and use elsewhere.

MORE TO READ:

 

Posted in .net, c#, General .NET, Sitecore, Sitecore 5, Sitecore 6, Sitecore 7, Sitecore 8 | Tagged , , , , | Leave a comment

Sitecore PostSessionEndPipeline failed – MongoDB.Driver.MongoQueryException

In my (huge) Sitecore solution we experienced the following error:

ERROR PostSessionEndPipeline failed.
Exception: MongoDB.Driver.MongoQueryException
Message: QueryFailure flag was Runner error: Overflow sort stage buffered data usage of 33570904 bytes exceeds internal limit of 33554432 bytes (response was { “$err” : “Runner error: Overflow sort stage buffered data usage of 33570904 bytes exceeds internal limit of 33554432 bytes”, “code” : 17144 }).
Source: MongoDB.Driver
at Sitecore.Analytics.Pipelines.CommitSession.SubmitSession.Process(CommitSessionPipelineArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Analytics.Pipelines.PostSessionEnd.CommitSession.Process(PostSessionEndArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Web.Application.RaiseSessionEndEvent(HttpApplication context)

Followed by the following message:

QueryFailure flag was Runner error: Overflow sort stage buffered data usage of xxxxxxxx bytes exceeds internal limit of 33554432 bytes (response was { “$err” : “Runner error: Overflow sort stage buffered data usage of xxxxxxxx bytes exceeds internal limit of 33554432 bytes”, “code” : 17144 }).

The error is caused by an internal 32MB memory size restriction in MongoDB.

The problem can be solved by adding an index to the Interactions collection. This will allow Sitecore to stream interactions sorted rather than attempting to load them all into memory and do the sorting in memory.

Connect to the analytics index using RoboMongo or MongoVUE and run the following query:

db.Interactions.createIndex(
{
    "StartDateTime" : 1,
    "_id" : 1
}
)

A WORD OF CAUTION:

According to MongoDB, adding indexes to a live environment does not have any side effects. But running the above query on my 7.5 million entries in the Interactions collection caused my entire Sitecore to fail (total catastrophic failure) for 5-10 minutes until the index was built.

Thank you to Sitecore Support and Joshua Wheelock for helping with the issue.

MORE TO READ:

Posted in Sitecore 8 | Tagged , , | Leave a comment

Sitecore get host name from a different context

Ohno, you are running in one context, say the “shell” context, and all of your URL’s have to point to the “website” context. The hustler will obviously hard-code the domain name of the “website” into the code. But the Sitecore aficionado will check the SiteContext and retrieve the “TargetHostName” property.

This tiny class will qualify your relative URL’s to absolute URL’s, matching the domain name as specified in the <sites> section of your configuration (code simplified for readability, you should apply your own exception handling):

using System;
using System.Web;
using Sitecore.Sites;

namespace MyNameSpace
{
  internal static class FullyQualifiedUrlService
  {
    public static string Qualify(string relativeUrl, string sitename)
    {
      SiteContext sitecontext = SiteContext.GetSite(sitename);
      return Qualify(relativeUrl, sitecontext);
    }

    public static string Qualify(string relativeUrl, SiteContext sitecontext)
    {
      if (!relativeUrl.StartsWith("/"))
        relativeUrl = relativeUrl + "/";
      return string.Format("{0}://{1}{2}", sitecontext.SiteInfo.Scheme, sitecontext.TargetHostName, relativeUrl);
    }
  }
}

USAGE:

Imagine this is your sites definition for the “website” context:

<sites>
  ...
  <site name="website" targetHostName="www.mysite.com" hostName="mysite.com|www.mysite.com" scheme="http" ... .../>
  ...
</sites>

The class is called using the following code:

string relativeUrl = "/this/is/my/page";
string absoluteUrl = FullyQualifiedUrlService.Qualify(relativeUrl, "website");

And will return the following result:

http://www.mysite.com/this/is/my/page

Both the scheme and the targetHostName is resolved from the context using the Scheme and TargetHostName properties of the SiteContext.

Sitecore uses 2 properties for host resolving:

  • The “hostName” can be a pipe-separated list of domains and is used to target the number of possible URL’s that points to this context.
  • The “targetHostName” is one URL which is used internally to resolve and fully qualify your URL.

MORE TO READ:

 

Posted in c#, General .NET, Sitecore 6, Sitecore 7, Sitecore 8 | Tagged , , , | Leave a comment

Sitecore EXM keeps reloading

Have you experienced that in Sitecore, the EXM Email Experience Manager window keeps reloading?

Email Experience Manager

Email Experience Manager

In my solution, the problem was that I have a cookie with a “:” (colon) in the cookie name. According to the old RFC 2616 specification, colons are not allowed in cookie names.

Although this is not a problem in the website or in the Sitecore Shell, it causes problems when this particular SPEAK app calls the server API.

The solution is to avoid using colons in cookie names.

 

Posted in Sitecore 8 | Tagged , , , | Leave a comment

Sitecore SVG files

In Sitecore 8.1, there is a tiny but annoying glitch in the matrix, where SVG files are not allowed in the Media Library.

But do not worry, the fix is easy. Go to your /App_Config/Sitecore.config, find the /sitecore/mediaLibrary/mediaTypes/mediaType section, and add the SVG section yourself:

<mediaType name="SVG" extensions="svg">
  <mimeType>image/svg+xml</mimeType>
  <forceDownload>false</forceDownload>
  <sharedTemplate>system/media/unversioned/image</sharedTemplate>
  <versionedTemplate>system/media/versioned/image</versionedTemplate>
  <mediaValidator type="Sitecore.Resources.Media.ImageValidator"/>
  <thumbnails>
    <generator type="Sitecore.Resources.Media.ImageThumbnailGenerator, Sitecore.Kernel">
      <extension>png</extension>
    </generator>
    <width>150</width>
    <height>150</height>
    <backgroundColor>#FFFFFF</backgroundColor>
  </thumbnails>
</mediaType>

MORE TO READ:

Posted in Sitecore, Sitecore 8 | Tagged , | 1 Comment

Sitecore EXM: Send an email from code

The Sitecore Email Experience Manager is your way to send personalized emails to Sitecore users.   You do not need to send bulk emails, you can easily send single emails with contents like “Here is your new password” or “Your profile has been updated”.

The emails to send are “Standard Messages” so the email you create must be of the type “Triggered message”:

Triggered Message Settings

Triggered Message Settings

There is 2 ways of sending emails: To Sitecore users or Sitecore Contacts.

SEND AN EMAIL TO A SITECORE USER

Your Sitecore user must exist in the Sitecore User repository (usually as an “extranet” user).

using Sitecore.Data;
using Sitecore.Diagnostics;
using Sitecore.Modules.EmailCampaign;
using Sitecore.Modules.EmailCampaign.Messages;
using Sitecore.Modules.EmailCampaign.Recipients;

public void Send(ID messageItemId, string userName)
{
  MessageItem message = Factory.GetMessage(messageItemId);
  Assert.IsNotNull(message, "Could not find message with ID " + messageItemId);
  RecipientId recipient = new SitecoreUserName(userName);
  Assert.IsNotNull(recipient, "Could not find recipient with username " + userName);
  new AsyncSendingManager(message).SendStandardMessage(recipient);
}

You call the function like this:

Send(new ID("{12A6D766-CA92-4303-81D2-57C66F20AB12}"), "extranet\\user@domain.com");

SEND AND EMAIL TO A CONTACT

The contact must (obviously) contain an email address. To create a contact see Sitecore Contacts – Create and save contacts to and from xDB (MongoDB). The code is near identical to the previous, but the Receipient is retrieved by resolving the contact ID:

using Sitecore.Data;
using Sitecore.Diagnostics;
using Sitecore.Modules.EmailCampaign;
using Sitecore.Modules.EmailCampaign.Messages;
using Sitecore.Modules.EmailCampaign.Recipients;

public void Send(ID messageItemId, Guid contactID)
{
  MessageItem message = Factory.GetMessage(messageItemId);
  Assert.IsNotNull(message, "Could not find message with ID " + messageItemId);
  RecipientId recipient = RecipientRepository.GetDefaultInstance().ResolveRecipientId("xdb:" + contactID);
  Assert.IsNotNull(recipient, "Could not find recipient with ID " + contactID);
  new AsyncSendingManager(message).SendStandardMessage(recipient);
}

You call the function like this:

Send(new ID("{12A6D766-CA92-4303-81D2-57C66F20AB12}"), Guid.Parse("c3b8329b-7930-405d-8852-7a88ef4f0cb1"));

MORE TO READ:

 

Posted in c#, General .NET, Sitecore 8 | Tagged , , , , , | Leave a comment

Sitecore 8 EXM: Failed to enroll a contact in the engagement plan

In Sitecore 8.1 update 1, you might experience the following error when trying to send an email:

ERROR Failed to enroll a contact in the engagement plan.
Exception: System.Net.WebException
Message: The remote name could not be resolved: ‘default-cd-cluster’


Message: Recipient sc:extranet\someemail@pentia.dk skipped. Failed to enroll its corresponding contact in the engagement plan.
Source: Sitecore.EmailCampaign
at Sitecore.Modules.EmailCampaign.Core.Dispatch.DispatchTask.OnSendToNextRecipient()

The error occurs if you do not change the Analytics.ClusterName in the /App_Config/Include/Sitecore.Analytics.Tracking.config.

<setting name="Analytics.ClusterName" value="default-cd-cluster" />

The Analytics.ClusterName must be a valid, reachable URL.

<setting name="Analytics.ClusterName" value="hostname.of.my.site.com" />

This is because Sitecore calls the /sitecore/AutomationStates.ashx page using the Analytics.ClusterName as the host name.

Posted in Sitecore 8 | Tagged , , , , | 1 Comment

Sitecore xDB – flush data to MongoDB

When debugging Sitecore xDB issues, it is a pain that data is not written to MongoDB directly, but you have to wait for your session to end.

The legend said that you could either set the session timeout to 1 minute, or call Session.Abandon() to write to MongoDB. None of this have ever worked for me, until my colleague Lasse Pedersen found the final, bullet-proof way of having the xDB session data written to MongoDB.

Create a page with the following contents, and call it when you wish to flush data from xDB to MongoDB:

Sitecore.Analytics.Tracker.Current.EndTracking();
HttpContext.Current.Session.Abandon();

We will all pray to the debugging gods that this fix will work in every scenario.

Posted in c#, General .NET, Sitecore 8 | Tagged , , , | 3 Comments