Sync Sitecore content to external database

The legal department of the client requested that we sync parts of the Sitecore tree to an external, system-versioned temporal table, so that we have a full audit trail of changes to these items including before-and-after values.

System-Versioned Table

Example of a System-Versioned Table. When looking into the History you can see a full audit trail of the Sitecore items

Fear not, this is what Sitecore does best. All we need to do is to create a SQL server database with temporal tables, create upsert (a combined insert and update statement) and delete Stored Procedures, and then hook into the Sitecore item:created, item:renamed, item:saved, item:moved and item:deleted events.

STEP 1: THE BASIC DATABASE PATTERN

First we need to create a table for the data. Maybe you like to create your tables in a different matter, and that is OK, but this is the minimum required fields needed to perform a data audit:

CREATE TABLE [dbo].[MyItem]
(
    [SitecoreItemID] UNIQUEIDENTIFIER NOT NULL PRIMARY KEY,
	...
	... here you define the fields from the Sitecore template
	...
    [Created] DATETIME NOT NULL DEFAULT GetDate(), 
    [Updated] DATETIME NULL, 
    [Deleted] DATETIME NULL, 
    [IsDeleted] BIT NOT NULL, 
    [SitecoreUserName] NVARCHAR(255) NOT NULL,
    [SysStartTime] DATETIME2 GENERATED ALWAYS AS ROW START NOT NULL,
    [SysEndTime] DATETIME2 GENERATED ALWAYS AS ROW END NOT NULL,
    PERIOD FOR SYSTEM_TIME (SysStartTime,SysEndTime)
    )
WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.MyItem));
  • [SitecoreItemID]
    Is the key value is the Sitecore item id. Some like to have an auto-incrementing integer as key, and that’s also fine, as long as you at least have the Sitecore item ID in the database.
  • [Created], [Updated], [Deleted]
    Timestamps are used to determine when database operations are carried out
  • [IsDeleted]
    Instead of deleting items, we mark them as deleted. The [Deleted] timestamp will tell us when the item was deleted.
  • [SitecoreUsername]
    Is the name of the Sitecore editor creating/updating/deleting the item
  • [SysStartTime],[SysEndTime]
    Is used by SQL to handle the system-versioning.
  • Remember to add all the fields from your Sitecore template that needs to be synchronized.

Also note that in my example, I do not store the item hierarchy, item languages or item versions.

STEP 2: UPSERT AND DELETE STORED PROCEDURES:

The basic UPSERT Stored procedure is as follows:

  • Select from database with [SitecoreItemID]
  • If found: Update
    Update with values from the Sitecore template
    Set [Updated] to current datetime
    Set [SitecoreUserName] with the Sitecore username
  • If not found: Insert
    Insert values from the Sitecore template
    Set [SitecoreUserName] with the Sitecore username

The basic DELETE Stored procedure is as follows:

  • Set [IsDeleted] to TRUE
    Set [SitecoreUserName] with the Sitecore username
    Set [Deleted] with the current datetime.

STEP 3: HOOK INTO SITECORE 

Now for the fun part, the Sitecore code.

We need to hook into the following Sitecore events:

  • item:created: Call the UPSERT stored procedure
  • item:renamed:Call the UPSERT stored procedure
  • item:saved: Call the UPSERT stored procedure
  • item:moved: Call the UPSERT stored procedure
  • item:deleted: Call the DELETE stored procedure

My code will only sync one language and always the latest version. If you need to sync more languages or versions, remember to add those as key values to your database.

First the code. This is a pseudo-code example, outlining what you need to implement.

using System;
using Sitecore.Data.Items;
using Sitecore.Events;

namespace MyCode
{
  public class SyncItems
  {
    public void OnItemChanged(object sender, EventArgs args)
    {
      Item changedItem = Event.ExtractParameter(args, 0) as Item;
      if (changedItem == null)
        return;
      // We only synchronize one language and only the latest version:
      if (changedItem.Language != Language.Parse("en"))
        return false;
      if (!changedItem.Versions.IsLatestVersion())
        return false;
        
      // If changed item is of the template we wish to synchronize:
      // Call the upsert stored procedure
    }

    public void OnItemMoved(object sender, EventArgs args)
    {
      Item movedItem = Event.ExtractParameter(args, 0) as Item;
      if (movedItem == null)
        return;
      // We only synchronize one language and only the latest version:
      if (movedItem.Language != Language.Parse("en"))
        return false;
      if (!movedItem.Versions.IsLatestVersion())
        return false;

      // If moved item is of the template we wish to synchronize:
      // Call the upsert stored procedure
    }
    
    public void OnItemDeleted(object sender, EventArgs args)
    {
      Item deletedItem = Event.ExtractParameter(args, 0) as Item;
      if (deletedItem == null)
        return;
      // We only synchronize one language and only the latest version:
      if (deletedItem.Language != Language.Parse("en"))
        return false;
      if (!deletedItem.Versions.IsLatestVersion())
        return false;

      // If deleted item is of the template we wish to synchronize:
      // Call the delete stored procedure
    }
}   

Then the Sitecore configuration:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:env="http://www.sitecore.net/xmlconfig/env/">
  <sitecore>
    <events>
      <event name="item:created">
        <handler type="MyCode.SyncItems, MyCode" method="OnItemChanged"/>
      </event>
      <event name="item:renamed">
        <handler type="MyCode.SyncItems, MyCode" method="OnItemChanged"/>
      </event>
      <event name="item:saved">
        <handler type="MyCode.SyncItems, MyCode" method="OnItemChanged"/>
      </event>
      <event name="item:moved">
        <handler type="MyCode.SyncItems, MyCode" method="OnItemMoved"/>
      </event>
      <event name="item:deleted">
        <handler type="MyCode.SyncItems, MyCode" method="OnItemDeleted"/>
      </event>
    <events>
  </sitecore>
</configuration>

THINGS TO CONSIDER:

Should you use a synchronous or asynchronous update method?

If you call the stored procedures synchronously, all code will wait for and answer from the  SQL server. If you a asynchronous approach, you don’t catch exceptions from the database. It is up to you how critical errors are. There are coding patterns that will give you a more foul proof approach such as database retry mechanisms, or implementing a queue in between Sitecore and the database. You decide what is best.

What happens if you move an item with children?

If you are synchronizing items with children, and you stored the hierarchy of items in the database, you will need to make a cascading update on those children. Sitecore will not throw individual events for the children, only the root item that is moved.

What happens if users restore items from the recycle bin?

This is a situation that is currently undetermined, as there is no event attached to the recycle bin restore.

Are there any benefits? Why don’t just use Sitecore versioning and workflows?

Workflows are designed to enforce business rules upon editors, and implies a very stringent way of working. Workflows discourages casual editing and is often abandoned for that very reason. You should decide if a simple workflow would work for you.

MORE TO READ:

About briancaos

Developer at Pentia A/S since 2003. Have developed Web Applications using Sitecore Since Sitecore 4.1.
This entry was posted in .net, c#, Sitecore 5, Sitecore 6, Sitecore 7, Sitecore 8, Sitecore 9 and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.