Migrating huge amounts of Sitecore content – Use the Sitecore Serializer

Migrating contents from one Sitecore to another is a common task. We move templates, layouts, content, media library items etc. from development to test and from test to production all the time.

Usually we use the Sitecore Pagkage Designer to pack contents into a package that can be installed on another Sitecore. But sometimes that’s not the best solution.

MIGRATING HUGE AMOUNTS OF DATA

Yesterday I had to move 28.000 items + 16 GB of Media Library contents from development to test.

The packager will not support this amount of contents. A Sitecore package is basically a .zip file with serialized Sitecore items. Each item is a file inside the .zip file.

I tried to create a package containing my 28.000 items, but the size of the zip file grew bigger than 2 GB which caused the package to fail.

HOW TO EASILY MIGRATE 16+GB OF DATA – THE INITIAL SETUP

Instead of creating one (or several) packages of data I create one package containing the “basics” of the project to migrate, i.e.:

• New Templates (adding dynamically from /sitecore/templates)
• New Layouts (adding dynamically from/sitecore/layouts)
• New System settings (adding dynamically from /sitecore/system)
• New …

Furthermore i add the ROOT node of the contents to migrate:

• The root node of the Media Library folder I used to create my media library items
• The root node of the contents to move

Adding root nodes only

The package is now substantially smaller, about 1 Mb  which is a great size for a package. It installs fast and is easy to move around.

This package is installed on the destination Sitecore.

MOVING THE 16+GB DATA

Now I am ready to migrate the actual contents. As said before, 16 GB of data is not easily movable. Unless you use … the Sitecore tree serializer!

The Sitecore serializer can be found on the “Developer” tab. Right click the tabs and select the Developer tab:

Right click to select the Developer Tab

Select the item to serialize and click the “Serialize Tree” button:

Serialize Tree

Now Sitecore is serializing the item + all sub items into separate files on your hard drive  usually in the /app_data/serialization folder:

Serialized Data

It will take some time. (It took me 3 hours to generate 16 GB media library items in 12.000 individual files.)

When it’s finished, copy the files to your destination Sitecore machine, and reverse the process by selecting the root node you included in your package and clicking “Revert Tree”:

Revert Tree

It will take a LOT of time if you have 16+gb of data. But don’t worry, Sitecore is creating a Sitecore Job to de-serialize the data. So you can close the browser while the process is running.

When the job is done you can remove the serialized files from the source and destination machines.