Migrating contents from one Sitecore to another is a common task. We move templates, layouts, content, media library items etc. from development to test and from test to production all the time.
Usually we use the Sitecore Pagkage Designer to pack contents into a package that can be installed on another Sitecore. But sometimes that’s not the best solution.
MIGRATING HUGE AMOUNTS OF DATA
Yesterday I had to move 28.000 items + 16 GB of Media Library contents from development to test.
The packager will not support this amount of contents. A Sitecore package is basically a .zip file with serialized Sitecore items. Each item is a file inside the .zip file.
I tried to create a package containing my 28.000 items, but the size of the zip file grew bigger than 2 GB which caused the package to fail.
HOW TO EASILY MIGRATE 16+GB OF DATA – THE INITIAL SETUP
Instead of creating one (or several) packages of data I create one package containing the “basics” of the project to migrate, i.e.:
- New Templates (adding dynamically from /sitecore/templates)
- New Layouts (adding dynamically from/sitecore/layouts)
- New System settings (adding dynamically from /sitecore/system)
- New …
Furthermore i add the ROOT node of the contents to migrate:
- The root node of the Media Library folder I used to create my media library items
- The root node of the contents to move
The package is now substantially smaller, about 1 Mb which is a great size for a package. It installs fast and is easy to move around.
This package is installed on the destination Sitecore.
MOVING THE 16+GB DATA
Now I am ready to migrate the actual contents. As said before, 16 GB of data is not easily movable. Unless you use … the Sitecore tree serializer!
The Sitecore serializer can be found on the “Developer” tab. Right click the tabs and select the Developer tab:
Select the item to serialize and click the “Serialize Tree” button:
Now Sitecore is serializing the item + all sub items into separate files on your hard drive usually in the /app_data/serialization folder:
It will take some time. (It took me 3 hours to generate 16 GB media library items in 12.000 individual files.)
When it’s finished, copy the files to your destination Sitecore machine, and reverse the process by selecting the root node you included in your package and clicking “Revert Tree”:
It will take a LOT of time if you have 16+gb of data. But don’t worry, Sitecore is creating a Sitecore Job to de-serialize the data. So you can close the browser while the process is running.
When the job is done you can remove the serialized files from the source and destination machines.
Could be useful for imporing new data as well? I mean, write files from some other system in serializer format, and then import those?
LikeLike
Yes I guess you could, provided that you can understand the serialized data. This is an example of a media folder item in the media library:
—-item—-
version: 1
id: {E109AB9A-1A1C-446D-BB16-5FF7CC275343}
database: master
path: /sitecore/media library/MyMediaLibraryFolder
parent: {3D6658D8-A0BF-4E75-B3E2-D050FABCF4E1}
name: DoctorsHandbook
master: {00000000-0000-0000-0000-000000000000}
template: {FE5DD826-48C6-436D-B87A-7C4210C7413B}
templatekey: Media folder
—-version—-
language: da
version: 1
revision: 06fe7a44-3646-4afa-a903-01ba78674bae
—-field—-
field: {52807595-0F8F-4B20-8D2A-CB71D28C6103}
name: __Owner
key: __owner
content-length: 14
sitecore\admin
—-field—-
field: {25BED78C-4957-4165-998A-CA1B52F67497}
name: __Created
key: __created
content-length: 15
20120116T110618
—-field—-
field: {5DD74568-4D4B-44C1-B513-0AF5F4CDA34F}
name: __Created by
key: __created by
content-length: 14
sitecore\admin
—-field—-
field: {8CDC337E-A112-42FB-BBB4-4143751E123F}
name: __Revision
key: __revision
content-length: 36
06fe7a44-3646-4afa-a903-01ba78674bae
—-field—-
field: {D9CF14B1-FA16-4BA6-9288-E8A174D4D522}
name: __Updated
key: __updated
content-length: 34
20120116T110618:634623087788763595
—-field—-
field: {BADD9CF9-53E0-4D0C-BCC0-2D784C282F6A}
name: __Updated by
key: __updated by
content-length: 14
sitecore\admin
I’m not sure the format was intended for outsiders :)
LikeLike
Something to keep in mind is that Windows has a file path limit of 255 characters. Since Sitecore recreates the folder structure of your Sitecore tree on the file system a deeply nested tree can easily exceed the limit. You will not receive any type of error message from Sitecore, the items that exceed the limit just won’t be serialized. If you’re not careful, you can lose data during a migration and not even realize it.
LikeLike
It seems the serializer is smart so that it doesn’t always create deep nesting on the file system. It does error if you have massive item names though which is a pain.
LikeLike
Yes, the file system has a 256 character limit to the length of the path crated. If the item names in the tree combined exceeds this limit, the last levels are not serialized.
It wold be nice if Sitecore changed the serializer so the folder names were not using the item name, but a short indicator, for example the sortorder of the item.
LikeLike
Agreed. Or a guid or something. Hmmm… I’m thinking feature request.
LikeLike