Sitecore PostSessionEndPipeline failed – MongoDB.Driver.MongoQueryException

In my (huge) Sitecore solution we experienced the following error:

ERROR PostSessionEndPipeline failed.
Exception: MongoDB.Driver.MongoQueryException
Message: QueryFailure flag was Runner error: Overflow sort stage buffered data usage of 33570904 bytes exceeds internal limit of 33554432 bytes (response was { “$err” : “Runner error: Overflow sort stage buffered data usage of 33570904 bytes exceeds internal limit of 33554432 bytes”, “code” : 17144 }).
Source: MongoDB.Driver
at Sitecore.Analytics.Pipelines.CommitSession.SubmitSession.Process(CommitSessionPipelineArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Analytics.Pipelines.PostSessionEnd.CommitSession.Process(PostSessionEndArgs args)
at (Object , Object[] )
at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)
at Sitecore.Web.Application.RaiseSessionEndEvent(HttpApplication context)

Followed by the following message:

QueryFailure flag was Runner error: Overflow sort stage buffered data usage of xxxxxxxx bytes exceeds internal limit of 33554432 bytes (response was { “$err” : “Runner error: Overflow sort stage buffered data usage of xxxxxxxx bytes exceeds internal limit of 33554432 bytes”, “code” : 17144 }).

The error is caused by an internal 32MB memory size restriction in MongoDB.

The problem can be solved by adding an index to the Interactions collection. This will allow Sitecore to stream interactions sorted rather than attempting to load them all into memory and do the sorting in memory.

Connect to the analytics index using RoboMongo or MongoVUE and run the following query:

db.Interactions.createIndex(
{
    "StartDateTime" : 1,
    "_id" : 1
}
)

A WORD OF CAUTION:

According to MongoDB, adding indexes to a live environment does not have any side effects. But running the above query on my 7.5 million entries in the Interactions collection caused my entire Sitecore to fail (total catastrophic failure) for 5-10 minutes until the index was built.

Thank you to Sitecore Support and Joshua Wheelock for helping with the issue.

MORE TO READ:

About briancaos

Developer at Pentia A/S since 2003. Have developed Web Applications using Sitecore Since Sitecore 4.1.
This entry was posted in Sitecore 8 and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s