Migrating data to a live on-premises CRM system?


While working on one of the project, we were required to move data from legacy system into a live production system with limited amount of black out window. One can calculate the time required based on number of records, server configuration, etc. and can even have a buffer. But you never know when Murphy will play his role.

Below are few points that, if considered can help Murphy stick to its seat and don’t show up. (Just some pre-checks that helps the processJ)

  • Confirm disk space on SQL, web and application server. Have some buffer space as logs will grow.
  • As your target system is a live production system, surly there will be some scheduled maintenance jobs running to maintain server health as part of disaster management. These jobs are life savers but problem with them is that these are scheduled in down time as they consume high resources. And unfortunately this is the only time when we can perform our import. Thus you may have to consider pausing the maintenance jobs. But do remember to turn them on once done with the import. Some of the resource consuming jobs are: consistency check, database backup, async-operation cleanup, POA, etc. Pausing this job helps the import utilize maximum available server resources.
  • Check the database log size and clear. The log will grow with the import and if it reaches the maximum available threshold the import processes throws timeout errors. Also check the shrink process, preferred if simple.
  • Check for CPU and Memory usage on SQL, web and application server.
  • Check for any blockage on SQL server, if any script is blocking or slow running queries.
  • Clear AsyncOperationBase table.
  • You may run into scenario where it is required to restart the SQL service. Make sure that this do not affect any other process. Also in case of NLB, upon restarting the SQL service switches the active node. Thus you will also have to consider that node for performance check.
  • You may have to disable the user logging and turn off the workflows and plugins.

And last but not the least is FULL database backup before start of the process. Also you make to do this in several passes due to amount of data and limited amount of down time. Identity the steps that can be performed outside black out that do not affect live system. This provides some extra time for completing the critical steps.

I would recommend to do the import in multiple small passes which helps in keeping the buffer and reduces the chances of breaking things or running on edges. After all “Rome was not built in a day”.

These are some of the steps that helped me. As always, these may not match exactly to your requirement but some of them will surly. And I don’t guaranty of anything from the steps as risk will be yours as it’s your production system.

If you have anything to add, please write in comment and I will update the content. Thanks!

Advertisements

2 thoughts on “Migrating data to a live on-premises CRM system?

  1. Adi November 22, 2015 / 8:49 pm

    Is there any limit for no. of records to be imported? or this process will help in any number of records?

    Like

    • Mandar Joshi November 23, 2015 / 11:00 am

      This is not a complete import process but just some pre-steps which will ensure that your import completes without any issues in analysed downtime. It is more helpful if number of records are high but there is no reason why you cannot use it for small number of records. Thanks!

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s