September 14, 2014 at 10:40 pm
I got this basic question while inserting huge number of records from production table to the test environment. Please let me know.
Thanks in advance.
______________________________________________________________Every Problem has a Solution; Every Solution has a Problem: 🙂
September 14, 2014 at 11:05 pm
karthik babu (9/14/2014)
I got this basic question while inserting huge number of records from production table to the test environment. Please let me know.Thanks in advance.
Quick questions:
1. Is the identity column the clustered index key?
2. Can you force the constantly increasing order of the insert?
3. Do the identity values need to be in sync?
4. What are you using for transferring the data?
5. Could restoring from production backup be an option?
6. Is there a clustered index on the table?
7. Are there other indexes on the table?
The answer is Normally not but it depends.
😎
September 14, 2014 at 11:59 pm
PFB the response...
1. Is the identity column the clustered index key? - Yes
2. Can you force the constantly increasing order of the insert? - No
3. Do the identity values need to be in sync? - Not necessary
4. What are you using for transferring the data? - SSMS Query Analyzer
5. Could restoring from production backup be an option? - Yes but size of DB is so huge, so stick to one particular table
6. Is there a clustered index on the table? - Yes
7. Are there other indexes on the table? - Yes
BTW it took 32 mins to complete 60 million records. 🙂
______________________________________________________________Every Problem has a Solution; Every Solution has a Problem: 🙂
September 15, 2014 at 1:46 am
Here is my suggestion
Leave the identity to the destination table, don't use the identity_insert. Consider dropping the non clustered indexes and recreate them after the load. In addition, an SSIS package with an OLE DB destination and fastload could help speed this up further.
😎
Yet another question, are the two databases on the same server?
September 15, 2014 at 3:24 am
I believe this is a good solution and it will work better than with identity_insert on.
Thnx.
______________________________________________________________Every Problem has a Solution; Every Solution has a Problem: 🙂
September 15, 2014 at 12:23 pm
The big delay is likely in allocating and formatting log space, not in the identity number assignment, which is trivial.
Thus, before you run the big INSERT, pre-allocate sufficient log space to handle logging for the whole statement.
Or, if the db is in simple mode and/or you can backup the log between batches, break the INSERT into multiple batches, with a CHECKPOINT inbetween each, rather than a single big statement.
SQL DBA,SQL Server MVP(07, 08, 09) "It's a dog-eat-dog world, and I'm wearing Milk-Bone underwear." "Norm", on "Cheers". Also from "Cheers", from "Carla": "You need to know 3 things about Tortelli men: Tortelli men draw women like flies; Tortelli men treat women like flies; Tortelli men's brains are in their flies".
Viewing 6 posts - 1 through 5 (of 5 total)
You must be logged in to reply to this topic. Login to reply