Make the load incremental

Question

Make the load incremental

SQL_Surfer

SSCrazy Eights

Points: 8320
More actions
June 14, 2013 at 9:34 am

#275051

I have a view that pulls the data from severla large tables and then I load it to another table. Currently, the load takes about 40 mins as it is a full load. I like to make it incremental. But I couldn't find a sets of key that would make the record unique. Any suggestions on how to make it incremental?
Forum Etiquette: How to post Reporting Services problems

[/url]

Forum Etiquette: How to post data/code on a forum to get the best help - by Jeff Moden
[/url]

How to Post Performance Problems - by Gail Shaw
[/url]

gravitysucks Ten Centuries Points: 1122 More actions · Answer 1

SQL_Surfer (6/14/2013)
I have a view that pulls the data from severla large tables and then I load it to another table. Currently, the load takes about 40 mins as it is a full load. I like to make it incremental. But I couldn't find a sets of key that would make the record unique. Any suggestions on how to make it incremental?

One way i cann think of is to make it incremental is by adding surrogate keys in the source table if you don't already have a datetime tracking column in those sources.

Learner44 SSCarpal Tunnel Points: 4466 More actions · Answer 2

Try the following one..

http://www.sqlservercentral.com/articles/EDW/77100/

You will get your solution i guess...

Learner44 SSCarpal Tunnel Points: 4466 More actions · Answer 3

try this one..

http://www.sqlservercentral.com/articles/EDW/77100/

gravitysucks Ten Centuries Points: 1122 More actions · Answer 4

Learner44 (6/14/2013)

try this one..

http://www.sqlservercentral.com/articles/EDW/77100/%5B/quote%5D

The Merge/Upsert will only work if there is (or create) defined set of unique key(s)-which according to the OP doesn't exist here.

SQL_Surfer SSCrazy Eights Points: 8320 More actions · Answer 5

SQL_Surfer

SSCrazy Eights

Points: 8320

June 14, 2013 at 5:18 pm

#1624295

That is absolutely correct.

mister.magoo SSC-Forever Points: 47174 More actions · Answer 6

SQL_Surfer (6/14/2013)
I have a view that pulls the data from severla large tables and then I load it to another table. Currently, the load takes about 40 mins as it is a full load. I like to make it incremental. But I couldn't find a sets of key that would make the record unique. Any suggestions on how to make it incremental?

Why worry about unique? I can have a table with 100000 identical rows. As long as row# 100001 has something about it that shows it is new - that is all I need.

Question : Can you manually - as a living breathing human - look at the currently loaded data and the output of the view and determine what to select for an incremental load? If not, you will need to modify the source tables. If you can, then you just need to translate that into code...

MM

select geometry::STGeomFromWKB(0x0106000000020000000103000000010000000B0000001000000000000840000000000000003DD8CCCCCCCCCC0840000000000000003DD8CCCCCCCCCC08408014AE47E17AFC3F040000000000104000CDCCCCCCCCEC3F9C999999999913408014AE47E17AFC3F9C99999999991340000000000000003D0000000000001440000000000000003D000000000000144000000000000000400400000000001040000000000000F03F100000000000084000000000000000401000000000000840000000000000003D0103000000010000000B000000000000000000143D000000000000003D009E99999999B93F000000000000003D009E99999999B93F8014AE47E17AFC3F400000000000F03F00CDCCCCCCCCEC3FA06666666666FE3F8014AE47E17AFC3FA06666666666FE3F000000000000003D1800000000000040000000000000003D18000000000000400000000000000040400000000000F03F000000000000F03F000000000000143D0000000000000040000000000000143D000000000000003D, 0);