Post reply

Best Strategy for Implementing (23) Incremental Table Loads

  • November 17, 2016 at 1:09 pm

    #401074

    Hi All,

    This is more a question of theory than a technical one, but here it goes. I need to mirror (23) SQL tables in one database over to another database. The goal is to maintain an exact copy of these tables found in the source db over in the destination db, and then tying those tables in with other tables to do BI.

    So, do I create (23) separate packages (one for each table) and then create a master package that employs the logic of activating them all? Or do I try to employ all the incremental table loads in one big package? I imagine the Control Flow logic will become a big bowl of spaghetti with the first option.

    Any suggestions?

    Thanks,

    Jerid

  • November 17, 2016 at 1:18 pm

    #1913229

    Jerid421 (11/17/2016)

    Hi All,

    This is more a question of theory than a technical one, but here it goes. I need to mirror (23) SQL tables in one database over to another database. The goal is to maintain an exact copy of these tables found in the source db over in the destination db, and then tying those tables in with other tables to do BI.

    So, do I create (23) separate packages (one for each table) and then create a master package that employs the logic of activating them all? Or do I try to employ all the incremental table loads in one big package? I imagine the Control Flow logic will become a big bowl of spaghetti with the first option.

    Any suggestions?

    Thanks,

    Jerid

    Separate packages is the way to go, IMO. Then you can easily play around with parallelism in the master package. You will also find that it is very much easier to maintain and to debug any errors (because the source of those errors will be much more readily apparent).

    The absence of evidence is not evidence of absence
    - Martin Rees
    The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
    - Phil Parkin

  • November 18, 2016 at 6:10 am

    #1913391

    Separate packages are advised for maintenance.

  • November 21, 2016 at 9:02 am

    #1913773

    Thanks Phil. I thought that I was going down the right path (and for the same reasons). : )

Viewing 4 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply