Export large amount of data from one table to another

  • Hi Guys,

    Is there a way to start up an export from one table to another, if the data export fails at some point? I want to create an SSIS package to load data to a table and in case the export fails at some point I can restart it again from the point that it fails.

    Please advise.

    Regards

    IC

  • Normally no, because the load happens in a transaction.

    It depends a bit on the settings of the OLE DB Destination. You might want to lower the batch size to try if batches are committed more quicker.

    At a restart, what you would have to do is check the destination which rows are already loaded and exclude them from the source.

    But, having smaller transactions/batches slows down the loading speed.

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

  • I was thinking there was some support for checkpoints in ssis, tho I have never used them since we mostly do kill n fill.

    My understanding is that you could restart from a checkpoint.

  • herladygeekedness (10/1/2013)


    I was thinking there was some support for checkpoints in ssis, tho I have never used them since we mostly do kill n fill.

    My understanding is that you could restart from a checkpoint.

    Yes, but you cannot restart in the middle of a data flow.

    Restart Packages by Using Checkpoints

    Need an answer? No, you need a question
    My blog at https://sqlkover.com.
    MCSE Business Intelligence - Microsoft Data Platform MVP

  • ah, yes, Makes sense to me. Possibly why I abandoned the idea for our use.

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply