October 5, 2012 at 9:56 am
Hi
I have one large table in one database and i am am going to move the data and turn it into a normalised relational database with a number of tables.
As all the data is coming from a single table is there a way of just importing it once (say into a recordset or something) and then running other data flow tasks from that recordset?
and is there a pause transform that will wait till one item is finished before starting?
Thanks in advance
October 5, 2012 at 10:40 am
your best solution would be to create multiple dataflows in the control flow area of your package.
create your first dataflow to move the data to the single table on your new server.
the create the remaining data flows within a sequence container and use the control flow expressions to completetion to ensure your data flows then execute in the correct order .....
alternatively, you may just want to have a second data flow reading of the single table and use conditional splits to split the data into logic paths........
(I don't know your data or it's complexity but is an alternative). Hope that helps
_____________________________________________________________________________MCITP: Business Intelligence Developer (2005)
October 5, 2012 at 1:01 pm
You might want to look into the use of .RAW files in SSIS. They're basically SSIS's version of temp tables. What they do is pull off a single system into a file (local to the SSIS server is best) in a high-speed access version of the data (you won't be able to read it directly).
That raw can then be manipulated repeatedly. My personal use for it is usually when dealing with multiple warehouses, where I need to do heavy transforms and then port the results to multiple targets. First I'll do all the transforms into the raw, and then start pumping it out to the target systems. The reason for this is so I can checkpoint the transformation, keeping me from having to hit the source system more than once in case of a connection failure or somesuch.
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
October 8, 2012 at 2:48 am
thanks very much for the replies guys
ill look into both methods
October 8, 2012 at 7:17 am
You can also use a multicast to direct the recordset to multiple Destination objects in one Data Flow.
Thomas LeBlanc, MVP Data Platform Consultant
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply