August 16, 2015 at 1:29 pm
I have a source from which I select the AccountID, StatusID, and value. I am using an aggregation Transform to find the SUM(Value) GROUP BY AccountID, StatusID.
I will have almost 3 million records, I sum all of them and look up the destination and insert / update / delete the values from destination.
Suggest me which is a good way? Aggregation or Script Task?
if script task, please suggest an article on how to group by using script since I have not much idea on writing script but can understand the code.
Thank you in advance
August 17, 2015 at 12:29 am
ilovedata (8/16/2015)
I have a source from which I select the AccountID, StatusID, and value. I am using an aggregation Transform to find the SUM(Value) GROUP BY AccountID, StatusID.I will have almost 3 million records, I sum all of them and look up the destination and insert / update / delete the values from destination.
Suggest me which is a good way? Aggregation or Script Task?
if script task, please suggest an article on how to group by using script since I have not much idea on writing script but can understand the code.
Thank you in advance
Quick question, can you tell us a little bit more, i.e. what is the data source? The destination?
😎
First thought is to do the aggregation in the select from the source but depending on the source that may not be an option.
August 17, 2015 at 12:38 am
Eirikur Eiriksson (8/17/2015)
ilovedata (8/16/2015)
I have a source from which I select the AccountID, StatusID, and value. I am using an aggregation Transform to find the SUM(Value) GROUP BY AccountID, StatusID.I will have almost 3 million records, I sum all of them and look up the destination and insert / update / delete the values from destination.
Suggest me which is a good way? Aggregation or Script Task?
if script task, please suggest an article on how to group by using script since I have not much idea on writing script but can understand the code.
Thank you in advance
Quick question, can you tell us a little bit more, i.e. what is the data source? The destination?
😎
First thought is to do the aggregation in the select from the source but depending on the source that may not be an option.
If the aggregation cannot be done at the source, I would suggest doing it at the target (ie, loading all of the data into a staging area and then using the target RDBMS to do the aggregation for you). Do not try this in SSIS, it will not perform well.
The absence of evidence is not evidence of absence
- Martin Rees
The absence of consumable DDL, sample data and desired results is, however, evidence of the absence of my response
- Phil Parkin
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply