January 21, 2011 at 11:59 am
Hi,
We process huge amounts of data using many SQL server 2008 SSIS packages and we do validations on the data.I want to move the validations to one central place so that we don't have to rewrite the same validations over and over again.Maintaining the validations in multple SSIS package is becoming a hassle as if we make a small change we need to make sure it is done in all the packages,We also want to move it out of sp and udf.What is best approach to do this?
Thanks
SSM
January 21, 2011 at 12:48 pm
Can you create a Validations DLL file using .NET studio and pass the data through that?
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
March 31, 2011 at 1:07 pm
if the amount of data to be vaidated is large, is it recommended to use the dll for validation or the sp's?
Thanks,
SSM
August 15, 2011 at 8:28 am
I am wondering about this as well. My current solution was to create a table that described the rules and associated tables that understand how the rules are to be implemented. What I am experimenting with right now is being able to execute these rules against large sets of data, say 500k at a time. In order to execute these rules I am currently using regular expressions, specifically regex.replace. While this seems to be working ok I too am wondering if there are better ways to implement validation rules. That being said, in my case I don't have access to SSIS so all of this needs to be able to be run through .net, tsql or possibly some 3rd party tool (rhino.etl?). Would love some opinions on very high performance validation rule processing.
Viewing 4 posts - 1 through 3 (of 3 total)
You must be logged in to reply to this topic. Login to reply