Error Handling and Fault Tolerance in ETL

• cutespn

  SSCommitted

  Points: 1669

  More actions

  July 30, 2009 at 11:45 pm

  #208850

  Hi ,

  I am looking for some ideas/patterns for row level error handling in ETL.

  Currently our ETL expects all the source system records to be in certain format. If one record is faulty our entire ETL fails .

  We are envisioning an architecture to park this faulty records in some kind of audit tables and continue with rest of records.

  We would like to seek some inputs if anybody has encountered similar scenario.

  Thanks in Advance,

  Sudheer.

  
  
  
  
  
  

  Regards,
  Sudheer 
  

  
  

  

• Grant Fritchey

  SSC Guru

  Points: 398880

  More actions

  July 31, 2009 at 5:45 am

  #1032866

  Many of the SSIS operators have the ability to pull rows with errors into a seperate path where you can store them, fix them, whatever, all without shutting down the process.

  "The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood"
  - Theodore Roosevelt

  Author of:
  SQL Server Execution Plans
  SQL Server Query Performance Tuning

• Jack Corbett

  SSC Guru

  Points: 184394

  More actions

  July 31, 2009 at 8:32 am

  #1032972

  Check out this article, Custom Error Handling in SSIS[/url], which contains a link to an earlier article that I wrote. Both expand on what Grant mentioned.

  Jack Corbett
  Consultant - Straight Path Solutions
  Check out these links on how to get faster and more accurate answers:
  Forum Etiquette: How to post data/code on a forum to get the best help
  Need an Answer? Actually, No ... You Need a Question