October 12, 2011 at 7:41 am
Hi
I have a comma delimited flat file of 1000 records (the delimiter is /&^$%@ as the columns are free text). However SSIS only seems to import 500 of them.
Is there a setting or some issue that I might not be aware of?
October 12, 2011 at 10:44 am
There is no setting I'm aware of that would cause this. Could be an some kind of special character in the data that is signalling an end of file.
Jack Corbett
Consultant - Straight Path Solutions
Check out these links on how to get faster and more accurate answers:
Forum Etiquette: How to post data/code on a forum to get the best help
Need an Answer? Actually, No ... You Need a Question
October 12, 2011 at 1:15 pm
All sorted, it was carriage return/line feed weirdness :blush:
October 12, 2011 at 4:44 pm
Cowboy DBA,
How did you determine what the issue was.
I ask because I am having the same issue.
Thanks
GF
October 12, 2011 at 11:13 pm
In essence I checked the flat file source and previewed the data. I saw that for some reason the odd numbered rows (i.e. rows 1,3,5,7,9 etc) seemed to be ignoring the carriage return/line feed. This meant that rows 2,4,6,8,10 etc were getting pivoted on the same row as the odd numbered row. This gave rise to there being only half the rows being passed through the dataflow.
There was no errors because I set the dataflow to only pass through specific rows into the database e.g. Name, DoB, Street, Town, City etc As the even numbered rows were being inserted into "Column 1", "Column 2" etc they were being ignored and no errors being reported.
The CSV I use is produced from an Excel spreadsheet via a .NET script task. I've managed to make the script task a bit smarter and so far it appears to do what I want i.e. enforce CRLF so that I don't get the even numbered rows being "pivoted" on the extra columns.
Hope that makes sense 🙂
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply