February 19, 2013 at 2:14 pm
I'm trying to replace an existing function with an SSIS package. Rows are exported to a comma delimited flat file for archiving and deleted from the source table.
The current method produces a file with data quoted and blank columns not quoted.
Ex "data1","data2",,"data4"
I set the Text Qualifier on my Flat File Connection Mgr to "
But I get this: "data1","data2","","data4"
All columns in the Flat File Connection Mgr are strings.
I don't normally quote text on an export, but since I'm mimicking another process I thought I'd try and be exact.
Can this be done?
Thanks all.
February 20, 2013 at 1:33 am
I believe you can use a derived column to null the field before it goes outbound. You've got a 0-length in there and that's what's trying to send, if my faulty memory serves me correctly.
Basically something like this:
TRIM(cola) == "" ? (DT_STR,<len>,1252)NULL(DT_STR,<len>,1252) : cola
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
February 20, 2013 at 9:58 am
I couldn't get that to work. I mean the syntax is good, but I got the same result. Its true that these are 0-length strings. So to make them NULL I changed the select query to:
case when ltrim(rtrim(mycol)) = '' then NULL else ltrim(rtrim(mycol)) end as mycol.
In the preview I saw the NULL value, but it still got quoted. Seems like the Text Qualifier in the flat file connection manager is overriding everything.
February 20, 2013 at 10:42 am
Hm. The couple of semi-official things I've seen seem to point to that functioning. Very strange.
Can you check the dataviewer right before the outbound flat file and make sure that the columns you have mapped are actually null?
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
February 20, 2013 at 12:25 pm
Yea, they are empty string before the derived column transformation
and NULL after the derived column transformation.
And the result is all columns quoted.
Screen shots are attached.
In the attached gifs you can see column EdiCarrier is one of the culprits. This columns has data in some rows, but not all. So I changed the query to only select rows where EdiCarrier = '' wondering if the column was NULL for every row, might it make a difference. There was no change in the behavior. I also deleted the flat file connection mgr and recreated it. I've seen it where it seems like a flat file connection definition gets "stuck" after making edits to it.
February 20, 2013 at 2:51 pm
I read elsewhere that since a flat file is a text file, if you use a Text Qualifier it will be applied to every column.
So here's what works... remove the Text Qualifier and as a derived column use:
mycol !="" ? "\"" + mycol + "\"" : mycol
This adds two to the length of every column with data. I have some datetime data types in the table that got converted to char(10) for formatting in the select query and I was failing with a truncation error on them. I could not edit the length in the Derived Column Transformation Editor.
So I tried mycol !="" ? DT_STR(<len+2>,1252)("\"" + mycol + "\"") : mycol. It didn't fail, but gave me "10/09/201 in the date columns in the output file. Not sure what I was doing wrong there.
So I changed the select query to pad all the columns giving ample space for the max data plus the quotes. Ex: convert(varchar(50),convert(varchar(10),datetimeColumn,101)) as mycol.
Learned a little more about SSIS Expression Language today!
Thanks for all your time and effort.
February 20, 2013 at 5:05 pm
Randy Doub (2/20/2013)
I read elsewhere that since a flat file is a text file, if you use a Text Qualifier it will be applied to every column.So here's what works... remove the Text Qualifier and as a derived column use:
mycol !="" ? "\"" + mycol + "\"" : mycol
This adds two to the length of every column with data. I have some datetime data types in the table that got converted to char(10) for formatting in the select query and I was failing with a truncation error on them. I could not edit the length in the Derived Column Transformation Editor.
So I tried mycol !="" ? DT_STR(<len+2>,1252)("\"" + mycol + "\"") : mycol. It didn't fail, but gave me "10/09/201 in the date columns in the output file. Not sure what I was doing wrong there.
So I changed the select query to pad all the columns giving ample space for the max data plus the quotes. Ex: convert(varchar(50),convert(varchar(10),datetimeColumn,101)) as mycol.
Learned a little more about SSIS Expression Language today!
Thanks for all your time and effort.
Randy, just watch out for when mycol contains any double quotes, then you have a problem...
MM
select geometry::STGeomFromWKB(0x0106000000020000000103000000010000000B0000001000000000000840000000000000003DD8CCCCCCCCCC0840000000000000003DD8CCCCCCCCCC08408014AE47E17AFC3F040000000000104000CDCCCCCCCCEC3F9C999999999913408014AE47E17AFC3F9C99999999991340000000000000003D0000000000001440000000000000003D000000000000144000000000000000400400000000001040000000000000F03F100000000000084000000000000000401000000000000840000000000000003D0103000000010000000B000000000000000000143D000000000000003D009E99999999B93F000000000000003D009E99999999B93F8014AE47E17AFC3F400000000000F03F00CDCCCCCCCCEC3FA06666666666FE3F8014AE47E17AFC3FA06666666666FE3F000000000000003D1800000000000040000000000000003D18000000000000400000000000000040400000000000F03F000000000000F03F000000000000143D0000000000000040000000000000143D000000000000003D, 0);Viewing 7 posts - 1 through 6 (of 6 total)
You must be logged in to reply to this topic. Login to reply