Flat File Destination Throws Error with Japanese Characters

  • I'm extracting data from a table (on sql server) and writing it to a flat file. I'm getting the error below. I have already tried modifying my package to Delay Validation = True and TruncationRowDisposition is set to Ignore. When I saw that this didn't solve my problem, I examined the data finding out that I had Japanese characters in the table (which sql server supports) but it brought my package to a hault. The last row/column in the file is the column prior to the column contain the japanese character. What can I do? My file is Pipe delimited. Is this an issue with SSIS or with the Server that I am writing the file to?

    ERROR:

    SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Flat File Destination" (16) failed with error code 0xC02020A0.

    The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and

    will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.

    Data conversion failed. The data conversion for column "Description" returned status value 4 and status text "Text was truncated or one or

    more characters had no match in the target code page.".

  • Is the data type you're writing to a DT_STR, and the data type in the flow a DT_STR of a different code page, or a DT_WSTR? If so:

    DT_WSTRs are double-byte characters. Each character is actuallly represented by two bytes - and any language's characters can be represented.

    DT_STRs are single-byte characters. Each characted is represented by only one byte - and there aren't enough combinations of bits in there to represent every language's character set. In fact, some languages simply can't be represented at all, because there aren't enough combinations in one byte to represent all the "letters" (characters) in their language - Oriental languages are a prime example. In those cases, DT_STR types sometimes take two bytes to represent one character.

    When a DT_WSTR is cast/converted to a DT_STR, or a DT_STR of one code page is cast/converted to a DT_STR of another code page, a code-page translation has to occur. In your case, the Japanese characters in your flow (likely a DT_WSTR type) need two bytes per character to store in their DT_STR codepage. The net result is that a DT_WSTR of size 4 needs a DT_STR of size 8 to store the information.

    Bottom line: increase the size of your destination column.

    Todd McDermid - SQL Server MVP, MCSD.Net, MCTS (SQL08BI)
    Find great SSIS extensions - most are open source - at SSIS Community Tasks and Components[/url]
    Are you doing Data Warehouse ETL with SSIS? Don't use the slow SCD Wizard, use the Kimball Method SCD component instead.

  • Thanks for the explanation. I've tried different things and here's my last attempt with NO success.

    Also I am doing a Multcast. Would the Multicast have an issue?

    Description nvarchar(200) - this is the column in the sql server table

    OLE DB Source

    External Columns

    Description Unicode string [DT_WSTR], Length 200

    Output Columns

    Description Unicode string [DT_WSTR], Length 400

    Flat file Destination

    External Columns

    Description Unicode string [DT_WSTR], Length 400

    Output Columns

    Description Unicode string [DT_WSTR], Length 400

Viewing 3 posts - 1 through 2 (of 2 total)

You must be logged in to reply to this topic. Login to reply