November 17, 2021 at 7:55 am
Hi,
I have a csv file which we export from Salesforce and flows to downstream sql server. So, the data flow is as follows:
Salesforce --> csv --> Sql server
The issue is that we have some unicode data in Name column ex; "Zuivelcoöperatie FrieslandCampina U.A. - NL" but while exporting to csv it comes as "Zuivelcoöperatie FrieslandCampina U.A. - NL".
Is there a way to get the corrected formatted while importing csv to ssis? I tried changing code page setting to Unicode and sql server column is nvarchar but it is not working.
November 17, 2021 at 8:47 am
If I understand what you wrote, you already get the wrong data in the CSV file. If this is so, then all the modification that you'll do for the import part of you process won't help. You have to check why the export from Salesfoce doesn't handle the names that use Unicode correctly.
Adi
November 17, 2021 at 9:57 am
Sounds like it's encode problem, if I am honest. Perhaps, for example, you're opening the file expecting it to be encoded in UTF-16 and it's actually in ANSI-1252 or UTF-8. If you find out the encode, the data might not actually be malformed.
At a guess, the file is encoded in UTF-8, and you're trying to read it in ANSI-1252. When I take your string, "Zuivelcoöperatie FrieslandCampina U.A. - NL" , and save it in a UTF-8 File, and then reopen said file but specify ANSI-1252 encoding I get the value "Zuivelcoöperatie FrieslandCampina U.A. - NL". Looks familiar.
Thom~
Excuse my typos and sometimes awful grammar. My fingers work faster than my brain does.
Larnu.uk
November 17, 2021 at 11:23 am
Sounds like it's encode problem, if I am honest. Perhaps, for example, you're opening the file expecting it to be encoded in UTF-16 and it's actually in ANSI-1252 or UTF-8. If you find out the encode, the data might not actually be malformed.
At a guess, the file is encoded in UTF-8, and you're trying to read it in ANSI-1252. When I take your string, "Zuivelcoöperatie FrieslandCampina U.A. - NL" , and save it in a UTF-8 File, and then reopen said file but specify ANSI-1252 encoding I get the value "Zuivelcoöperatie FrieslandCampina U.A. - NL". Looks familiar.
Good answer. Sounds like you've had to deal with this stuff before!
The absence of evidence is not evidence of absence.
Martin Rees
You can lead a horse to water, but a pencil must be lead.
Stan Laurel
November 17, 2021 at 11:31 am
Good answer. Sounds like you've had to deal with this stuff before!
Yep, however, the answer for me was to talk to the vendor, who was more than happy to provide the files in ANSI-1252 instead. 🙂
Thom~
Excuse my typos and sometimes awful grammar. My fingers work faster than my brain does.
Larnu.uk
November 17, 2021 at 2:16 pm
You have a very good vendor.
November 17, 2021 at 2:59 pm
You have a very good vendor.
I've, personally, found that most are happy to accommodate if you reach out. I find more that people aren't willing to try asking, but then still blame the vendor as the file isn't the format they want. Don't ask, don't get.
Also, on the flip side, if the vendor is happy to tweak things for you, then you are likely to do the same for them when they need them. I know I've pushed back on a data change for a 3rd party in the past because we'd asked for a couple and they refused. Suddenly they were quite open to getting those changes in for us when I mentioned in a call it was a bug bare of mine (what a coincidence 😉 ).
Thom~
Excuse my typos and sometimes awful grammar. My fingers work faster than my brain does.
Larnu.uk
November 17, 2021 at 9:37 pm
ratbak wrote:You have a very good vendor.
I've, personally, found that most are happy to accommodate if you reach out. I find more that people aren't willing to try asking, but then still blame the vendor as the file isn't the format they want. Don't ask, don't get.
Also, on the flip side, if the vendor is happy to tweak things for you, then you are likely to do the same for them when they need them. I know I've pushed back on a data change for a 3rd party in the past because we'd asked for a couple and they refused. Suddenly they were quite open to getting those changes in for us when I mentioned in a call it was a bug bare of mine (what a coincidence 😉 ).
I've only had one vendor over the years that was impossible. They insisted on the service account for the app being a sysadmin AND being a domain admin. They lost.
The rest, they are usually pretty happy to get some positive feedback. I just had one where I described how they can remove a source of deadlocks by changing an index and tweaking a proc. It took them a while, but they made the change. They were pretty thankful.
Michael L John
If you assassinate a DBA, would you pull a trigger?
To properly post on a forum:
http://www.sqlservercentral.com/articles/61537/
Viewing 8 posts - 1 through 7 (of 7 total)
You must be logged in to reply to this topic. Login to reply