April 11, 2012 at 12:59 pm
Herez a jpeg attachment.Its a screenshot of downgrade of unicode. Instead of ?? that we usually get, the 7th and 8th column has some gibberish for conversion of some japanese/chinese characters. I wonder if anybody has any idea what kind of algorithm can be used to get those jibbersish instead of the question marks. My client is Ok with that kind of gibberish. Thanks.
April 11, 2012 at 2:06 pm
What shold the question marks be do you know? should they be chinese characters? if so on the columns where the data is ? what is the datatype of these columns? for unicode they need to be NVARCHAR.
***The first step is always the hardest *******
April 11, 2012 at 2:43 pm
SGT_squeequal (4/11/2012)
What shold the question marks be do you know? should they be chinese characters? if so on the columns where the data is ? what is the datatype of these columns? for unicode they need to be NVARCHAR.
when you convert unicode to non-unicode (ansi/ascii), the unicode foreieng characters corrupt into ???. But instead of that happening, in my attached jpg, there are gibberish characters when the conversion took place. All i know is those gibberish letters are residing as varchar. My questions remains, if anybody has any clue as to how those gibberish can be produced instead of the question marks.
April 11, 2012 at 2:50 pm
Let me get this straight... Instead of meaningless ?'s, your client wants other meaningless characters? Clearly, the conversion is finding some conversion for the characters in columns 7 and 8, but not for the others. If it is all meaningless anyway, why not just replace ? with something else? I guess I don't understand the difference between ? and "gibberish."
Jared
CE - Microsoft
April 11, 2012 at 3:15 pm
SQLKnowItAll (4/11/2012)
Let me get this straight... Instead of meaningless ?'s, your client wants other meaningless characters? Clearly, the conversion is finding some conversion for the characters in columns 7 and 8, but not for the others. If it is all meaningless anyway, why not just replace ? with something else? I guess I don't understand the difference between ? and "gibberish."
Exactly SQLKnowItAll. That was my first opinion. But my client's stand is that those gibberish actually is a format and they have custom functions (outside of sql i think) to bring it back to unicode.And so thats how they want the data to be. I have a few varchar columns with these gibberish characters and the problem is I have no idea how to figure out what algorithm/function or whatever was used to get them. How do you get gibberish characters in sql, all i get is question marks. May be i can dig in from there.
April 11, 2012 at 3:22 pm
Murphy'sLaw (4/11/2012)
SQLKnowItAll (4/11/2012)
Let me get this straight... Instead of meaningless ?'s, your client wants other meaningless characters? Clearly, the conversion is finding some conversion for the characters in columns 7 and 8, but not for the others. If it is all meaningless anyway, why not just replace ? with something else? I guess I don't understand the difference between ? and "gibberish."Exactly SQLKnowItAll. That was my first opinion. But my client's stand is that those gibberish actually is a format and they have custom functions (outside of sql i think) to bring it back to unicode.And so thats how they want the data to be. I have a few varchar columns with these gibberish characters and the problem is I have no idea how to figure out what algorithm/function or whatever was used to get them. How do you get gibberish characters in sql, all i get is question marks. May be i can dig in from there.
Unless you plan on writing the translation code yourself, I don't think you can do what the customer wants. If they can convert the gibberish back, why not just keep the data as unicode data and not worry about it?
April 11, 2012 at 3:26 pm
Lynn Pettis (4/11/2012)
Murphy'sLaw (4/11/2012)
SQLKnowItAll (4/11/2012)
Let me get this straight... Instead of meaningless ?'s, your client wants other meaningless characters? Clearly, the conversion is finding some conversion for the characters in columns 7 and 8, but not for the others. If it is all meaningless anyway, why not just replace ? with something else? I guess I don't understand the difference between ? and "gibberish."Exactly SQLKnowItAll. That was my first opinion. But my client's stand is that those gibberish actually is a format and they have custom functions (outside of sql i think) to bring it back to unicode.And so thats how they want the data to be. I have a few varchar columns with these gibberish characters and the problem is I have no idea how to figure out what algorithm/function or whatever was used to get them. How do you get gibberish characters in sql, all i get is question marks. May be i can dig in from there.
Unless you plan on writing the translation code yourself, I don't think you can do what the customer wants. If they can convert the gibberish back, why not just keep the data as unicode data and not worry about it?
+1 Clearly, there is some conversion going on for those "gibberish" and none for the others. So why even change it? Why translate an article from Hungarian to English only to translate it back to Hungarian?
Jared
CE - Microsoft
April 11, 2012 at 3:34 pm
SQLKnowItAll (4/11/2012)
Lynn Pettis (4/11/2012)
Murphy'sLaw (4/11/2012)
SQLKnowItAll (4/11/2012)
Let me get this straight... Instead of meaningless ?'s, your client wants other meaningless characters? Clearly, the conversion is finding some conversion for the characters in columns 7 and 8, but not for the others. If it is all meaningless anyway, why not just replace ? with something else? I guess I don't understand the difference between ? and "gibberish."Exactly SQLKnowItAll. That was my first opinion. But my client's stand is that those gibberish actually is a format and they have custom functions (outside of sql i think) to bring it back to unicode.And so thats how they want the data to be. I have a few varchar columns with these gibberish characters and the problem is I have no idea how to figure out what algorithm/function or whatever was used to get them. How do you get gibberish characters in sql, all i get is question marks. May be i can dig in from there.
Unless you plan on writing the translation code yourself, I don't think you can do what the customer wants. If they can convert the gibberish back, why not just keep the data as unicode data and not worry about it?
+1 Clearly, there is some conversion going on for those "gibberish" and none for the others. So why even change it? Why translate an article from Hungarian to English only to translate it back to Hungarian?
I understand what you mean. I also suggested them to retain the columns with strange looking characters as it is and just convert the rest of columns(100's of them') so there is not data loss. The data flows from different system back and forth(SAP-SQL-SAP--??) hence the whole conversion. However this is customer requirement any my job is to deliver to the best of possibility.
April 11, 2012 at 4:39 pm
As it many times is 🙂 I'm out of guesses...
Jared
CE - Microsoft
April 11, 2012 at 4:50 pm
SQLKnowItAll (4/11/2012)
As it many times is 🙂 I'm out of guesses...
Thanks as always.
I wonder if there is a way to force sql server to restrict the ANSI conversion values between 127 and 255. That way we wouldn't get the question mark whose ascii value is 63. Just trying anything here. 🙂
Viewing 10 posts - 1 through 9 (of 9 total)
You must be logged in to reply to this topic. Login to reply