Error converting characters into server's character set

[former_member197006]

Getting error message 2402 "'Error converting characters into server's character set. Some character(s) could not be converted."  Here are the principles involved:

Server          Charset                    Version

PDB             HP ROMAN8            ASE 15.5

RS               HP ROMAN8            RS 15.6

RDB             ISO_1                      ASE 15.7

Error when transaction arrives at RDB and DSI goes down.  However, I am able to BCP the data out of the PDB and into the RDB without error.

Are there any settings that will allow conversion?  I have experimented with 'dsi_charset_convert' but to no avail.

[former_member197006]

Yes, all characters loaded successfully.

BCP command used to extract row from PDB:

bcp tempdb..tmp out tmp.bcp -c -Jroman8 -Usa -SPDB -ebcp.err

BCP command used to insert row into RDB:

bcp tempdb..tmp in tmp.bcp -c -SRDB -Usa -ebcp.err

I tried all settings of dsi_charset_convert (on, off, allow).  And suspend/resumed connection each time with no change in error message.

Yes, we have to skip the transaction and manually inserted the row.

Your help is appreciated.

[Mark_A_Parsons]

I've run a few different tests but I either get the same error (2402) as you, or I'm able to replicate a problematic roman8 character as '?' (not what I want for one particular character in my test - see below).

Have you tried opening a ticket with tech support?

------------------------------

I've run some tests in my environment but I either get the same error (2402) or a problematic character comes through as a '?'.

RS: RS 15.7.1 SP 303

     instance RS100 : iso_1

     instalnce RS103 : roman8

ASE: ASE 15.7 SP 136

     instance ASE100 : iso_1

     instalnce ASE103 : roman8

test table: charset_test (id int, string varchar(30))

Through some trial and error I tracked down a character that generates a 2402 when replicated from roman8 to iso_1:

     character ascii code: 0xa9

     character : looks like a small letter 'r' with a long top bar (can't get to display correctly in this post)

sp_helpsort shows the character (ascii 0xa9) does exist in both roman8 and iso_1

-----

First set of tests:

ASE103 (roman8) -> RS103 (roman8) -> ASE100 (iso_1)

PDB: insert charset_test values (5,char(0xa9))

DSI to RDB: insert into dbo.charset_test (id, string) values (5, u&'\02cb')

     * DSI goes down with error 2402

I tried all 3 different settings for dsi_charset_convert but got the same results as you ... DSI goes down with error 2402.

I've verified that if I bcp the row out of roman8 and into iso_1 (out / -Jroman8 ; in / no '-J' clause), that the data is loaded without a problem, and the following shows the same character displayed in both the PDB and RDB when using clients with roman8/iso_1 as appropriate:

select *,convert(varbinary,string) from charset_test

NOTE: I also tried running RS103 with the 'iso_1' and 'utf8' charsets but with no luck, ie, the DSI still went down with error 2402.

-----

Second set of tests:

A lof the discussions re: character set conversion in the RS manuals make reference to what happens in the PRS and RRS, and while I'd expect the same behavior with a single RS (ie, PRS == RRS), I went ahead and setup the following test just to see if there's a difference:

ASE103 (roman8) -> RS103 (roman8) -> RS100 (iso_1) -> ASE100 (iso_1)

PDB: insert charset_test values (5,char(0xa9))

DSI: insert into dbo.charset_test (id, string) values (5, '?')

     * DSI stays up; no messages in either repserver errorlog about charset conversion issues

So while the DSI doesn't go down, unfortunately the RDB table ends up with the '?' character instead of char(0xa9) (small 'r' with long top bar).

[former_member197006]

Wow!  Thanks for testing in various scenarios. No, haven't opened a case with TS yet.  But glad to see you were able to repro.

In your last setup (RS to RS), the reference manual indicates that the 'dsi_charset_convert' setting is applicable in that situation.  Did you try changing that?  I believe the default is 'on'.

[Mark_A_Parsons]

Knew I was forgetting something ...

With the dual repserver setup:

ASE103 (roman8) -> RS103 (roman8) -> RS100 (iso_1) -> ASE100 (iso_1)

I tried the following tests:

dsi_charset_convert = 'on' => character comes through as a '?' (0x3f)

dsi_charset_convert = 'off' => character comes through as a 'r' with long top bar (0xa9)

dsi_charset_convert = 'allow' => character comes through as a '?' (0x3f)

So, dual repserver setup, each repserver's charset matching the dataserver it services, and dsi_charset_convert = 'off' appears to replicate my particular problematic character (0xa9) ok.

-----

Eyeballing the roman8 and iso_1 character sets I'm not finding any actual roman8 characters that don't exist in the iso_1 charset so can't test my setup with any roman8 characters that should come through as '?'.  [There are 2 characters in iso_1 that don't show up in roman8 : 0xa0 and oxff.]

[former_member197006]

Thanks Mark.  Good to know that works.  However, looks like I need to contact support.  BTW I identified the character that I'm having issue with as ASCII decimal 170 (looks like a raised lowercase a with an underscore).

And thanks again for all the testing.  Will let you know what they say.