Simplified Chinese format problem in File-File Scenario.

[Former Member]

Hi,

I have configured File(FTP) - > File(FTP) Scenario.

I am expecting a Chinese File from Source system which is AS400. I am using the file encoding format UTF-8 while the chinese characters are in GB2312 format. My sender file adapter picks up the file correctly, but SXMB_MONI gives Mapping related Parser error. Kindly Suggest.

Do i need to change some character set or code page? if yes, where should it be done?

Thanks Charu.

[Former Member]

Thanks Stefan,

The file was corrupt.

It worked fine.

Kind Regards,

Charu

[stefan_grube]

Good to hear.

I learned a lot with this question

I put my learnings in this how-to guide:

https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/com.sap.km.cm.docs/library/uuid/502991a2-45d9...

I hope, this is helpful.

Regards

Stefan

[bhavesh_kantilal]

Hi,

In the sender file adapter change the file encoding Scheme.

Make the file type as text and then make the encoding as ,

US-ASCII or ISO-8859-1.

Let us know if this does the trick.

Also check the other encodingh schemes in thsi link,

http://help.sap.com/saphelp_nw2004s/helpdata/en/e3/94007075cae04f930cc4c034e411e1/content.htm

Regards

Bhavesh

[Shabarish_Nair]

i guess maybe even providing the encoding as GB2312 should work !!! Just give it a try and let us know.

Ref: http://en.wikipedia.org/wiki/GB2312_80

[Former Member]

Thanks for your quick reply.

I tried with all the encoding formats including GB2312.

But it still gives the same error.

Regards,

Charu.

[Former Member]

HI,

See

http://en.wikipedia.org/wiki/Chinese_character_encoding

http://en.wikipedia.org/wiki/Chinese_input_methods_for_computers

Regards

Chilla

[Former Member]

The Character set used by AS400(Source system) is 935. It supports GB2312 and GB18030. I tried with both but getting the same error.

I have declared the file transfer mode as text.

For both Sender and Receiver communication channel i m specifying same encoding format.

Regards,

Charu

[stefan_grube]

When your file is an xml file, then the encoding declaration in the XML header has to be set correclty:

<?xml version="1.0" encoding="GB2312"?>

See note <a href="https://service.sap.com/sap/support/notes/821267">821267</a>, question 16.

Regards

Stefan

[stefan_grube]

For a plain text file it works with:

File Type: text

File Encoding: GB2312

Make sure, that the operating system provides the routines to transfer GB2312 to UTF-8

Regards

Stefan

[Former Member]

Thanks Stefan,

How to check whether the operating system provides the routine to transfer GB2312 to UTF-8?

I checked ControlPanel -> regional Languages -> Advanced -> Simplified chinese format is enabled.

-Charu

[stefan_grube]

So your XI runs on Windows? In case that should work anyway. Is your file a plain text or XML file?

Stefan

[Former Member]

My file in a plain text file with Chinese Characters.

It still gives the same error.

Thanks,

Charu

[stefan_grube]

What is the effect, when you apply the File Encoding: GB2312,

are characters transfered wrong, or aren't they transfered at all?

[stefan_grube]

Maybe you could send me the file to stefan dot sdn at gmail dot com

Regards

Stefan

[stefan_grube]

The files that you provided are not chinese, the might be japanese (JIS).

I am not sure what codepage is behind, but it is definitively not GB2312.

To verify, that you have a japanese text, open the file with Word. Word will ask for the correct codepage. When you walk through all codepages, you will see that JIS is the only one, that provides a correct text.

Make sure, what language you really have and what codepage it should be and apply this codepage in the file adapter.

Regards

Stefan

[Former Member]

Hi Stefan,

You are right, I noticed it, but the problem is that when these files are opened on AS400(FTP) server, they show Chinese characters with codepage GB2312.

But when I download the file from FTP server to my local machine, I am not able to see the Chinese character. Also in SXMB_MONI, the inbound xml shows Chinese characters, but i continue to get the parser error from Mapping.

Regards,

Charu

[stefan_grube]

Yeah, the files shows casually some chinese characters. But it is definitively not GB2312.

When you opne the file with a hex editor, you see the foreign characters like this:

0E 24 36 5D 5B B1 DC AF 30 5B B9 A8 3B 5D D4 A8 43 A9 3F 0F

0E and 0F are ASCII signs for shift-in and shift-out. These signs cause the trouble in mapping.

Then you have a sequence that looks like:

$6][&#21348;&#33082;&#28809;0[&#40575;&#31699;;]&#33069;&#31699;C&#28431;? (GB2312)

This is to random to be real chinese. Could you check, if the file is corrupted by the FTP? Compare the files with a hex editor.

Regards

Stefan

[stefan_grube]

> when these files are opened on AS400(FTP) server,

> they show Chinese characters with codepage GB2312.

Are you sure, that this is real chinese, and not random text?

Are you supporting a chinese company?

Regards

Stefan

[Former Member]

Actually in AS400 they are having CCSID which is 935. This CCSID is compatible with GB2312.

Customer has given this file to me. He has downloaded that file in outbound library of FTP Server. Through my scenario I am accessing this file.

I dont have any idea about this file.As per customer it is correct file.

I think AS400 is able to generate the chinese file but FTP server is having some problem while accessing the file. Do we need to specify any attribute in configuration of FTP server?

Warm Regards,

Charu

[stefan_grube]

I don't think that the FTP server causes the trouble. The CCSID 935 is a double codepage. The 0E and 0F switch the codepage.

http://www-03.ibm.com/servers/eserver/iseries/software/globalization/ccsid/00935.htm

When it is say, that the codepage is compatible to GB2312, that only means, that the signs are the same, but not that the hex code of the signs is the same.

I think you have to apply the conversion routines from CCSID 935 to UTF-8.

Maybe you could ask, if you could receive the file in GB2312 or in UTF-8.

I cannot help you here further, because I have not worked with AS400 so far and I do not know how to install the conversion routines on the XI server.

Regards

Stefan

[Former Member]

Hi,

Under File Encoding, specify a code page.

The default setting is to use the system code page that is specific to the configuration of the installed operating system. The file content is converted to the UTF-8 code page before it is sent.

Permitted values for the code page are the existing charsets of the Java runtime. According to the SUN specification for the Java runtime, at least the following standard character sets must be supported:

US-ASCII

Seven-bit ASCII, also known as ISO646-US, or Basic Latin block of the Unicode character set

ISO-8859-1

ISO character set for Western European languages (Latin Alphabet No. 1), also known as ISO-LATIN-1

UTF-8

8-Bit Unicode Character Format

UTF-16BE

16-bit Unicode character format, big-endian byte order

UTF-16LE

16-bit Unicode character format, little-endian byte order

UTF-16

16-bit Unicode character format, byte order

Check which other character sets are supported in the documentation for your Java runtime implementation.

XML text documents generally contain their own code page description and should be treated as binary data type.

If it is not there the you need write the Adapter module for sender side .

Or you can use the XSLT mapping using CDATA.

see the below link

http://help.sap.com/saphelp_nw2004s/helpdata/en/e3/94007075cae04f930cc4c034e411e1/content.htm

Regards

Chilla