JMS Adapter Error

[Former Member]

Hi Experts,

We are havingone problem in Sender JMS Adapter. From Sender (SeeBeyond) xml files are getting posted into Remote Queue in MQ, then it has been sent to Local Queue of MQ, and then finally from Local MQ Queue PI is picking up the data and sending it to SNC system.

The problem lies in PI itself. While coming into PI, the value in the incoming payload where special characters(say, swedish character) are coming, are coming into PI as a garbage value. Whereas, in Sender (SeeBeyond), MQ and in PI Character set is ISO 8859_1. We have stopped the Sender JMS channel in PI, and checked the payload in MQ and in SeeBeyond, in both of the cases it's fine, only when coming into PI, data are getting corrupted.

For this we have done some investigation like below,

1) We had changed the CCSID from 00819 (ISO 8859_1) to 1208 (UTF-8), 1200 (UTF 16), but no luck.

2) We had added th below mentioned modules into Sender JMS in Processing Sequence with the existing modules,

AF_Modules/MessageTransformBean Local Enterprise Bean ContentType

(ii) AF_Modules/TextCodepageConversionBean Local Enterprise Bean codepageConversion

But yet we are getting the same error in encoding in the payload while coming into PI.

Can anybody help me out with it, or probably let me know if I am missing something in my investigation.

//BR

Soumya

[stefan_grube · Accepted solution]

When it is an XML file, do not apply a CCSID or set the adapter module parameter to binary, so there is no conversion. As the encoding is part of the XML header, the XML file will be processed correctly.

Regards

Stefan

[Former Member · Accepted solution]

Hi Stefan,

We have applied as per your comment, like, removing CCSID or putting the adapter module parameter to binary, but still we are having the same encoding issue.

Is there something I am missing or any more suggestion on this Stefan.

//BR

Soumya

[stefan_grube · Accepted solution]

Does your XML have an encoding header?

[Former Member · Accepted solution]

Hi Stefan,

Yes stefan we are having the following encoding already in incoming message. Here is some part of the xml being sent from remote system:-

//BR

Soumya

Edited by: Soumya Brata Das on Jul 2, 2009 1:43 PM

[stefan_grube · Accepted solution]

Hi Soumya,

I have tested the same in my system. The result is: When I put 819 as CCISD, the result is correct.

So check, if the entry in the MQ is correct. When you put a test message to the MQ manually, the encoding is changed to UTF-8, so check the hex values in the queue, if they are correct.

Regards

Stefan

[Former Member · Accepted solution]

Hi Stefan,

I have tried your option. Here is the result. I have told Remote System (SeeBeyond) to send some XML messages into MQ, whereas we have stopped our Sender Communication Channel of JMS Adapter as it won't pick up any data. The problem here in MQ is that, the files are coming into MQ as Text Mode, as a result, the Hex value of the special character is getting corrupted. For example, whereas the Hex value for the character " ä " should be " E4 ", but here in MQ it's showing as " 2C ", which is incorrect.

I am unsure whether MQ can use Binary mode in this case or not. But I ccould see the data is processing correctly in Unix platform in MQ, I can see the special character as it has been sent from Remote System (SeeBeyond), that means the character is displaying correctly there, whereas in the Hex code, it's showing something else in Queue Manager as it's in Text Mode.

In that case, what do you suggest Stefan?

//BR

Soumya

[stefan_grube · Accepted solution]

Hi Soumya,

that is really an odd value. An ä is utf-8 c3 a4, so I have no idea, where the 2c comes from.

I do not know anything about seebeyond, but it seems that the character are translated wrong when it comes to MQ.

Maybe you try another ccsid, where the ä fits the 2c? But I do not know which ccsid might fit here.

Regards

Stefan

[Former Member · Accepted solution]

Hi Stefan,

I have tried with different CCSID in Sender JMS Adapter, but no luck as if now. I got a reply from SeeBeyond system. You can assume SeeBeyond as a separate system which sends and receives XMLs. So, what SeeBeyond guy is saying, he has tested by sending a test message of having the same special character and receives the same character from MQ just for testing. So, he concluded that there is no problem in MQ, since he has received the same character as he is sending to MQ. But while it's coming into PI, we are having issue.

In that case, what do you suggest Stefan, how to move forward.

//BR

Soumya

[stefan_grube · Accepted solution]

The question is not, whether seebeyond can receive the same message as it has sended.

Ask your collegue, if this is a proper ISO 8859-1 encoding, or what else.

You should ask an IBM MQ expert for deciding, what hex values are correct and which CCSID should be used.

Regards

Stefan

[Former Member · Accepted solution]

Hi Stefan,

Sorry for the late reply. We were testing in different cases in PI, and here what I have found. In Sender JMS adapter we had put the module "AF_Modules/MessageTransformBean" with Module Key "Transform". And the configuration of this Module Key is as follows:-

Transform Transform.ContentType text/plain;charset="UTF-8"

And we changed the CCSID to 1208 (UTF-8). Now when we saw the payload in SXI_MONITOR in PI, we have found that the special character is being distorted whereas in View Sourse the payload is fine including special character also. That means there might be some display problem in SXI_MONITOR.

Now, the Sender sends the data in XML iin ISO 8859-1nto MQ, MQ send the same as a string in PI, from PI it's going to SCM system (R/3, Supply Chain Management), where we could see ABAP.APPLICATION Error. We tested the interface in SCM system by running SPROXY and found that it's not able to recognise ISO 8859-1, since it's a Non-Unicode System, it will only understan UTF-8 and hence the error. Now manually we have changed the XML Header into UTF-8 and tested at SCM side, and this goes fine.

So, we could see the changing of XML Header from ISO 8859-1 to UTF-8 would be running fine at receiver end (SCM). Since SCM is a Non-Unicode System, the XML header cannot be changed to UTF-8, is there any possibility in PI that we can change the XML Header from ISO 8859-1 to UTF-8.

For your information, we are using Sender JMS adapter to receive the File from Sender and we are using Receiver XI adapter to post the file into SCM system.

What's your suggestion Stefan, how can we change the XML Header from ISO 8859-1 to UTF-8 in PI?

Many Thanks

Soumya

[stefan_grube · Accepted solution]

> What's your suggestion Stefan, how can we change the XML Header from ISO 8859-1 to UTF-8 in PI?

You can use the XMLAnonymizerbean. This changes the encoding and the XML header:

/people/stefan.grube/blog/2007/02/02/remove-namespace-prefix-or-change-xml-encoding-with-the-xmlanonymizerbean

[Former Member · Accepted solution]

Hi Stefan,

We have tried the option specified in your blog. We have added the below module:-

AF_Modules/XMLAnonymizerBean Local Enterprise Bean 0

And then in the Module Configuration we have used,

anonymizer.encoding UTF-8.

Now what is happenning, the character "Gävle" it self is getting converted into "Gävle" in PI which is incorrect. And hence failing in R/3 side.

The data has to be converted into UTF-8 at PI end including the Header of the XML, but here the Header is getting changed into UTF-8, whereas the value is also getting distorted into the above.

Whats your suggestion on this Stefan?

//BR

Soumya

[stefan_grube · Accepted solution]

When you use XMLAnonymizerBean, do not use MessageTransformBean or TextCodepageConversionBean.

[Former Member · Accepted solution]

Hi Stefan,

We have tried Anonymizer Bean, but it seems it's converting the special character into some other unrecognizable format and hence again the issue.

What I am thinking whether there is any way to change the XML Header without using Anonymizer Bean. Can you suggest anything?

//BR

Soumya

[stefan_grube · Accepted solution]

OK, that means the XML header and the encoding are different.

That is difficult.

You could simply remove the XML header with following XSLT:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="xml" omit-xml-declaration="yes"/> 

 <!--  copy payload   -->
 
  <xsl:template match="*">
  <xsl:copy-of select="." /> 
  </xsl:template>
  </xsl:stylesheet>

When the XML has no header, it is considered as UTF-8

[Former Member · Accepted solution]

Hi Stefan,

Thanks a lot for your tips. Finally we got the solution with the help of your tips. What we did is, In Sender JMS Adapter, we have put the CCSID as 00819, that is for ISO 8859-1, since the data is coming into PI as ISO 8859-1 only. And we had put text/xml;charset="UTF-8" as in the Parameter Value for the module configuration of MessageTransformBean. This accepts the XML as ISO 8859-1 and converts the content into UTF-8, but still the XML Header is ISO 8859-1, which is unrecognizable in Receiver, since Receiver is a Non-Unicode System.

For this, we have used JAVA Mapping which contains a JAVA Class which is truncating the XML Header. Now the data is getting posted into Receiver end with proper encoding, that is UTF-8 with the special character proper in place.

All the credits go to you Stefan :).. I have no more words to thank you, your suggestion really helped me a lot to do the proper configuration as per the requirement. Thanks a lot for your coordination and help so far. You can expect full score for that.

My query got answered, and hence closing this thread.

//BR

Soumya

[stefan_grube · Accepted solution]

> In Sender JMS Adapter, we have put the CCSID as 00819, that is for ISO 8859-1, since the data is coming into PI as ISO 8859-1 only. And we had put text/xml;charset="UTF-8" as in the Parameter Value for the module configuration of MessageTransformBean. This accepts the XML as ISO 8859-1 and converts the content into UTF-8, but still the XML Header is ISO 8859-1, which is unrecognizable in Receiver, since Receiver is a Non-Unicode System.

At that point the XMLAnonymizerBean should convert the XML into UTF-8 and change the XML header.

I do not understand, why this does not work for you.

But anyway, when you have a solution, that is fine.

Regards

Stefan

[Former Member · Accepted solution]

Hi Stefan,

Yeah, you are right, that solution should work as per your suggestion. But I think perhaps it is not working for me due to the service pack upgrade from SP13 to SP14. I don't know really whether this would impact in the JMS driver or in any of the pre-defined settings which made the Sender JMS Adapter to function in a lunatic manner. Whereas I had checked the same configuration with some of my colleagues in different Project, and it was working for them. I am not sure whether service pack upgrade would make any impact on this or not. Anyway the problem got resolved I am happy with that. Thanks again Stefan

//BR

Soumya

[Former Member]

Hi Soumya,

Sender (SeeBeyond) xml files are getting posted into Remote Queue in MQ, then it has been sent to Local Queue of MQ, and then finally from Local MQ Queue PI is picking up the data and sending it to SNC system

Since the message is being processed from Remote Queue to local queue, the encoding settings(CCSID) for the local queue manager is not the same as the queue manager for the remote queue. So check the CCSID at the queue manager level and maintain the same.

Regards,

Ramkiran

[Former Member]

Hi Ramkiran,

I have checked with the help of my MQ guys. The CCSID of the Queue Manager in Remote and Local Queue is 00819, which is nothing but ISO 8859_1. In that case, we can say for all the systems, SeeBeyond, MQ, PI are of ISO 8859_1 encoded.

//BR

Soumya

Edited by: Soumya Brata Das on Jul 2, 2009 8:24 AM