Special Character Handling (&) in the payload content in PI 7.1

[Former Member]

Hello All,

My Scenario is Idoc to File. I need to handle some special characters like "&" in XML payload content in PI 7.1. I am receiving & character in the text field.

eg:

<RECORD>

<BEGDA>20100901</BEGDA>

<STAT1>A</STAT1>

<NAME1>Grandview&MO CLMR</NAME1>

<PERL_SUB_AREA>0005</PERL_SUB_AREA>

</RECORD>

I created java mapping by using the blog: http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420. [original link is broken] [original link is broken] [original link is broken]

"The code used in java mapping is converting & into &amp"

Mapping is working fine in the test tab of Operation mapping and the message is successfully handling the record with & character(message with Chequered Flag).In the result tree view,it is showing

<PERL_TEXT>Grandview&MO CLMR</PERL_TEXT

and in the source XML view

<PERL_TEXT>"Grandview&ampMO CLMR"</PERL_TEXT>

Issue:

Now the record is written as

<RECORD>

<CUST_STATUS>A</CUST_STATUS>

<COMPANY>1050</COMPANY>

<PERL_AREA>1MOB</PERL_AREA>

<PERL_TEXT>"Grandview&ampMO CLMR"</PERL_TEXT>

<PERL_SUB_AREA>0005</PERL_SUB_AREA>

</RECORD>

Since & is coming as part of text, "it is converting & -> &amp" in the text field and the record is written in the file as

"A|1050|1MOB|Grandview&ampMO CLMR|0005 ....."

While it should come as:

A|1050|1MOB|Grandview&MO CLMR|0005 .....

Java mapping code used is as:

import java.io.InputStream;

import java.io.OutputStream;

import java.util.Map;

import com.sap.aii.mapping.api.StreamTransformation;

public class HandleAmpersand implements StreamTransformation {

public void setParameter (Map param) { }

public void execute (InputStream in, OutputStream out) {

try {

int read_data;

while ((read_data = in.read() ) != -1) {

if (read_data != '&') {

out.write(read_data);

} else {

out.write("&amp;".getBytes());

}

}

out.flush();

} catch (Exception e) { }

}

}

Also, i am using message mapping after java mapping in operation mapping

Please guide me where it is going wrong.

Thanks

Shikha Jain

Edited by: Jain Shikha on Jan 5, 2011 4:42 AM

[RaghuVamseedhar]

package com.Mapping;

import java.io.InputStream;
import java.io.OutputStream;

import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;

public class Mapping_Java extends AbstractTransformation {
	public void transform(TransformationInput transformationInput,
			TransformationOutput transformationOutput)
			throws StreamTransformationException {
		try {
			InputStream inputstream = transformationInput.getInputPayload()
					.getInputStream();
			OutputStream outputstream = transformationOutput.getOutputPayload()
					.getOutputStream();

			byte[] b = new byte[inputstream.available()];
			inputstream.read(b);
			String strContent = new String(b);
			
			if (strContent.contains("&")) {
				getTrace().addInfo("& is present");
				strContent = strContent.replaceAll("&amp ;", "&");

				strContent = strContent.replaceAll("&quot ;", "\"");
				strContent = strContent.replaceAll("&apos ;", "\'");
				strContent = strContent.replaceAll("&amp ;", "&");
				strContent = strContent.replaceAll("&lt ;", "<");
				strContent = strContent.replaceAll("&gt ;", ">");
			}
			outputstream.write(strContent.getBytes());

		} catch (Exception exception) {
			exception.printStackTrace();
		}
	}
}

Edited by: Raghu Vamsee on Jan 6, 2011 1:59 PM

[Former Member]

Thanks for valuable inputs

[Former Member]

Hello All,

I would like to re-open this issue once again - As I face similar problem but in different scenario:

Scenario:

SOAP -> PI -> RFC (calling BAPI_DOCUMENT_CREATE2)

The soap message contains some special characters in the description fields ex: "DESC <SDH>"

when I am map the request to this RFC - I place the description field inside CDATA Structure and the RFC Request will look like as follows:

  <?xml version="1.0" encoding="UTF-8" ?> 
  <ns0:BAPI_DOCUMENT_CREATE2 xmlns:ns0="urn:sap-com:document:sap:rfc:functions">
  <DOCUMENTDATA>
  <DOCUMENTTYPE>DRW</DOCUMENTTYPE> 
  <DOCUMENTNUMBER>SDH_DOC_005</DOCUMENTNUMBER> 
  <DOCUMENTVERSION>00</DOCUMENTVERSION> 
  <DOCUMENTPART>000</DOCUMENTPART> 
  <STATUSEXTERN>IA</STATUSEXTERN> 
  <VALIDFROMDATE /> 
  </DOCUMENTDATA>
  <DOCUMENTDESCRIPTIONS>
  <item>
  <LANGUAGE>DE</LANGUAGE> 
  <DESCRIPTION><![CDATA[DESC <SDH> 
  
  
  EN 
  DESC <SDH> 
  
  
  
]]>

This request is successfully executed by RFC - but when I go to R/3 and look for the result - in the document description it looks as follow:

DESC & ltSDH& gt
Note: I purposefully inserted the space after & - because this editor does 
not allow me to write together!

Does anyboday have solution to overcome this effect in SAP R/3- do I need any more mapping?

[stefan_grube]

> DESC & ltSDH& gt

Of course it does. What do you expect when you use CDATA?

Open a new thread as this is some totally different as the original post.

[RaghuVamseedhar]

Hi Jain Shikha,

I have faced similar issue in our production system.

Our issue:-

Sender system is sending :- DEKOR " OBS!

Other middleware is converting it and sending to SAP PI as :- DEKOR &amp ;quot; OBS!

Because of this, there is increase in size from 12 characters to 21 characters. We were getting error as the target side ABAP table has column width 15.

Our solution:- Wrote a java mapping to convert &amp;quot; to u201C

if (strContent.contains("&")) {
				getTrace().addInfo("& is present");
				strContent = strContent.replaceAll("&amp ;", "&");

				strContent = strContent.replaceAll("&quot ;", "\"");
				strContent = strContent.replaceAll("&apos ;", "\'");
				strContent = strContent.replaceAll("&amp ;", "&");
				strContent = strContent.replaceAll("&lt ;", "<");
				strContent = strContent.replaceAll("&gt ;", ">");
			}

Note: - quote ("), apostrophe ('), ampersand (&), less than (<), greater than (>) are special characters in XML. They should not be present in data. http://www.w3.org/TR/REC-xml/

Not Well formed XML:

<Name>Raghuu2019s</Name>   <Name> Raghu & Vamsee </Name>

Well formed XML:

<Name>Raghu&apos;s</Name>   <Name>Raghu &amp; Vamsee</Name>

Graphical mapping, XSLT, DOM, SAX need well formed XML as input, otherwise they give parser exception.

Your solution:-

I understand you are using a graphical mapping. Follow below steps

Step 1: Convert u201CNot Well formed XMLu201D to u201CWell formed XMLu201D, using java mapping (which your doing already).

Step 2: Use Graphical mapping for your transformation logic.

Step 3: Again convert back u201CWell formed XMLu201D to u201CNot Well formed XMLu201D (As this what target system is expecting. Use below java mapping.

Regards ,

Raghu

Edited by: Raghu Vamsee on Jan 6, 2011 1:45 PM

Edited by: Raghu Vamsee on Jan 6, 2011 1:59 PM

[stefan_grube]

> My Scenario is Idoc to File. I need to handle some special characters like "&" in XML payload content in PI 7.1. I am receiving & character in the text field.

You need not do it, as IDoc adapter escapes & values automatically.

You are waisting your time.

[Former Member]

Hi Stefan,

I have similar requirement , mine is Proxy to file scenario.Could you please guide me how to handle '&' without convert into '&amp;'

Thanks,

Raju

[stefan_grube]

> Could you please guide me how to handle '&' without convert into '&amp;'

It must be converted to have valid XML.

Receiver adapters reverse the conversion, so you should not have an issue.

[Former Member]

HI Stefen,

Some text is missing in my thread, mine is converting '&' to '& amp;' (NO SPACE IN BETWEEN & and amp; just to represent) but i need to pass '&' to '&' as it is without converting to other strings, becasue while writing final output in file it is writing as &amp which i need to replace this to &

For example my input is <TEXT> SG&A</TEXT> i need to get output as <TEXT>SG&A</TEXT> but not <TEXT> SG& amp;A</TEXT> (NO SPACE IN BETWEEN & and amp; just to represent)

Thanks,

Raju

[stefan_grube]

> becasue while writing final output in file it is writing as &amp which i need to replace this to &

What is your adapter for output? file adapter?

[Former Member]

Hi Stefan,

YES,its file adapter ( file content conversion).Below is sample output.

00401978|C|1020|1GOA|SG&amp ;A|0005|||1|02|

Thanks,

Raju

[stefan_grube]

> 00401978|C|1020|1GOA|SG&amp ;A|0005|||1|02|

This sceanrio works for me perfect.

So you need a patch. Which PI version do you use?

Check out for SAP notes.

[Former Member]

Hi Stefan,

I am using PI 7.1 SPS 4

Thanks,

Raju

[stefan_grube]

> I am using PI 7.1 SPS 4

This is a quite old SPS level. So you might consider a patch.

It works with PI 7.11 SPS 5

Are you sure that you have & amp ; in your payload and not & amp ; amp ; ?

[Former Member]

YES, its & amp; ( no space between & and amp; to represent) in payload after transformation

[stefan_grube]

did you check with Notepad? (In SXMB_MONI right click on message and choose display source)

[Former Member]

Hi Stefan,

In Notepad, by right clicking in MONI, it is coming as

<TEXT> SG& amp; amp;A</TEXT> ( no spaces between amp; and amp; to represent)

Many Thanks for your help.

Thanks,

Raju

[stefan_grube]

So you have to check where this & amp amp comes from.

In your Proxy: Do you send & or & amp ?

Do you have any special mapping?

[Former Member]

Hi Stefan,

-> So you have to check where this & amp amp comes from.

not sure where this extra amp coming from...

-> In your Proxy: Do you send & or & amp ?

i am sending only & from proxy

Do you have any special mapping?

using below two mapping in operational mapping

Java mapping ( to convert & to & amp; used same code as mentioned in the thread http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420. [original link is broken] [original link is broken] [original link is broken])

Message mapping ( its real direct one to one mesage mapping )

Thanks,

Raju

[stefan_grube]

> Java mapping ( to convert & to & amp; used same code as mentioned in the thread http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420. [original link is broken] [original link is broken] [original link is broken])

Remove this. You should never use this Java mapping.

[Former Member]

Hi Stefan,

->So you have to check where this & amp amp comes from.

not sure where this extra amp coming from

In your Proxy: Do you send & or & amp ?

from proxy i send only &

Do you have any special mapping?

in operational mapping i use below mappings.

java mapping ( to handle & and convert into & amp: code used same as suggested in the thread http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420. [original link is broken] [original link is broken] [original link is broken]) whn i test in OM test tab in tree view i am getting only & but in xml view am getting & amp; but when i right click MONI into note pad am getting & amp; amp;

Message mapping ( one to one mapping to convert source to target)

Thanks,

Raju

[Former Member]

Hi Stefan,

The whole idea is to handle '&' in xml in input payload, can you suggest me how to handle without java mapping?

If i dont use java mapping the messages not getting processed, throwing error as xml contains '&'.

Thanks,

Raju

[stefan_grube]

> If i dont use java mapping the messages not getting processed, throwing error as xml contains '&'.

From ABAP proxy or IDoc? I don't believe this.

Check where you get the XML from and fix it in sender system.

[Former Member]

Hi Stefan,

i have the same scenario proxy(Idoc is generated within proxy) to file. Data is extracted from SAP and is converted into Idoc within proxy. & is coming in the field content of text field(like company name e.g S&G Company). "&" cannot be removed and needs to be handled.

Please advise.

Thanks

Shikha Jain

Edited by: Jain Shikha on Jan 5, 2011 4:25 PM

[stefan_grube]

Is there an error with that & ?

[Former Member]

Yes.. It gives an invalid XML error while testing through message mapping and operational mapping test tab ...Nothing comes up in Moni in the Response structure and in file

Thanks

Shikha Jain

[stefan_grube]

> test tab ..

Arrggg!!!!

Please can ANYONE ELSE help on this topic?

[Former Member]

Stefan,

We were not getting any output in the file if & sign is there. Blank file was generated. If we remove & from the text full file is generated.

So java mapping was used to handle & . Now File is generated but (in Notepad, by right clicking in MONI,) it is coming as

<PERL_TEXT>Grandview& amp; amp;MO CLMR</PERL_TEXT> (no space in between & and amp; and amp;)

Not sure from where extra amp; is coming from?

Java code is taken from blog: http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420 [original link is broken] [original link is broken] [original link is broken] as mentioned earlier

Thanks

Shikha Jain

Edited by: Jain Shikha on Jan 5, 2011 6:06 PM

[Former Member]

Hello All,

The issue is resolved(Changed java map). Many thanks for help.

Thanks

Shikha Jain

[Former Member]

Close the thread... If your requirement is fulfilled.

Thanks,

[Former Member]

Dear Shikha,

I know it was an old issue, but if you remember which Java mapping did you change? and what did you change?

I am also having same issue...You reply will be of great help

Thanks.

Sumeet.

[RKothari]

Hello,

You are using :

if (read_data != '&') {
out.write(read_data);
} else { 
out.write("&".getBytes());
}

And blog suggests:

if (read_data != '&') {
out.write(read_data);
} else { 
out.write("&".getBytes());
}

Please verify the code again.