Square boxes in file content

[Former Member]

Hi,

We have got a text file containing 'square boxes' in the field 'Item_description'.

The file content has to be passed to BW.

How can we eliminate these wild characters from being passed into BW?

First option is to omit them while reading the file. Here can we use the file encoding option? If yes, please confirm the correct way to do that.

The second option is to eliminate that in mapping. But I wonder how we can identify a 'square box' and then replace them.

Kindly help.

Thanks,

John

[stefan_grube]

First of all you have to verify, what these check boxes are and why they are in the file.

The next thing is asking the sender of the file, if the check boxes can be avoided by sending the file.

At last you need to know, how BI expects the checkboxes to be.

The worst thing that you can do is: Just not knowing anything, but trying to do something in PI to change the message.

You will have a corrupt message which leads to issues in receiver system.

[Former Member]

Hi Stefan,

The situation is better than the worst case you have mentioned.

The sender application team has confirmed that they cannot be unicode compatible and hence cannot avoid sending the junk 'square boxes'.

The objective is clear. We have to remove them before they reach BW, since it can make the process chain fail.

Anyways those characters does not make much sense to any system.

It is not possible to remove it in graphical mapping in XI, since the SAX parser is throwing the error.

Can we remove it in a java mapping, which can be called before the graphical mapping? I am not good with java mapping.

Any help in detail is appreciated.

Thanks,

John

[stefan_grube]

> The sender application team has confirmed that they cannot be unicode compatible and hence cannot avoid sending the junk 'square boxes'.

And what are those square boxes?

If it is just a "non-unicode" issue, you just need to assign the correct code page to the file adapter channel.

[Former Member]

Hi,

I have tried all unicode encoding formats like ISO-8859-1, UTF-8 and all the others as well.

What I need is a java code which can filter only the alpha numeric characters from all the XML field values.

KIndly help me with the code.

If I call this java mapping before Message mapping, I am sure it can be solved.

Can anyone help me with that?

Thanks,

John

[stefan_grube]

> What I need is a java code which can filter only the alpha numeric characters from all the XML field values.

Every character is alphanumeric. So you want to delete the whole payload?

What exactly are those boxes?

Use a hex editor.

[stefan_grube]

> I have tried all unicode encoding formats like ISO-8859-1, UTF-8 and all the others as well.

System integration is not try and error. Why you do not ask your collegues, which encoding the file has?

If you don not know this, you cannot proceed with your task.

[Former Member]

It might be too late, but here is a java code that removes all bad characters from the payload. Should be imported in PI and added to Operation Mapping before the Mesage Mapping:

import java.io.BufferedReader;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.io.Reader;

import com.sap.aii.mapping.api.AbstractTransformation;

import com.sap.aii.mapping.api.StreamTransformationException;

import com.sap.aii.mapping.api.TransformationInput;

import com.sap.aii.mapping.api.TransformationOutput;

public class RemoveBadCharacters extends AbstractTransformation {

public void transform(TransformationInput arg0, TransformationOutput arg1)

throws StreamTransformationException {

getTrace().addInfo("JAVA program RemoveBadCharacters is Initiated");

String inputPayload = convertInputStreamToString(arg0.getInputPayload().getInputStream());

String outputPayload = cleanUp(inputPayload);

try {

arg1.getOutputPayload().getOutputStream().write(

outputPayload.getBytes("UTF-8"));

} catch (Exception exception1) {

}

}

public String convertInputStreamToString(InputStream in) {

StringBuffer sb = new StringBuffer();

try {

InputStreamReader isr = new InputStreamReader(in);

Reader reader = new BufferedReader(isr);

int ch;

while ((ch = in.read()) > -1) {

sb.append((char) ch);

}

reader.close();

} catch (Exception exception) {

}

return sb.toString();

}

public String cleanUp(String instructions){

String res = "";

String c = "";

int hexCode = 0;

if (instructions != null && instructions.length()>0){

for (int i=0; i<instructions.length(); i++){

c = instructions.substring(i, i+1);

hexCode = c.hashCode();

if (hexCode>31 && hexCode<127 && hexCode != 96){

res = res.concat(c);

}

}

}

return res;

}

}

[Former Member]

Hi,

Need to handle the Encoding for this...

find out the encoding or char set used by the system/application which is generating the data to file...

based on that info need to handle appropriate encoding in File adpater ...

But dont proceed with replacement this will cause the data mismatch in target system..and will lead to errors..

HTH

Rajesh

[Former Member]

Hi John,

Ref: http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79f...

SAP Note: 821267

Thanks,

[Former Member]

Hi,

I have gone throught those documents. But is of no help.

The error which I am getting seems to have been triggered in the SAX parser.

We are able to pick the file.

No error in adapter. It has reached integration engine. I can see the XML in the inbound payload in sxi_monitor.

The error description is "Fatal Error: com.sap.engine.lib.xml.parser.ParserException: Invalid char #0x0 (:main:, row:12, col:~)".

Hence we might have to take care of this before the message reaches the parser. So may be only an EJB module might do.

Any alternatives which anyone can think of?

Thanks,

John

[Former Member]

Hi,

looks like mapping got failed...

Did you checked the mapping using the same xml which is visible in SXI_MONITOR...

test the same ...if still fails need to handle the codepage as mentioned above..

HTH

Rajesh