Special characters issue

[Former Member]

Hi Experts,

I have a scenario were an idoc xml is converted to xml file.I have a interface mapping with 2 mapping progs - 1st the graphical mapping and 2nd a java class.

I have some special characters coming in the file which get converted to a hex values after the java class is called.

I have 2 issues:

1) special char conversion in the java class

2) i have added trace msgs in the java class but i cannot see any msgs in the trace log in moni.

before

<ns1:shipToLine1 xmlns:ns1="http://hnw/Format.wsdl/types/">K LETIu0160TI 1792/1</ns1:shipToLine1>

<ns1:shipToLine2 xmlns:ns1="http://hnw/Format.wsdl/types/">.</ns1:shipToLine2>

<ns1:shipToLine3 xmlns:ns1="http://hnw/Format.wsdl/types/">u0160LAPANICE U BRNA</ns1:shipToLine3>

after :

<ns1:shipToLine1 xmlns:ns1="http://hnw/Format.wsdl/types/">K LETI&#x160;TI 1792/1</ns1:shipToLine1>

<ns1:shipToLine2 xmlns:ns1="http://hnw/Format.wsdl/types/">.</ns1:shipToLine2>

<ns1:shipToLine3 xmlns:ns1="http://hnw/Format.wsdl/types/">&#x160;LAPANICE U BRNA</ns1:shipToLine3>

my java class code is below:

public class carriagereturn implements StreamTransformation

{

private Map param = null;

private AbstractTrace trace = null;

public void setParameter (Map param) {

this.param = param;

if (param == null) {

this.param = new HashMap();

}

}

public void execute(InputStream in, OutputStream out) throws StreamTransformationException

{

try

{

trace.addInfo("Entered execute");

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();

factory.setNamespaceAware(false);

DocumentBuilder builder = factory.newDocumentBuilder();

// change namespace

ByteArrayInputStream byteIn;

byte[] bytes = new byte[in.available()];

in.read(bytes, 0, in.available());

String inStr = new String(bytes);

inStr = inStr.replaceAll("xmlns:ns1", "xmlns:ns0");

inStr = inStr.replaceAll("xmlns:ns2", "xmlns:ns1");

inStr = inStr.replaceAll("<ns1:", "<ns0:");

inStr = inStr.replaceAll("<ns2:", "<ns1:");

inStr = inStr.replaceAll("</ns1:", "</ns0:");

inStr = inStr.replaceAll("</ns2:", "</ns1:");

bytes = inStr.getBytes("UTF-8");

byteIn = new ByteArrayInputStream(bytes);

Document doc = builder.parse(byteIn);

// substitute blank node text with special text

/NodeList nodeList = doc.getElementsByTagName("") ;

for (int i=0; i<nodeList.getLength() ; i++ )

{

Node node = nodeList.item(i);

if ( node.getNodeType() != Node.ELEMENT_NODE || !node.hasChildNodes() )

continue;

if ( node.hasChildNodes() )

{

NodeList nList1 = node.getChildNodes();

for (int j=0; j<nList1.getLength() ; j++)

{

Node subNode = nList1.item(j);

if ( subNode.getNodeType() != Node.TEXT_NODE )

continue;

if ( subNode.getNodeValue().trim().equals(""))

subNode.setNodeValue("###");

}

}

}*/

// add indent

ByteArrayOutputStream byteOut = new ByteArrayOutputStream();

TransformerFactory tFactory = TransformerFactory.newInstance();

Transformer t = tFactory.newTransformer();

t.setOutputProperty(OutputKeys.INDENT, "yes");

t.transform(new DOMSource(doc), new StreamResult(byteOut));

// substitute special text with space

/*String outStr = byteOut.toString();

System.out.println(outStr);

outStr = outStr.replaceAll("xmlns:ns1", "xmlns:ns0");

outStr = outStr.replaceAll("xmlns:ns2", "xmlns:ns1");

outStr = outStr.replaceAll("<ns1:", "<ns0:");

outStr = outStr.replaceAll("<ns2:", "<ns1:");

outStr = outStr.replaceAll("</ns1:", "</ns0:");

outStr = outStr.replaceAll("</ns2:", "</ns1:");

outStr = outStr.replaceAll("###", "");

byte[] bytes = outStr.getBytes();

out.write(bytes);

byte[] bytes = byteOut.toByteArray();

String outStr = new String(bytes);

System.out.println(outStr);*/

out.write(byteOut.toString().getBytes("UTF-8"));

//byteOut.writeTo(out);

}

catch(Exception e)

{

e.printStackTrace();

throw new StreamTransformationException("Mapping Error!", e);

}

}

public static void main(String args[]){

if (args.length != 2)

return;

try{

FileInputStream fileIn = new FileInputStream(args[0]);

InputStream strIn = (InputStream) fileIn;

FileOutputStream fileOut = new FileOutputStream(args[1]);

carriagereturn app = new carriagereturn();

app.execute(strIn, (OutputStream)fileOut);

fileOut.flush();

}catch(Exception e)

{

e.printStackTrace();

}

}

}

please suggest some solution

[Former Member · Accepted solution]

Hi Pragati,

Your code was truncated so I could not see it all, but try this and see if it helps.

String inStr = new String(bytes, "UTF-8"); 

Thanks,

-Russ

[Former Member · Accepted solution]

Hi Russell,

Thanks for the reply,below is my remaining code:

bytes = inStr.getBytes();

byteIn = new ByteArrayInputStream(bytes);

Document doc = builder.parse(byteIn);

ByteArrayOutputStream byteOut = new ByteArrayOutputStream();

TransformerFactory tFactory = TransformerFactory.newInstance();

Transformer t = tFactory.newTransformer();

t.setOutputProperty(OutputKeys.INDENT, "yes");

t.transform(new DOMSource(doc), new StreamResult(byteOut));

out.write(byteOut.toString().getBytes("UTF-8"));

I have added ur suggestion to the code.With the current code the word 'K LETIu0160TI 1792/1' to 'K LETI?TI 1792/1' .

whats wrong now the hex value have disappeared.

[Former Member · Accepted solution]

Hi Pragati,

I'm not sure exactly what you need so there might be a better way to do what you need to do. Please try the following and see if your character comes through ok:

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); 
factory.setNamespaceAware(false);			
DocumentBuilder builder = factory.newDocumentBuilder(); 
TransformerFactory tf = TransformerFactory.newInstance();
Transformer t = tf.newTransformer();
 			
byte[] bytes = new byte[in.available()]; 
in.read(bytes, 0, in.available()); 
String inStr = new String(bytes, "UTF-8"); 
inStr = inStr.replaceAll("xmlns:ns1", "xmlns:ns0"); 
inStr = inStr.replaceAll("xmlns:ns2", "xmlns:ns1"); 

Document doc =  builder.parse(new InputSource(new StringReader(inStr))); 
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(out, "UTF-8"));

t.transform(new DOMSource(doc), new StreamResult(writer));

Thanks,

-Russ

[Former Member · Accepted solution]

Thanks Buddy,

Thanks for the direction,I simply did

  byte[] bytesArray = inStr.getBytes("UTF-8");
out.write(bytesArray);
  

Here are your points for guiding me.

[Former Member · Accepted solution]

Hi Russell,

i tried your code but it doesn't seen to work and the previous code which I wrote gives me the output as a string whereas I require it in xml format.

Now I modified my code to include theencoding property at transformation:

byte[] bytes = new byte [in.available()]; 
in.read(bytes, 0, in.available()); 
String inStr = new String(bytes, "UTF-8");
inStr = inStr.replaceAll("</ns2:", "</ns1:");
bytes = inStr.getBytes();
byteIn = new ByteArrayInputStream(bytes);
Document doc = builder.parse(byteIn);
ByteArrayOutputStream byteOut = new ByteArrayOutputStream();
TransformerFactory tFactory = TransformerFactory.newInstance();
Transformer t = tFactory.newTransformer();
t.setOutputProperty(OutputKeys.INDENT, "yes");
t.setOutputProperty(OutputKeys.ENCODING,"UTF-8");
t.transform(new DOMSource(doc), new StreamResult(byteOut));
byteOut.writeTo(out);

I get the output in xml format but the character doesn't come out correctly,it comes as ? mark:

<ns1:shipToLine1 xmlns:ns1="http://hnw/Format.wsdl/types/">K LETI?TI 1792/1</ns1:shipToLine1>

<ns1:shipToLine2 xmlns:ns1="http://hnw/Format.wsdl/types/">.</ns1:shipToLine2>

<ns1:shipToLine3 xmlns:ns1="http://hnw/Format.wsdl/types/">?LAPANICE U BRNA</ns1:shipToLine3>

Can u suggest whats wrong now.

Edited by: Pragati Divekar on Oct 20, 2009 10:44 AM

Edited by: Pragati Divekar on Oct 20, 2009 10:44 AM

[Former Member · Accepted solution]

Hi Pragati,

If you were using something like this, it should work:

byte[] bytes = new byte[in.available()];
in.read(bytes, 0, in.available());
String inStr  = new String(bytes, "UTF-8");
inStr  = inStr.replaceAll("sometext", "othertext"); 
out.write(inStr.getBytes("UTF-8"));

If you need to run it through the DOM parser for some reason, then the code I showed before worked for me. To fix your example I think you can change

 bytes = inStr.getBytes(); 

to this

 bytes = inStr.getBytes("UTF-8");

If you have trouble with the output after this change then try removing the t.setOutputProperty statements.

Thanks,

-Russ

[Former Member · Accepted solution]

Hi Russell,

I have tried all the options given by you.

The 1st one does work but the problem with that is the output comes as one string and I need it to be in a document format.

The other options give me output as

<ns1:shipToLine1 xmlns:ns1="http://hnw/Format.wsdl/types/">K LETIu0160TI 1792/1</ns1:shipToLine1>
<ns1:shipToLine2 xmlns:ns1="http://hnw/Format.wsdl/types/">.</ns1:shipToLine2>
<ns1:shipToLine3 xmlns:ns1="http://hnw/Format.wsdl/types/">u0160LAPANICE U BRNA</ns1:shipToLine3>

I have even tried your code but still the output is as above.Really wondering whats wrong with the code and struggling to get it working.

If you don't mind can you share your code , I want to verify it with mine.

A strange thing the characters u0160 appear so right in browser whereas if I open the xml payloa din notepad it is a hex value and posted as hex value in file ..Really don't know whats wrong

Thanks,

Pragati

Edited by: Pragati Divekar on Oct 21, 2009 10:33 PM

Edited by: Pragati Divekar on Oct 21, 2009 10:58 PM