taking long time to process .txt file in java mapping.

[Former Member]

Hi Experts,

I have a scenario where PI will pick the .txt file and send as an attachment to ECC through inbound proxy.

Here we have written a java code which is reading the file line by line. The actual file size is more that 13MB. Could you please suggest me any changes required in the below code to speed up the process in PI.

Java Code:

import java.io.InputStream;

import java.io.OutputStream;

import com.sap.aii.mapping.api.AbstractTransformation;

import com.sap.aii.mapping.api.Attachment;

import com.sap.aii.mapping.api.DynamicConfiguration;

import com.sap.aii.mapping.api.DynamicConfigurationKey;

import com.sap.aii.mapping.api.OutputAttachments;

import com.sap.aii.mapping.api.StreamTransformationException;

import com.sap.aii.mapping.api.TransformationInput;

import com.sap.aii.mapping.api.TransformationOutput;

public class setAttachment extends AbstractTransformation

{

  public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput) throws StreamTransformationException

  {

  InputStream inputstream = transformationInput.getInputPayload().getInputStream();

  OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();

  DynamicConfiguration conf = transformationInput.getDynamicConfiguration();

  DynamicConfigurationKey key = DynamicConfigurationKey.create( "http:/"+"/sap.com/xi/XI/System/File","FileName");

  String filename = "";

  filename = conf.get(key);

  conf.put(key,filename);

  try

  {

  String output = "";

  output = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"+"<ns0:MT_receiver_ECC xmlns:ns0=\"http://www.varian.com/JABIL/inbound\">"+"<FileName>";

  output = output + filename;

  output = output + "</FileName>";

  output = output + "</ns0:MT_receiver_ECC>";

  outputstream.write(output.getBytes("UTF-8"));

  OutputAttachments outputAttachments = transformationOutput.getOutputAttachments();

  byte[] b = new byte[inputstream.available()];

  inputstream.read(b);

  String str = new String(b);

  String out = "";

  boolean text = true;

  while(text)

         {

          if(str.contains("</FileName>"))

          {

          int firstindex = str.indexOf("<FileName>");

                 int lastindex = str.indexOf("</FileName>");

                 out = out + "\n" + str.substring(firstindex+10, lastindex);

                 str = str.substring(lastindex+1);

          }

          else

          text = false;

         }

  byte[] bytes = out.getBytes("UTF-8");

  Attachment newAttachment = outputAttachments.create(filename,"text/plain",bytes);

  outputAttachments.setAttachment(newAttachment);

  }

  catch(Exception e)

  {

  getTrace().addDebugMessage(e.getMessage());

  }

  }

}

Thanks,

Gayathri.

[former_member182412 · Accepted solution]

Hi Gayatri,

I think you created one field called FileName in sender data type and you are doing FCC in SFTP server and again you are reading line by line in the java mapping, don't need any FCC in SFTP adapter and just pass the same input text file to java mapping.

Use below java mapping.

import java.io.InputStream;


import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;


import org.w3c.dom.Document;
import org.w3c.dom.Element;


import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.Attachment;
import com.sap.aii.mapping.api.DynamicConfiguration;
import com.sap.aii.mapping.api.DynamicConfigurationKey;
import com.sap.aii.mapping.api.OutputAttachments;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;


public class AddAttachmentJavaMap extends AbstractTransformation {
  @Override
  public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput)
  throws StreamTransformationException {
  try {
  DynamicConfiguration conf = transformationInput.getDynamicConfiguration();
  DynamicConfigurationKey KEY_FILENAME = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File", "FileName");
  String fileName = conf.get(KEY_FILENAME);


  DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
  factory.setIgnoringElementContentWhitespace(true);
  DocumentBuilder builder = factory.newDocumentBuilder();
  InputStream inputstream = transformationInput.getInputPayload().getInputStream();
  Document outputDoc = builder.newDocument();
  Element outMsgType = outputDoc.createElement("MT_receiver_ECC");
  outMsgType.setAttribute("ns0", "http://www.varian.com/JABIL/inbound");
  outputDoc.appendChild(outMsgType);
  Element fileNameElement = outputDoc.createElement("FileName");
  fileNameElement.setTextContent(fileName);
  outMsgType.appendChild(fileNameElement);
  Transformer transformer = TransformerFactory.newInstance().newTransformer();
  transformer.setOutputProperty("indent", "yes");
  transformer.transform(new DOMSource(outputDoc), new StreamResult(transformationOutput.getOutputPayload()
  .getOutputStream()));


  OutputAttachments outputAttachments = transformationOutput.getOutputAttachments();
  byte[] buf = new byte[inputstream.available()];
  inputstream.read(buf);
  Attachment attachment = outputAttachments.create(fileName, "text/plain", buf);
  outputAttachments.setAttachment(attachment);
  } catch (Exception e) {
  getTrace().addDebugMessage(e.getMessage());
  throw new StreamTransformationException(e.getMessage());
  }
  }
}

Regards,

Praveen.

[Former Member · Accepted solution]

Hi Praveen,

As usual your solution helped me in resolving the issue. And even in the code which I have written earlier also got worked fast when I removed the while loop.

I have one more query in this. I have one more requirement where I need to read the .xml  instead of .txt file. I have written the below code for that but its not working ,I am unable to read the file as an attachment. Please suggest.

Java Code:

import java.io.InputStream;

import javax.xml.parsers.DocumentBuilder;

import javax.xml.parsers.DocumentBuilderFactory;

import javax.xml.transform.Transformer;

import javax.xml.transform.TransformerFactory;

import javax.xml.transform.dom.DOMSource;

import javax.xml.transform.stream.StreamResult;

import org.w3c.dom.Document;

import org.w3c.dom.Element;

import com.sap.aii.mapping.api.AbstractTransformation;

import com.sap.aii.mapping.api.Attachment;

import com.sap.aii.mapping.api.DynamicConfiguration;

import com.sap.aii.mapping.api.DynamicConfigurationKey;

import com.sap.aii.mapping.api.OutputAttachments;

import com.sap.aii.mapping.api.StreamTransformationException;

import com.sap.aii.mapping.api.TransformationInput;

import com.sap.aii.mapping.api.TransformationOutput;

public class setAttachmentNew1 extends AbstractTransformation

{ 

  public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput) throws StreamTransformationException

  {

  try

  {

  InputStream inputstream = transformationInput.getInputPayload().getInputStream();

  DynamicConfiguration conf = transformationInput.getDynamicConfiguration();

  DynamicConfigurationKey key = DynamicConfigurationKey.create( "http:/"+"/sap.com/xi/XI/System/File","FileName");

  String fileName = conf.get(key);

  OutputAttachments outputAttachments = transformationOutput.getOutputAttachments();

  byte[] b = new byte[inputstream.available()];

  inputstream.read(b);

  String attachmentContent = new String(b);

  DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();

  factory.setIgnoringElementContentWhitespace(true);

  DocumentBuilder builder = factory.newDocumentBuilder();

  Document outputDoc = builder.newDocument();

  Element outMsgType = outputDoc.createElement("ns0:MT_GUDIDDATA1_ECC");

  outMsgType.setAttribute("xmlns:ns0", "http://www.varian.com/GUDIDDATA1/Inbound");

  outputDoc.appendChild(outMsgType);

  Element fileNameElement = outputDoc.createElement("FileName");

  fileNameElement.setTextContent(fileName);

  outMsgType.appendChild(fileNameElement);

  Transformer transformer = TransformerFactory.newInstance().newTransformer();

  transformer.setOutputProperty("indent", "yes");

  transformer.transform(new DOMSource(outputDoc), new StreamResult(transformationOutput.getOutputPayload().getOutputStream()));

  byte[] bytes = attachmentContent.getBytes("UTF-8");

  Attachment newAttachment = outputAttachments.create(fileName,"text/xml",bytes);

  outputAttachments.setAttachment(newAttachment);

  }

  catch (Exception e)

  {

  getTrace().addDebugMessage(e.getMessage());

  throw new StreamTransformationException(e.getMessage());

  }

  }

} 

Thanks,

Gayathri.

[Former Member · Accepted solution]

Hi All,

I rectified the code.

Correct statement is below.

Attachment newAttachment = outputAttachments.create(fileName,"Application/xml",bytes);

Thanks,

Gayathri.

[iaki_vila]

Hi Gayathiri,

You should use a parser like the DOM parser for example and you could access to <FileName> content without do a while. For example:

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import org.w3c.dom.NodeList;
import org.w3c.dom.Document;
import org.w3c.dom.Node;

public Document getParser(InputStream in) throws Exception {


   
 /* transforms source to destination object*/


   
 TransformerFactory tf = TransformerFactory.newInstance();


   
 Transformer transform = tf.newTransformer();


   
 /*creates new instance of parser*/


   
 DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();


   
 //ignore whitespace within document


   
 factory.setIgnoringElementContentWhitespace(true);


   
 //parser is aware of namespace


   
 factory.setNamespaceAware(true);


   
 DocumentBuilder builderel = factory.newDocumentBuilder();


   
 /*input document in form of XML*/


   
 Document docIn = builderel.parse(in);


   
 return docIn;



    }

Later in the main code you can get values (docIn is of Document type that you get with the previous method):

   

 
NodeList nlFileName = docIn.getElementsByTagName("FileName");


   

 String fileValue = nlFicherosNombre.item(0).getFirstChild().getNodeValue();;

Also you should thin that with message greater than 4 or 5Mb the PÎ performance decrease considerably.

Hope this helps.

[Former Member]

Thanks for your quick Reply Inaki.

But unfortunately i am not good at java coding can you elaborate more on this.

And PI approximately it is taking 1 hr time to process the file.

Thanks,

Gayathri.

[engswee]

Gayathri

Few things:-

1) PI taking 1 hour to process does not sound right - can you provide a screenshot of the audit/message log that shows the section where the mapping is being executed along with the timestamp

2) Can you explain what you are trying to achieve in the while() loop? It looks like you are trying to extract content that is within the FileName tags.

3) Can you share what is the input file format as well as a sample, and are you performing any FCC if it is a flat file?

Regards

Eng Swee