Read zipped Base64-Stream and convert it to xml

[Former Member]

Hello exprts,

I'm currently working on an interface where I have to read a zip-file from a webservice. This zp files ist base64-encoded and contains an xml-list that I need to populate to the target structure. I got the scenario so far, that it is working in the eclipse enviroment as clean java code. There I can decode the base64-string, uzipp the archive and save the inherit xml-file on my filesystem.

But I don't get it working as java mapping in a udf. I don't know how to convert the (unzipped) bytesream in order to return the xml file to the repsonse structure. Currently I'm only haveing the zip-entry as outputstream:

Does anyone of you has a sugesstion on how to get xml file in the zip-archive as output of the mapping?

regards

Christian

[markangelo_dihiansan · Accepted solution]

Hi Christian,

I'm afraid this cannot be tested at all using test tab in ESR. You need to do an end-to-end testing to confirm if your scenario is working or at least make it in the request message mapping pipeline in sxi_monitor.

As far as I know, for FileOutputStream, you need to set the path to a directory in AL11 e.g

FileOutputStream fout = new FileOutputStream("/tmp/"+ze.getName());

Hope this helps,

Mark

[engswee · Accepted solution]

I think the FileOutputStream should not be used unless the intention is to really write directly to the PI file system, instead of sending the output payload through a File receiver adapter.

The output should be written to the output stream of the mapping, as access to the stream is available with the following logic.

OutputStream os = output.getOutputPayload().getOutputStream();

This would then allow the output payload to be viewed via the standard monitoring tools. Writing directly to the FileOutputStream would bypass everything (no logs, monitoring, etc) - hard to trace, hard to support, hard to debug.

Just my two cents.

Rgds

Eng Swee

[markangelo_dihiansan · Accepted solution]

Yes, I agree with this. It is always better to use OutputStream directly because it is cleaner I am actually using ByteArrayOutputStream e.g

ByteArrayOutputStream baos = new ByteArrayOutputStream();
WritableWorkbook workbook = Workbook.createWorkbook(baos);

and then to write it as an attachment

Attachment attachments = output.getOutputAttachments().create("Test.xls", "application/octet-stream", baos.toByteArray());
output.getOutputAttachments().setAttachment(attachments);

or as a main payload

output.getOutputPayload().getOutputStream().write(baos.toByteArray());

Regards,

Mark

Message was edited by: Mark Dihiansan

[Former Member · Accepted solution]

Hi Mark,

this looks like quite a good solution. But I am haveing a little understanding problem here or just a little twist in my thoughts:

How can I manage to map the content of the response xml ( as it is a whole xml file) to different fields within the mapping? The different nodes and their content of the response xml file need to be mapped to the corresponding fields of the return structure.

regards

Christian

[engswee · Accepted solution]

Christian

Would you be able to share an example of the source and expected target for your response mapping? This will help us to have a better understanding of your requirement.

Rgds

Eng Swee

[markangelo_dihiansan · Accepted solution]

Hi Christian,

How can I manage to map the content of the response xml ( as it is a whole xml file) to different fields within the mapping? The different nodes and their content of the response xml file need to be mapped to the corresponding fields of the return structure.

Not sure if I understood it correctly, but by this you mean that once the zipped file is written as a main payload, you'll still need to map it?

If yes, all you have to do is to create a second mapping e.g zipped file structure to final target message. You can call this second mapping in the operation mapping. Although you can already handle the output of the zipped file in a single java mapping, this is not recommended because it will be hard trace (think about who will support it)

Regards,

Mark

[Former Member · Accepted solution]

Hi Mark,

to make it a little bit clearer I have attached a screenshot of the mapping:

On the left side is the base64 coded zip-file. In the zip-file you can find the xml-file with the structure on the right side. And on this structure the content of the xml-file has to be mapped.

regards

Christian

[engswee · Accepted solution]

Hi Christian

Your requirement is much clearer, now that we can see that the Base64 value is actually embedded in one of the fields in the XML source.

You can try out the following source code for the Java mapping. With this you won't need a graphical mapping with UDF. If you don't have a Java IDE environment to compile the archive, you can try the technique I mentioned earlier on writing the Java mapping directly in ESR. If you do that, just make sure you add the necessary imports.

package com.equalize.xpi.esr.mapping.refactor;


import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import javax.xml.bind.DatatypeConverter;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.NodeList;
import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;


public class Base64DecodeAndUnzip extends AbstractTransformation {


  @Override
  public void transform(TransformationInput input, TransformationOutput output) throws StreamTransformationException {
  try {
  // Get base64 string from DOM input
  InputStream is = input.getInputPayload().getInputStream();
  DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
  DocumentBuilder builder = factory.newDocumentBuilder();
  Document docIn = builder.parse(is);
  NodeList details = docIn.getElementsByTagName("CustomerList");
  String b64 = details.item(0).getFirstChild().getNodeValue();


  // First decode the base 64 string
  byte[] decoded = DatatypeConverter.parseBase64Binary(b64);


  // Next, unzip the file (assumption only 1 entry in zip file)
  ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(decoded));
  ByteArrayOutputStream baos = new ByteArrayOutputStream();
  ZipEntry ze = zis.getNextEntry();
  if (ze != null) {
  byte[] buffer = new byte[1024];
  int read = 0;
  while ((read = zis.read(buffer, 0, buffer.length)) != -1) {
  baos.write(buffer, 0, read);
  }
  baos.flush();
  zis.closeEntry();
  }
  zis.close();


  // Write to output stream
  OutputStream os = output.getOutputPayload().getOutputStream();
  os.write(baos.toByteArray());  
  } catch (Exception e) {
  throw new StreamTransformationException("Exception: " + e.getMessage(), e);  
  }
  }


}

Rgds

Eng Swee

[Former Member · Accepted solution]

Hi Mark,

this is a great answer! I have implemented the java-function and got it to working with in the test enviroment. But now I am getting a strange error that I really don't have a clue about.

When I test the funtion with a testreport I get the following error:

But the payload then is returned as one structure:

What am I missing here? Do I need to transform the message within the mapping or within the response data-type?

regards

Christian

[markangelo_dihiansan · Accepted solution]

Hi Christian,

It was 's answer. It just means that the message type of your proxy and the one of your java mapping did not match. To see what should be the correct structure, just generate a sample instance from your proxy and then compare it with the one your UDF produced. Check for namespaces on the message type or if the field names are correct.

Regards,

Mark

[engswee · Accepted solution]

Hi Christian

Mark's assessment is correct, there is a mismatch between what your proxy expects for response vs. the XML that was embedded in the Base64 string.

If you can identify the mismatch (wrong field name, additional levels in the XML hierarchy, namespaces), then you can have additional logic to cater for this. You can either tweak the above Java mapping logic, or have a subsequent mapping step in the Operation Mapping.

If you need further assistance, please provide the sample XML for:

i) XML output after conversion by Java mapping logic above (either from ESR testing or PI runtime monitor)

ii) XML structure expected by Proxy

Also, what is the response message of the proxy based on - is it using the external definition, or based on PI custom defined message type?

Rgds

Eng Swee

[Former Member · Accepted solution]

Hi experts,

I think I found the solution. The Problem is the missing namespace in the xml-file within the archive. Does anyone of you has any clue how to add the namespace to the file while I write it to the outputsream within the mapping?

regards

Christian Gessner

[Former Member · Accepted solution]

I have the solution by myself. I havbe added a additional xslt-mapping step where I add the required namesace. Now it is working.

Thanks for your input!

[former_member184944 · Accepted solution]

Hello Eng Swee Yeoh,

I have a similar urgetn requirement in my current project , where i have to read zipped base64,  csv file and convert it to xml .

Can you please help me with the java mapping code where after unzipping you get a csv file and then convert the csv file to xml .

Thanks in advance.

[engswee]

Hi Christian

Just to confirm - is your input payload already in the zipped format? If so, how do you convert it to a String to feed to your zipBase64 parameter of your UDF (which is expecting a String input)?

For your scenario, it would probably be better to handle it entirely as a Java mapping. You can either or develop it directly in your Eclipse IDE. You will need to have a class to extend the AbstractTransformation class and override the transform(TransformationInput input, TransformationOutput output) method.

You can get the InputStream by using the following code, followed by your logic to decode/unzip, etc.

InputStream is = input.getInputPayload().getInputStream();

Is the unzipped and decoded content already in XML format that you expect? If yes, then you can just directly write it to the output stream. If not, then you can further parse the content and build the XML output yourself (either using DOM or String processing.)

Regardless of either way you develop, you can still test it in ESR using the technique below which allows you to enter the binary zip file as the source.

Rgds

Eng Swee

[Former Member]

Hi Eng,

yes, it is. To get it clear let me desrcibe the Input: The bytestream is in the return value of the response structure as a tag. I just pass this value to the udf.

Yes, the content of the zip file is already a xml file and just needs to be past to the output field.

Do I need a java mapping or can I can also use a UDF-mapping?

reagrds

Christian

[engswee]

Hi Christian,

Just need to clarify this part, to determine if you need Java mapping or UDF in Message mapping.

The bytestream is in the return value of the response structure as a tag. I just pass this value to the udf.

If your input format is already zipped/encoded, how is it passed as a String to the UDF? I still can't visualize your input. In order to pass a value as a parameter into a UDF, the source structure has to be in XML format.

Is your input an XML file with one of the fields having an embedded base64 value?

Rgds

Eng Swee