Sender Mail Adapter-Content‐Transfer‐Encoding:Quoted‐Printable Issue

[azharshaikh]

Hi All,

We have Email (using Mail Package XSD) to File Scenario. PI reads Email from Microsoft Exchange Server and writes the HTML content from Content Tag into File Dir which is further processed by Receiving system.

We are able to handle Content Type data for Content-Type: text/html , Content-Type: text/plain, Content-Type: Base64 without any issue.

However we notice (In PI Input Payload from Moni) for few of the Emails that come into PI, there is additional Encoding as : Content-Transfer-Encoding: quoted-printable. Once PI writes the data into HTML for these Emails, the HTML file does not display the data correctly rather there are some unwanted =, =3D, <p/> values appearing in the HTML file which makes it unreadable (sample Screenshot attached)

We tried to use the standard Java APIs in PI UDF to Decode this "quoted-printable" format, but its not solving our issue completely. We still get =, <= /p> (these are Soft Line Breaks that are added as part of quoted-printable encoding) in the HTML file. If we need to remove these unwanted chars from Content, then we would need to read the data Line by Line in PI and apply our logic in UDF- but with this approach there is risk of PI performance.

Please suggest steps / java Code/API to handle & decode : quoted-printable to readable format. And also suggest how much it will impact PI performances (there are about 50+ mails daily in this quoted-printable format).

(The receiving system is not capable to Decode the File and expects PI to place HTML files in readable / exact format as Sent by Sender).

Regards,

Azhar

[RaghuVamseedhar · Accepted solution]

Azhar,

1) Implement below Java mapping before your message mapping.

Java Mapping

package javaapplication3;


import com.sap.aii.mapping.api.*;
import java.io.*;


public class JavaApplication3 extends AbstractTransformation {


    @Override
    public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput) throws StreamTransformationException {
        try {
            InputStream inputstream = transformationInput.getInputPayload().getInputStream();
            inputstream = javax.mail.internet.MimeUtility.decode(inputstream, "quoted-printable");
            OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();


            //Just copy Input file content to Output file content
            byte[] b = new byte[inputstream.available()];
            inputstream.read(b);
            outputstream.write(b);
        } catch (Exception exception) {
            getTrace().addDebugMessage(exception.getMessage());
            throw new StreamTransformationException(exception.toString());
        }
    }
}

OR

1) Please download jaxws

https://jax-ws.java.net/2.2.10/

2) Unzip it. There is mail.jar in jaxws-ri\lib\plugins

3) Import mail.jar into ESR.

4) Use below UDF. FYI, I was not able to point to MailUtilty class using UDF (was able to test in Eclipse).

UDF

try {
    InputStream inputstream = javax.mail.internet.MimeUtility.decode(new ByteArrayInputStream(var1.getBytes()), "quoted-printable");
    byte[] b = new byte[inputstream.available()];
inputstream.read(b);
String s = new String(b);
return s;
} catch (Exception ex) {
    throw new StreamTransformationException(ex.toString());
}

[azharshaikh · Accepted solution]

Hi Raghu,

As suggested in your abv UDF, we are able to decode the Quoted Printable content in PI using: javax.mail.internet.MimeUtility.

Thanks for your inputs

Regards,

Azhar

[RaghuVamseedhar]

Azhar,

Below document has example of "quoted-printable" handling.

[azharshaikh]

Hi Raghu,

Thanks for the document. Very informative & helpful

Regards,

Azhar

[engswee]

Edit: Oops, Raghu got to this before me! 

Hi Azhar

I haven't done this myself but here are my two cents.

You can probably try out the following codec from Apache Commons as it looks pretty well documented.

QuotedPrintableCodec (Apache Commons Codec 1.10 API)

It would be better to use this in a multi-step operation - first step Java mapping to decode the quoted printable content, followed by normal graphical mapping. The Java mapping would have the following logic:

1) Determine if encoding is quoted printable

2) If not, then just perform a 1-1 transfer of input bytes to output bytes

3) If yes, then read all the bytes of input stream, decode the whole byte array contents, and write the decoded content to output stream

Alternatively, you can combine it into a single Message mapping by embedding the Java mapping into the Message Mapping. This will allow you to improve the efficiency a little by skipping step 2. Refer to my comment in the blog below on a similar approach I used:

ESR: Message and Java Mapping's - in a Single Message Mapping!!

I can't comment on the performance of this codec as I haven't tried it myself before, but I think that a one-time decode for the whole input bytes in Java mapping would likely perform better than a line-by-line decode in UDF. You can possibly add log entries before and after the decode to see how much time it takes. You just have to try it out and run some regression to see how it performs. Anyway, 50+ messages of this type a day does not seem that much.

I actually came across a QuotedPrintableInputStream class in SAP's com.sap.aii.af.sdk.xi.util package that implements java.io.FilterInputStream. It might just work, but if it does not it's a bit harder to figure out as there is no supporting JavaDoc for this.

Rgds

Eng Swee

Message was edited by: Eng Swee Yeoh