Encoding Issue in SFTP Sender Adapter (AnonymizerBean)

[Former Member]

Hi All,

We are having an encoding requirement .For this requirement when we use File Adapter and use the encoding ISO-8859-1 ,it fixes the issue but when we try to use it in SFTP Adapter using the bean XMLAnonymizerBean,we are not able to convert it to required format.Could anyone please help us in  fixing the issue.

Example

In SFTP Adapter:   TÓMB on using ISO-8859-2  gets converted to T?MB but it should be same as input i.e  TÓMB                                     

Thanks

Dhileep

[anand_shankar10 · Accepted solution]

Hi Dhileep.

Refer to the link: http://docs.oracle.com/javase/1.5.0/docs/guide/intl/encoding.doc.html

Hope it will help you to identify the correct UNICODE>

Regards

Anand

[Former Member · Accepted solution]

HI Anand,

its helpful link, but i din`t get the expected output.

In Processing Sequence , i have added localejbs/AF_Modules/TextCodepageConversionBean.

And in parameters tab,added Conversion.charset --- ISO-8895-2.

I am getting TÓMB as T�MB,please suggest me.

Thanks

Dhileep

[anand_shankar10 · Accepted solution]

why not try once by using the UNICODE in Processing TAB of SFTP adapter and see the result... In case you are not on PI 7.3x

In the SFTP adapter provided by Advantco we had an option of UNICODE in processing tab

Thanks

Anand

Message was edited by: Anand Shankar

[Former Member · Accepted solution]

Hi ,

We are working on PI 7.3 currently, please suggest me into this.

Thanks

Dhileep

[markangelo_dihiansan · Accepted solution]

Hello,

Why are you using encoding ISO-8859-2 when in the FTP Adapter it worked with the ISO-8859-1 encoding?

Regards,

Mark

[Former Member · Accepted solution]

Hi Mark,

In Business requirement,Sender System is using SFTP adapter, we can`t go for FTP adapter.

plz help me in encoding issue by using SFTP Sender Adapter.

Thanks

Dhileep

[markangelo_dihiansan · Accepted solution]

Hello Dhileep,

Whatt is preventing you from using ISO-8859-1? In the thread, you mentioned that you are using ISO-8859-2

Regards,

Mark

[Former Member · Accepted solution]

Hi Mark,

I tried with ISO_8859-1 and ISO_8859-2 at sender SFTP adapter ,but dint see any change in the encoding.But with FTP adapter i am getting as expected.

Example

In SFTP Adapter:   TÓMB on using ISO-8859-2  gets converted to T?MB but it should be same as input i.e  TÓMB

In FTP Adapter:   TÓMB on using ISO-8859-2  gets converted to TÓMB.

Thanks

Dhileep

[markangelo_dihiansan · Accepted solution]

Hello,

Can you show us a screenshot of your module configuration?

Regards,

Mark

[Former Member · Accepted solution]

Hi Mark,

Please let me know if changes required in module configuration or missed anything.

I also tried AnonymizerBean but not getting expected output.

Processing Sequence:

localejbs/AF_Modules/XMLAnonymizerBean

Module Configuration :

anonymizer.acceptNamespaces   urn:ABC.com:XXXX ns

anonymizer.encoding                   ISO-8859-2

Thanks,

Dhileep

[markangelo_dihiansan · Accepted solution]

Hello,

Can you try removing the TextCodepageConversionBean and then add the Transform.ContentType of the MessageTransformBean under Plain2XML?

Rehards,

Mark

[Former Member · Accepted solution]

HI Mark,

I have done changes but not getting correct output.

TÓMB is converting as T�MB.

Module configurations are : 

By using utf-8,getting the same T�MB.Please suggest me to solve it

Thanks

Dhileep

[Former Member · Accepted solution]

Hi,

ISO-8859-2 is supported in JDK 1.1.6. Possibly due to this reason you are not getting the desired result.

Please refer the below suggested link:

Charsets (Java Servlet Programming)

Regards,

Souvik

[former_member519052]

Hello.

This blog was very helpful to me

I want to share one adapter receiver who believes in a PI 7.3

which in the end you should create a .CSV file with the SFTP addon occupying modules.

I hope as a reference.

PI_share

OmAR.

[rafael_trbswasser]

Hi.

The problem with the SFTP-Adapter is that you can't tell it which source encoding a text file has and it reads the file always in binary mode. If you have plain text files which are not UTF-8 encoded you have to convert it to UTF-8 first which can be archieved the following way:

1. First you have to set the source encoding of the text file. Set the Content-Type and Charset of the XI Payload via the MessageTransformBean:
   1. Transform.ContentType = text/plain;charset=iso-8859-2 (=>the source encoding of the text file)
2. Then you have to convert the text file into UTF-8 via the TextCodepageConversionBean(This bean only works in conjunction with plain text files which you indicate via the Content-Type !) . Now you set the target encoding for the text file:
   1. Conversion.charset = utf-8

Now you have a text file loaded into PI via the SFTP-Adapter which is really UTF-8 encoded and from there you can work as usual.

This problem could have been solved via a custom adapter module as well but this would have caused more work to do.

Of course you could also proceed after Step1 with further processing without converting the Textfile to utf-8 since the source encoding is correctly set in the Content-Type of the message which allows following modules or mappings to correctly convert the charset as needed.

Kr,

Rafael

[gabor_hornyak]

Hi All,

I ve run in the same issue.

I ve tried to use text codepage conversion bean and anonymizer bean as well.

They did the conversion but not for the original characters.

In the original I have öüáé characters.

They were picked up using the same hexa code (ef bf bd):

Dél-Dunántúl -> Dďż˝l-Dunďż˝ntďż˝l

Afterwards I can apply whatever encoding logic (anonymizer, text codepage conversion, xslt etc), they dont have the original characters, they will convert always that same hexa char for every special char.

My guess is that MessageTransformBean is doing that replacement of the special chars- although I have specified the proper charset in ContentType.

Any idea, suggestion?

[rafael_trbswasser]

Hi. What is your scenario? You have to be aware of the encoding of the source file, since PI is expecting UTF-8 for all character based functions (hence it ist based on Java which is only working with Unicode characters). So special characters are NOT indicating the encoding of a file. If it is already UTF-8 the codepage conversion is rather doing harm to the characters.

[gabor_hornyak]

Hi Rafael,

in my scenario I am picking up a csv file with iso-8859-2 encoded content.

We have also third party advantco sftp adapter, where it is working (and my guess would be that it is also java based) -  there we dont have to use modules, there is an encoding field.

Now we are trying to move to sap sftp adapter (it has no extra charge) but it lacks a lot of feature which is present in sap file or third party sftp channel. I am looking for workarounds - like using modules but I am a bit disappointed. Its a great step back compared to the file adapter and I hope that sap will fill that gap in the near future.

[rafael_trbswasser]

Hi.

This is no real problem since you have those 2 modules. If you do exactly as I mentioned in my original post it should work without Problems. Please double check the module parameter names and the order of the modules.

[gabor_hornyak]

Hi Rafael,

Here are the configs with anonymizer and text code page conversion

none of them worked - in the output of

msg transform bean we already lost the special chars as mentioned in my earlier post

btw I have 50 versions of that channels, with other encodings and module orders, you wont find any combination that I didnt try (combining iso-8859-2 and utf-8 in every way)

maybe on the screenshots you can see sthing that I ve missed

[rafael_trbswasser]

There it is. You've got the order of the modules wrong.

1. Module should be MessageTransformBean setting ONLY the Transform.ContentType!!!

2. Module is TextCodepageConversionBean to convert the text Content to UTF-8.

3. Module is MessageTransformBean AGAIN with transformation parameters!!!

This is due to the fact, as I mentioned, that all string operations are based on UTF-8. And you already do some string operations in the first MessageTransformBean which can not work since the content is not transformed to UTF-8 before!

I hope this finally help you resolving your issue.

[gabor_hornyak]

Well Rafael, all my respect is yours.

Based on your first screenshot I didnt get the point but with your explanation it is clear and works.

many thanks

For those who are visual type, here is the config that did the job for me:

[former_member201275]

Hi Rafael,

I saw your post and realize it was some years ago, but hope you are still around to help me.

The module keys that you have created i.e. 'trans' 'con' and 'exit' should these exist in the SFTP Adapter Metadata on ESB?

I have in the SFTP adapter Metadata display the 'exit' module with the 'localejbs/ModuleProcessorExitBean' listed there as well, but cannot find the other modules or the module names beginning with 'AF_Module*'. Is this because I am on version 7.1, or can I simply enter these in my Receiver Channel and they will work?

Thank you,

Glen  

[Former Member]

Hi Rafael,

thanks a lot for your Instructions!

That helped me alot for processing cyrillic textfiles with SFTP Adapter. (plain2Xml)

Thanks again,

Best regards,
Fabian

[Former Member]

For reasons of completeness

SFTP Receiver Channel with SimpleXML2Plain file conversion and ISO-8859-1 content conversion:

[markangelo_dihiansan]

Hello,

We are having an encoding requirement .For this requirement when we use File Adapter and use the encoding ISO-8859-1 ,it fixes the issue but when we try to use it in SFTP Adapter using the bean XMLAnonymizerBean,we are not able to convert it to required format.Could anyone please help us in  fixing the issue. 

If your payload is not XML but a text document, you can try using TextCodepageConversionBean instead

http://help.sap.com/saphelp_nwpi711/helpdata/en/57/0b2c4142aef623e10000000a155106/frameset.htm

Hope this helps,

Mark