SFTP Sender Channel To Pick multiple files with out ESR

[vishnu_pallamreddy]

Hi All,

I have requirement where I need to pick the files from SFTP and and places them in local folder.

Here I need to Pick the files like below:

AI_Hdr_23022016 and AI_Item_23022016 i should pick these two files and I need to place these files in 2 seperate folders.

Second one is Archive folder. for this we have option in SFTP channel like Archive files in on PI Server.

First one is, main folder.

And here I need to remove the _23022016 from file and need to place in main folder. if multiple files are there in Source folder we need to add counter, for this also we have option like counter.

How it is possible.?

Source file:

AI_Hdr_23022016.txt and AI_Item_23022016.txt

Receiver file:

In Normal Folder:

AI_Hdr.txt

AI_Item.txt

AI_Hdr2.txt

AI_Ite2.txt

Archive Folder:

AI_Hdr_23022016.txt and AI_Item_23022016.txt

Please help me to get this requirement.

There is no ESR.

Thanks In Advance,

Vishnu.

[former_member182412 · Accepted solution]

Hi Vishnu,

You can do without ESR objects like below:

If you can configure the sender SFTP channel like below channel will pickup both files.

In the receiver NFS channel configure DynamicAttributeChangeBean like below. You need to deploy this bean as per the document already mentioned by Eng.

It will remove the date part of the file name and write it to target directory, add the counter also.

To write the files to archive directory you can use sender SFTP channel option archive files on PI server.

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

My client not interested in Custom Module Beans.

My Sender is SFTP. I didnt map any filed in Mapping and written the code in functions Tab...

to generate receiver file I have written java code in ESR.

I have created message mapping and in functions I have written this code.

but it is not working.

it is not replacing the file name.

public String DynamicFileName(Container container)

{

try {

String filename    = "";

DynamicConfiguration conf1 = (DynamicConfiguration) container

.getTransformationParameters()

.get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);

DynamicConfigurationKey key1 = DynamicConfigurationKey.create( "http:/"+"/sap.com/xi/XI/System/File","FileName");

filename = conf1.get(key1);

filename = filename.replaceAll( "_01012016" ,"" );

filename = filename+".txt";

conf1.put(key1,filename);

return filename;

}

catch(Exception e)

{

     String exception = e.toString();

      return exception;

}

}

Message was edited by: Vishnu Pallamreddy

[former_member182412 · Accepted solution]

Hi Vishnu,

Use below java mapping.

public class RenameFileNameJavaMapping extends AbstractTransformation {
  @Override
  public void transform(TransformationInput transformationInput, TransformationOutput transformationOutput)
  throws StreamTransformationException {
  try {
  InputStream inputStream = transformationInput.getInputPayload().getInputStream();
  OutputStream outputStream = transformationOutput.getOutputPayload().getOutputStream();
  DynamicConfiguration conf = transformationInput.getDynamicConfiguration();
  DynamicConfigurationKey FILE_NAME_KEY = DynamicConfigurationKey
  .create("http://sap.com/xi/XI/System/File", "FileName");
  String fileName = conf.get(FILE_NAME_KEY);
  conf.put(FILE_NAME_KEY, fileName.replaceAll("(.*)_(.*)_(\\d+).txt", "$1_$2.txt"));
  byte[] b = new byte[inputStream.available()];
  inputStream.read(b);
  outputStream.write(b);
  } catch (Exception ex) {
  getTrace().addDebugMessage(ex.getMessage());
  throw new StreamTransformationException(ex.toString());
  }
  }
}

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

I will test and let you know...

[former_member182412 · Accepted solution]

Hi Vishnu,

Another solution if you create two interfaces, one for header and another one for item like below then you no need java mapping.

Sender channel you can use below file name scheme.

Receiver NFS channel file name like below.

You can configure second interface channels similarly like above.

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

Thank You.

Already I have done the configuration like 2 interfaces.

I need a small confirmation here, if Sender SFTP process one file in SFTP folder, then it wont process same file right?

[engswee · Accepted solution]

My suggestion to use the additional files was based on assumption that both Hdr and Item file need to be delivered as a pair (arriving nearly same time) to the target.

If that is not the case, Praveen's suggestion of separate interface for each file is a much cleaner solution.

[former_member182412 · Accepted solution]

Hi Vishnu,

Yes SFTP adapter will delete the file after pick up the file from the directory when you enable checkbox Delete file in processing tab.

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

If I don't select Delete File option, what will happen?

it will pick the same file again?

[vishnu_pallamreddy · Accepted solution]

Hi Yeoh,

Thanks a lot Yeoh.

I have learned Good Ideas with your suggestions.

We are maintaining poll interval sec as 30 in SFTP sender channel.

even though in SFTP server the Hdr and Item files are placing at a time.

So 30 sec is no time difference right?

i will Activate the 2 channels at a time and I f needed maintain ATP also.

Thanks a lot again.

[vishnu_pallamreddy · Accepted solution]

Hi All,

To get AI-Hrd2.txt and AI_Item2.txt I have selected add counter option and mentioned step as 2 and selected after first file.

every day I will get 2 Hdr and 2 Item files in SFTP folder.

I need to pick and drop in main folder and need to archive.. up to now all are successful.

for testing I have processed 3rd file, it generates file in main folder as AI_Hdr6.txt and AI_Item6.txt like this.

From Main folder Hdr and Hdr2.txt files will be picked and deleted by SAP jobs.

So every day only 2 files. But the add counter option placing files like 6 8 10.

how can I achieve like below, it should place like Hdr and Hdr2 only every day.

[former_member182412 · Accepted solution]

Hi Vishnu,

Yes if you don't delete the file the same file will be picked by SFTP adapter.

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

No praveen,

in my case it showing like No New File found 0 files.

[former_member182412 · Accepted solution]

Hi Vishnu,

I just tested my side if i don't enable duplicate checking parameter in the channel then channel is picking up the same file again again.

If i enable the duplicate checking parameter then the channel is identifying the file as duplicate.

Can you paste your channel configuration screen shot of processing tab?

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

Could please help me on counter thing..

Please find the below sender processing tab screen shot:

[former_member182412 · Accepted solution]

Hi Vishnu,

You enable the duplicate checking that is why channel is saying no new new file, if you disable the duplicate checking then channel will pick up the same file.

For add counter in the receiver file name you can configure the receiver channel like below.

Regards,

Praveen.

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

For second file it should show like Hdr2.txt

for 3rd file Hdr3.txt.

if I select Immediately it is placing first file as Hdr1.txt

[vishnu_pallamreddy · Accepted solution]

Hi Praveen,

every job picks the hdr.txt and hdr2.txt and delets them in main folder.

next day we need to place files like hdr.txt and hdr2.txt is it possible.

[engswee]

Hi Vishnu

I don't understand why the insistence of having no ESR objects. Who places this restrictions? Sometimes the insistence of having no ESR object means you can't do any mapping, which leads to more complicated solutions like adapter module or scripts.

Anyway, to answer your queries.

1. Picking up multiple files

You can use the additional files feature in the SFTP channel. Configure one of the file as the main file and the other one as the additional file. For example, if Hdr as main file as Item as additional. You can configure as below so that when it picks up AI_Hdr_23022016.txt, it will look for an additional file with exactly the same name except Item replaces Hdr.

The additional file will then be available as an attachment in the message.

Refer Q4 of the following Wiki for further info on these functionality.

Sender File Adapter Frequently Asked Questions - Process Integration - SCN Wiki

2. Receiver main folder

The target local folder is in PI or ECC. Will you be using NFS as the receiver channel?

If NFS is the receiver channel then it does not support attachment. As such you will need to split the file into two separate interface, with one of them using the PayloadSwapBean to swap the attachment into the main payload.

About the renaming, this is the harder part without ESR. You will need to resort to scripts of adapter module. IMHO, you should just go for message mapping in ESR to change the name using Dynamic Configuration. If you insist on no ESR, you can consider deploying my custom module below and try using regex as shown in Scenario 4.

3. Receiver archive folder

Lastly, I'm guessing you want to archive at the receiver system and not the target system, right? Unfortunately, receiver channels do not have archive feature. If you want to, you can split the message out and use a separate receiver channel to archive the file.

Regards

Eng Swee

[vishnu_pallamreddy]

Hi Yeoh,

1) to pick multiple files from same folder I have done configuration as u suggested.

But I got error in SFTP sender channel.: please see below screen shots for configuration and error.

2) I need to place in PI folder. NFS receiver channel. can i create 2 channels for AI Hdr and AI Item?

can i use your regex or could you please provide some light on payload swapbean.

3) in SFTP sender channel we have option like archive files on PI Server.

how it behaves? it archives the files into PI folder right?

If we use ESR, whether we need to create data type based on filed names or we can create single filed for file name change?

I have one more doubt?

is it possible to pass sender file name dynamically in mapping.

Actually I need to pick the files based on date. some times need to pick today files in other interface need to pick yesterdays file.

if it is possible, could you please provide me the UDF for sender file name.

[engswee]

Whoops.. sorry, to fix the error please change it to

"Hdr"="Item"

I will answer the others in a while.

[engswee]

2) You have two things to handle here

i) Deliver two files - since it's NFS receiver which can't handle attachments, you need two receiver channel. This is because the message created from the SFTP sender channel has Hdr as main document and Item as attachment. To deliver the Item to the second NFS receiver, the channel needs to use PayloadSwapBean to swap the Item attachment into the main document. See the screenshots in Scenario 2 of my article below for an example of using the PayloadSwapBean to swap an attachment.

ii) For the renaming of the files, with ESR mapping you can do the renaming using Dynamic Configuration. Without ESR, you will need scripts or try out my module to see if it fits your scenario.

3) Since the target file system is also on PI, you can do archiving on the sender SFTP side. You can select the option "Archiving Files on File System".

With ESR, you can create dummy DT, MT, SI and use Java Mapping to change the filename in Dynamic Configuration without changing the content.

Lastly, you cannot dynamically choose to pick up only certain dates. The adapter will pick up everything in the folder that matches the pattern.

[maheswarareddykonda]

As Eng swee suggest , if you go with java mapping ...i guess this would help you Dynamic file name for pass-through scenario - Process Integration - SCN Wiki

Hi Eng Swee,

Can you please explain little more about Payload swap bean.

actually i can see your blog about that swap bean...but i could see many time that bean using for different purposes..and i am very curious to know about when exactly we can that bean .

thanks in advance.

[vishnu_pallamreddy]

to file name change, I can create one data type with one filed and need to map root node with UDF right?

to place Item file in receiver folder I need to use payload swap bean right? in this situation I need to use swapbean in sender channel right?

if I develop swap bean module, then no need to create 2nd receiver channel right?

Please confirm me on the below?

is it possible to pass date dynamically To pick files from sender folder?

like if we need to pick todays file then is it possible to pass today date or yesterday date?

[engswee]

Most of the functionality in PI deals with processing of the main document of a message. Sometimes in a PI message we might have additional attachments. If we want to explicitly process these we use the bean to swap it, i.e. replace the main document with the attachment. Internally, I think it does not really replace it, it just change the pointer to which payload is the main document.

There are various cases where we might want to use this, i.e. map the attachment instead of the main payload, transfer the attachment to the file system instead of the main payload.

[engswee]

to file name change, I can create one data type with one filed and need to map root node with UDF right?

Nope. You don't use Message mapping with UDF. Use Java mapping as mentioned by Mahes's Wiki link above. Dummy structure is used so doesn't matter what fields are defined.

to place Item file in receiver folder I need to use payload swap bean right? in this situation I need to use swapbean in sender channel right?

if I develop swap bean module, then no need to create 2nd receiver channel right?

PayloadSwapBean is a standard module. Use it in the 2nd receiver channel.

Adding PayloadSwapBean in the Module Processor -  Adding Modules to the Module Processor - SAP Libr...

Please confirm me on the below?
is it possible to pass date dynamically To pick files from sender folder?
like if we need to pick todays file then is it possible to pass today date or yesterday date?

Static patterns are configured in the SFTP channel. It is not possible to specify dynamic dates.

[vishnu_pallamreddy]

If I create 2nd channel, How the sender recognizes the 2nd receiver

I need to create 2 receiver agreements right?

[engswee]

Yes. Additionally you will need to have two receiver interface in Interface Determination.

[vishnu_pallamreddy]

Hi Yeoh and Maheshwara,

I have copied the code into notepad and saved it as .jar file and compressed it as .zip file and imported in to ESR as a imported archives.

in OM i have selected java class and i am unable to find the imported archive in f4.

I have tried drag and drop it is showing like java class has no program .

please see the below screen shots:

Imported Archive:

[vishnu_pallamreddy]

Hi Yeoh,

If I create 2nd channel, I need to maintain same options in that channel right, like ASMA and file name.

[maheswarareddykonda]

the way u did is wrong,

try like below blog http://scn.sap.com/community/pi-and-soa-middleware/blog/2013/03/13/write-java-mapping-directly-in-es...

or you have to run that code from eclips then you need to import that class

[former_member186851]

Vishnu use exclusion mask option to pick from multiple folders..and for filename you can use asma.

[vishnu_pallamreddy]

Hi Raghu,

There in no exclusion mask in SFTP adapter.

And we need to pick the 2 different files from same folder.

and we need to delete _date from file name.