Pickup the file based on the date in file name

[Former Member]

Dear all,

I have requirement like this.

EX:

I have file name called ABC_20110721.txt , i have to pickup that file today and should not delete it and should not mark as 'Read only'. We have to keep that file as is in FTP.

I will have another file tomorrow like ABC_20110722.txt , i have to pickup this file alone.

So that means date in the file name is dynamic based on the system date( current date interface runs on )

Please let me know how can we do this?

Regards

[anupam_ghosh2]

Hi ,

You have not specified your operating system on which PI is running. You need scripts to achieve this for you. In case you are using UNIX/Solaris/Linux server the following script "filename.sh" should run fine

lines=`expr 0 + 0`
maxlines=`expr 0 + 100`
while [ 1 -le 5 ] ; do
	d=`date +%G%m%d`
	count=`ls -1 | grep $d |wc -l`
	c=`expr $count + 0`
	while [ $c -gt 0 ]; do
		file=`ls -1 | grep $d |tail -$c |head -1`
		searchFile=`grep $file temp | head -1 | wc -l`
		searchFile=`expr $searchFile + 0`
		if [  $searchFile -eq 0 ]; then
			cp $file /home/anghosh0/sdn/f
			echo $file >>temp
			lines=`expr $lines + 1`
		fi 
		c=`expr $c - 1`
	done	
	if [ $lines -gt $maxlines ]; then 
	  	k=`date +%G%m`
		grep $k temp > t
		mv -f t temp
		lines=`expr 0 + 0`
	fi	
done

Say you are poutting the source files say ABC_20110721.txt,ABC_20110722.txt,ABC_20110725.txt in a directory. Say the name of the directory is "/usr/home/dx7" . You have to put this scipt "filename.sh" in "usr/home/dx7". Set the permission bits of this script to "777" or you can use command "chmod 777 filename.sh". This script will choose only the files for todays date as per server time. This script is running in an infinite loop. Thus whenever there is a new file in the directory it checks the file name, and if the name contains todays date, it puts the file in "home/anghosh0/sdn/f" directory from where PI server picks it up for reading. Please change the folder name according to your convinience. Now in the folder "/usr/home/dx7" you need to keep a file called "temp" this file will conatin only one word or any symbol say "hi". The use of temp is to collect all file names that the script has already transferred to PI folder. Thus if script has transferred file say "ABC_20110725.txt " and "DEF_20110725.txt ", then "temp" will have contents

hi
ABC_20110725.txt 
DEF_20110725.txt 

This prevents same file being put to PI source directory twice. Once temp reaches a maximum of 100 lines, this script clears the contents of temp automatically, except for the lines which belong to current year and current month. You can alter this value by altering the line "maxlines=`expr 0 + 100`" in the script shown above.

This is because the file grows in length over time.Please take help of you BASIS team to put this file in correct folder and run it. It is working fine in our Linux server. Please run this as a background process with this command "filename.sh &".

I think this will solve your problem. In case your operating system is windows then Please let me know I need to write batch file for the same.

regards

Anupam

Edited by: anupamsap on Jul 26, 2011 8:18 AM

[former_member601528]

Hi Anupam,

We have same requirement, we need to Pick the inbound files(Sender) based on Date.

1. Process only Yesterday's Date

2. Process for all Dates...

Point 2, we can be achieved by file name scheme in Source directory.

But how to achieve point 1?

we are using WINDOWS NT OS..in our SAP Landscape. kindly suggest the solution and share the script for Windows NT OS.

Regards,

Aravind

[Former Member]

Hi Bhavana,

There is only one way to achieve this requirement.

You need to write OS command script on the FTP to archive the files that does not match to the current date. Thats is the only feasible option.

Even if write a custom adapter module, everytime while polling it will check all the files and process the current date file.

Thanks,

[Former Member]

As the filename is common in all the day files only date will get change and the file is read only.

How many days of file will be lying in the FTP server. If only one month files are present then we can think of below approach

Comm channel will pick all the files and the custom module will check for system date will the filename date.

We can achieve this by using the Custom Module.

In the module we need to get the today system date and read the sender filename and once we have both the date, compare both the dates that means if file date and today date matches process the file other wise throw an error.

In case if alerts are configured and dates are not matched then make the file content to empty.

[Former Member]

Hi Bhavana,

I guess there is no standard option available to achieve this other than developing a module.

Please refer this blog about duplicate file check - /people/sandeep.jaiswal/blog/2008/05/13/adapter-module-to-stop-processing-of-duplicate-file-ftp-location

You need to modify (remove the code that will make the file empty and exception part) it according to your requirement. But the sender channel will keep picking the old files everyday and the adapter module restrict them from being sent to IE.

The easy and cleaner approach is to have archive folder. Please talk to the ftp admin and try to find alternatives.

[Former Member]

If you are looking forward to implement adapter module method, you can consider the following option which is simple compared to module development.

Select the ASMA and use an udf to retrive filename and compare the current date part.

The receiver businness system/service has to be determined in mapping.

Use the standard receivers message type (available at http://sap.com/xi/XI/System , SAP BASIS) and populate a non exist receiver for old files.

Use the enhanced receiver determination and select the mapping that determines receiver service. Also select 'end processing without errors' so that you don't see any errors in PI for old files and only today's file will be sent to target system.

But as I said, you still need to use 'Test' mode and should be willing to process old files everyday (Uneccessary load on system).

[former_member303666]

hi bhavana again,

when we placed the file in sender folder ,at the time we do not give file permissions to that folder....the sender communication will read the file and send it to the forther process, once it is prossed ...we have to stop the sender communication channel...if sender communiction c in active....it will pick the file again and again...generate duplicate files.....

regards,

kesava

[baskar_gopalakrishnan2]

>I will have another file tomorrow like ABC_20110722.txt , i have to pickup this file alone.

>Please let me know how can we do this?

One way of doing is ..

Assume your first string prefix ABC going to be same always then you might want to expore using variable substitution to create the filename using prefix string and todays date. So that at runtime the file adapter uses this filename to pickup.

>should not delete it and should not mark as 'Read only'. We have to keep that file as is in FTP.

Set Processing Mode to Test (SAP says not recommended for production env)

[Former Member]

Baskar,

>>Set Processing Mode to Test (SAP says not recommended for production env)

I too agree this.

One way of doing is ..

Assume your first string prefix ABC going to be same always then you might want to expore using variable substitution to create the filename using prefix string and todays date. So that at runtime the file adapter uses this filename to pickup.

Do we have Variable Substitution in Sender File adapter?? I don't see that here

Regards

[baskar_gopalakrishnan2]

Other possible ways are

a) Write a script to archive all the files except the current date file in the folder. Call this script in the RUN OS command before processing field in the file adapter. Reference filename in the file adapter as ABC*

b) Since this file processing takes place once per day. After processing move the processed files to archive folder. So that you dont need to maintain script to run in the adapter too.

[Former Member]

chk this:

[former_member303666]

hi bhavana,

hear we have one option ....have 2 set the archive mode...once pick the sender communication that file will be deleated in adopter eng level....we can get back the file form archiving folder(with time stamp)

regards,

kesava.