Picking bulk files

[former_member194011]

Hi Sdners,

i have an requirement of picking 500 files.....nd i dont want to pick all those 500 files at a time....

so i have to pick 100 files in my first poll and in second poll i have to pick another 100 files like this i have to pick all 500 files

in 5 consecutive polls....and it should not repeat any file....

so experts plzz let me know how to achieve this requirement....!!!!

and also plz tell me all possible ways

waiting for your response.....!!!

thanx in advance

[Former Member · Accepted solution]

To achieve this requirement you have to use script which will first move 'n' number of files from one directory to another directory and from there file adapter will pick these 'n' number of files.

The logic of script will be something like this..

1. Script should first check the target directory and move the files only if it is empty.

2. Script should move 'n' number of files and delete the moved files from source directory so that they should not picked up again.

Apart from this your file adapter should archieve the files so that it should not process them again.

[former_member194011 · Accepted solution]

thanx Sarvesh,

is this the only possible way.....??

why should be the target directory empty.....if suppose after first poll i have placed 100 files then for the 2nd poll target directory will not be empty the 1st polls 100 files will be there....then in such case what about the script will it pick another 100files and place those files in the target directory if the target folder is not empty....??

and by the way i dont want to delete the files from my source directory.

[Former Member · Accepted solution]

> is this the only possible way.....??

No, you can also go for your adapter module developement.

> why should be the target directory empty.....

Let me explain... There will be 3 dir in total.. e.g.

First dir is source dir 'ABC' which will contain your 500 files. Second dir 'Target_temp', in this dir the files will come from 'ABC' dir by script. And your Final destination dir "XYZ" where adapter will finally move the files in chunk of 100.

So as per my previous reply, the 'Target_temp' dir should be empty.

I have just given an idea, you can apply your own logic as per your exact need.

> and by the way i dont want to delete the files from my source directory.

For this you can re-name the files in 'ABC' dir itself by using the script after each move of 100 files.

[former_member194011 · Accepted solution]

@Sarvesh

k...now i got it....

cant v do it with only 2directories source and target....i have heard that there is some option like "no of records per mssg"

can v do some thing with it...??

thnx for u r replies

[Former Member · Accepted solution]

> cant v do it with only 2directories source and target....

If your files names are same with timestamp e.g. TestFile_20100531.09:11:443 then in this case you can live with 2 directories by re-naming the files after each move. But if you have different file names (in this case you will be using . in your file adapter) then you have to use 3 directories.

>i have heard that there is some option like "no of records per mssg"

This technique works if you have a very big file and then you can split the content in chunks. Let's say you have 50000 records then you can specify 10000 records per msg so that the same file will be processed in 5 chunks.

This will not help in your case since you want to pick multiple files.

[former_member194011 · Accepted solution]

@Sarvesh

thank you very much Sarvesh for sharing u r valuable idea's with me....!!

this thread is closed go the answer

!

[former_member194011 · Accepted solution]

No, you can also go for your adapter module developement.

can u also please let me know how to do this using adapter module development

[Former Member · Accepted solution]

> can u also please let me know how to do this using adapter module development

For developing adapter module you can search on sdn, there are lots of good documents on this.

The logic to pick 'n' number of files, that you have to write by your own inside your module.

[former_member194011 · Accepted solution]

k thnx