UNIX OS Commands to Append multiple files before getting picked by Sender File channel

[Former Member]

Hello! All

I am trying execute OS Commands in Sender file channel so that:

1. All Files placed on source directory get appended in to 1 file.

2. PI then moves only this one file.

3. Sender system places 5 files on the source directory eg: File1, File2, File3, File4, File5

4. Need to execute UNIX Shell command to combile/APPEND all 5 files in to 1 file eg: FileALL -- to be executed from the Sender  File execute OS command (Pre-processing of the file).

5. PI Sender File adapter channel then moves this 1 file -- FileALL to the target directory.

Please, do let me know if anyone has used OS level (UNIX commands) to append all files in to 1 file.

Your help is greatly appreciated!!

Thank you,

Ritu

[Former Member · Accepted solution]

Hi Ritu,

There are many examples of this on the internet, try search term "Unix concatenate files"

One example of the "cat" unix command:

http://superuser.com/questions/228878/how-can-i-concatenate-two-files-in-unix

This will append 2 files (file1.txt & file2.txt) to a new file "new.txt"

You might want to also build some intelligence into the script to move the original source files (as they appear before they are appended) to another directory. Just so that the old files are not appended (and duplicated) again the next time the channel polls the directory, only new files will be there.

Regards, Trevor

[Former Member · Accepted solution]

Hi Ritu,

You may also build a scenario in PI to append files to some temporary folder and then pick up the file from temporary folder to target directory.

Regards,

Beena.

[Former Member · Accepted solution]

Thank you! Everyone for your responses.

Trevor,

The CAT command would work fine, However trying to find a way to clear the old files after the new file is created. Like you mentioned above - any recommendation on how this could be achieved in the same command / script execution.

Thanks again,

Ritu

[baskar_gopalakrishnan2 · Accepted solution]

cat file1 file2 file3 file4 file5 > FileALL

rm file1 file2 file3 file4 file5

That's it

Please refert this link for your script

http://www-users.cs.umn.edu/~skim/cs1901/tutorial.html

[Former Member · Accepted solution]

Thank you! Bhaskar

I was trying to make a few more changes and wanted to seek help with this regard:

1. The Files can grow from 5 to 10 - hence would like to make the script generic. Si there a way to use the a wild character to handle multiple files instead of just 5.

2. We would need to have the rm command too changed accordingly.

3. Can we add a timestamps to the appended file something like : FileAll_DDMMYYYY-HH-MM-SS

Also, tried to use the link you had mentioned was not opening - so I am trying to find tutorial with the above info as well. IF you happen to solution to the above - that will be of great HELP!

Thanks much,

Ritu

[baskar_gopalakrishnan2 · Accepted solution]

go to your directory using cd command

example:

cd /ss/ss/sss/

for file in fil*; do cat $file >> concatfile; done

Note: Give the name for the concat final output file something like concatfile

See this link for creating timestamp for you file

http://www.unix.com/shell-programming-scripting/3833-creating-file-date-timestamp.html

[vkaushik82]

below command can help you

cat file1 file2 file3 > newfile

if u want to add file to existing instead of ">" use ">>"

or one more option why not in same scenario u pick all file and append them using receiver file adapter.