Redwood - file is overwritten if it already exists

[Former Member]

Scenario (I am at customer site): A file event moves a file to the destination directory. Subsequently, a second file arrives before first file is finished processing (due to job error, long-running, etc). The file event moves the file to the the destination directory and overwrites the file.

Especially, if wild-cards are being used, multiple files can build up in the input directory and the loss of files as the file event moves them and they overlay each other is not a workable production solution.

So the question would be... When using the file event, how to not overwrite a file if it is already there?

[Former Member]

You would set the scheduler parameter atomic_file_events to true. Log into RS and issue the command:

RS> set atomic_file_events 'true'

Verify the parameter is set by logging into RS and issuing the command:

RS> show parameters 

You must restart the scheduler before the new setting will be implemented. If you don't use atomic file events, then you'll need to setup an archive directory and add it to the file event. You can even add timestamps to the filename while you archive. Once you do this, the file event will raise the event for each move to the archive directory.

Search for 'agent parameters' and 'event agent behavior' in Redwood Help for more info.

[Former Member]

If you to use these settings only for one specific group of files and not entire scheduling environment, write the job in redwood in such way that it copies latest file as the name required by subsequent jobs while the older files are renamed to filename_<timestamp>.

If you want to use these settings for all files in your scheduling environment, change the scheduler parameters so that it copies all files as filename_<timestamp>. But with these settings you will have either define logic in subsequent jobs to use latest file. Same can be archived by creating intermediate job which copied latest file as filename from filename_<timestamp>.

Hope this helps!

- Bhushan

[Former Member]

Hello Renee - was wondering if Redwood adds anything to the filename when the "atomic_file_event" parameter is set to true? or does it leave it alone?

Thank you,

Dean.

[Former Member]

Hi Dean,

Actually, all atomic_file_events does is allow you to capture each file that arrives regardless of event status. Filenames are left alone.

You would rename the file by using the archive feature of the file event. You can chose to move the file or keep it in the same directory and just add a datestamp. For example...

Say you have a scheduler file event that monitors for /tmp/today.dat. If you wanted to add a datestamp but leave it in the current directory, you could add /tmp/today.dat.%t to the "Destination directory." This would add a datestamp of format YYYY-MM-DD-HH24-MI-SS. Using /tmp/%n.%x.%i instead would deliver identical results.

For more on this topic, search Redwood help for "event agent behavior".