Few Files missing from source to target but all archived

[Former Member]

Hi Experts,

we are facing a strange issue in one file to file scenario in production environment while moving the files from system say Sys1 to Sys2 as below:

1) All the files generated in Sys1 are not reaching Sys2.For e.g. 14 files with data were generated in sytem Sys1 and only 8 reached in System Sys2 ,remaining 6 are missing and does not even show up in PI Monitoring(RWB and SXMB_MONI).This is happening every day for this interface.

2) Once a file is put in ‘Ready’ folder in Sys1 it is picked by PI to move to Sys2 and PI is moving it to Sys1 ‘Archive’ folder adding date time prefix to file name.

3) All 14 files are available in Sys1 Archive folder which means PI recognized these files and moved to Archive.But We need to find out why only 8 files were moved to Sys2 and remaining 6 were not moved.

4) The program to generate files runs every hour in Sys1 ,if data is available this program generates file of size 1 KB or more depending on data volume and if no data is available it generates a 0 KB file.Now PI ignores 0 kb files which is perfectly fine but in our case sometime it ignores files which have data too.

5) Also in RWB Message Monitoring we have only entries for 8 files which were moved to Sys2 both in Adapter Engine and Integration Engine.There is no entries for 6 files which were archived in archive folder but not picked up by PI system.

6) Also both Sender and Reciever channels are running fine and no error encountered and are green status.

7) we have used following parameters in sender file channel:

Msecs to wait Before Modification Check - 10000

Empty File Handling : Do Not Create Message

Need your help on these urgent issue.

Regards

Deepak

[Former Member · Accepted solution]

Hi All,

I have changed parameter in sender FTP channel Empty File handling from "Donot create Message" to "SKIP EMPTY FILES" and problem solved.

We have following root cause and resolution analysed and suggested from SAP on this issue:

"It seems that the file is being read from

the source directory before it is completely written. As the files are

getting created in the source directory by the custom program in ECC

system, it seems thast, it has created the file and before it writes

the content, the same file has been read by the adapter. Since the file

has not contents till that time, it is being treated as an empty file

and processing happens accordingly. Later, after few milisec/secs it is

being archived as per the 'Processing Mode' setting in the channel

configuration.

To avoid this issue, we have the parameter "Msecs to Wait Before

Modification Check" which can be set in the Sender channel configurationunder "Advanced" mode. But this parameter is only valid for "NFS"

protocol and not for "FTP". As you are using FTP mode as "Transport

Protocol", you can use the following steps to avoid this issue:

1. Make the changes in the custom program in ECC system in such a way

that it only picks up the completely written files (as per the SAP Note

821267 point 3) i.e.

- Create the file using an extension, which does not get processed by the File Adapter, e.g., ".tmp"

- Write the file content

- Rename the file to its final name, so the File Adapter will notice

its existence and pick it up

2. Create the file in some other folder and move it to the source

directory once it is completely written."

Hope this will help someone facing similar issue.

Regards
Deepak

[agasthuri_doss · Accepted solution]

Thanks for sharing Deepak.

Cheers

Agasthuri

[Former Member]

Hi Deepak,

is this only one time issue or happening everyday?

Cheers

Hidayat

[Former Member]

Hi Hidayat,

This is happening almost everyday.

Regards

Deepak

[nabendu_sen]

Hi Deepak,

I assume you have checked different nodes (different Java instances to pick the file) mentioned within a Communication Channel and you did not find any log. Please ask Basis team to restart the Java stack and check whether you can still see the issue.

[Former Member]

Hi Nabendu,

We have two cluster nodeID.But there is no logs of issues in any of these two nodes.

Also,for getting approval for Java stack Restart.We need to highlight actual cause of issue.

We have raised this issue as an OSS message to SAP also.They are also investigating but till now no response from them.

Regards

Deepak

[Ryan-Crosby]

Hi Deepak,

Have you checked the queue monitors to make sure something has not got stuck in an error status there?  Every once in a while we have messages that get stuck and we'll find messages waiting in SMQ1/SMQ2.

Regards,

Ryan Crosby

[Former Member]

Hi Ryan,

We have no logs of any error in SMQ1 and SMQ2.

Regards

Deepak

[former_member192851]

And nothing in channel monitoring?

if messages don't reach Engine - check channel logs.

[Former Member]

Hi Artem,

Yes we have nothing in Channel log for missing files which were archieved.

Only all processed messages upto sys2 has log in sender and receiver channels.

[rajasekhar_reddy14]

Hi,

If you are archiving fault files also in to same directory then this might be reason, compare missing files content .

Regards,

Raj

[Former Member]

Hi Raja Sekhar,

We are placing error and faulty files in separate directory.No files exisit in error directory and all files were successfully archieved.

[nabendu_sen]

Hi Deepak,

Please let us know the size of the 6 files for which messages are missing in PI.

[Former Member]

Hi Nabendu,

All files are having data and size of files are greater than or equal to 1KB atleast.There is no empty 0bytes files.

Regards

Deepak

[agasthuri_doss]

Deepak,

Does all the files are handled by the same sender & receiver adapter ?

Cheers

Agasthuri

[Former Member]

Hi Agasthuri,

Yes all files are maintained by a sender file and a receiver file channel only.

Regards

Deepak