FTP Get File List Action Block, It's double listing files! ver 11.5

[Former Member]

Hi guys.. I have a good one! I have an FTP Get File List action block in my BLS transaction. Occasionally, it double lists the files in its output. For testing I put a repeater with a logevent output where I log the filename, date, and size. Heres what I saw for my action block output.

 
2009-02-13 00:38:00,963  [UserEvent] : File Name: DMM_Export_0010056.txt, File Date 2009-02-13T00:36:00, File Size 339 
2009-02-13 00:38:00,963  [UserEvent] : File Name: DMM_Export_0010056.txt, File Date 2009-02-13T00:36:00, File Size 339 

This is xMII version 11.5.6 b73 with java 1.4.2_07

I have a workaround by putting in a distinct action block, after the filelist, but anybody have an idea why this might happen? My theory is that something might be occuring if the file is being written to while we try to process it, but not sure.

I've been building BLS parsers since 2003, (Remember those fun times with Jeremy?) I've never seen this happen.

[Former Member · Accepted solution]

Doug,

Is there any information in the logs when you run the original transaction (w/out the distinct)?

Kevin

[Former Member · Accepted solution]

My example is a sample log file before the distinct action. The general log shows nothing other than the subsequent transaction errors I get as a result of running the same error twice (Tcode return from BAPI calls etc)

Here is something else interesting.. my userlog file is acting funny, like its trying to write on top of itself. could it be the transaction is actually running twice or parts of it?

For example look at the following log entries

This is how my log file entry for a production confirmation should look

2009-02-13 00:38:06,854 [LHScheduler-Int10_NestingWOProdConf] INFO   UserLog - [UserEvent] : 

However sometimes... its looking like this...

2009-02-13 2009-02-13 00:38:11,854 [LHScheduler-Int10_NestingWOProdConf] INFO   UserLog - [UserEvent] :

Like it started writing to the log, then started again.

The problem we are having is that we have JCO calls to SAP in this transaction that does goods movement, we get locking / block errors back from our saying that we (our sap account) is already updating the information. Sometimes the information would be posted twice! You can see how this has become a HUGE issue posting data to a LIVE system twice.

This is happening on 2 xMII servers.

[jcgood25 · Accepted solution]

You're not using an iGrid updateGrid(true) to trigger this are you? If you don't return something and have DisplayColumns mapped accordingly it may hit the TRX twice.

[Former Member · Accepted solution]

Nope, its running on a schedule every 2 minutes. No user interaction.

[agentry_src · Accepted solution]

Doug,

How long does the transaction take to execute? And how much variation in execution time do you see?

Thanks,

Mike

[Former Member · Accepted solution]

Mike,

It really depends on the number of files. We have a pause in a repeater that waits 5 seconds after we do a goods movement before we do a production confirmation. The blocking error comes in the goods movement. With the pauses, it probably lasts 45 to 60 seconds if we have a lot of records. But there is a two minute cycle time between executions starts, and I've never seen it execute them concurrently due to the scheduler.

[agentry_src · Accepted solution]

Doug,

It is unlikely to be due to the scheduler itself, but I have run into some pretty large variations in execution times in the ERP system (especially at month ends and beginnings and at the end of shifts). And timeouts are generally set pretty high on that end (30 or 60 seconds). I am not definitely saying that is the problem cause I have no idea how your transactions are configured and what volumes, etc. But this sounds like a problem that has come up with other clients I have done interface work with.

For those following this thread, what I am suggesting (actually I am just raising the possibility) is:

The execution of a scheduled transaction takes too long and the same transaction then kicks off by schedule and can see that a goods movement has not completed and grabs the same data that the previous transaction is using.

This one is my own curiosity, why do the goods movements first and then the confirmations? I invariably do it the other way around.

Thanks,

Mike

[Former Member · Accepted solution]

Michael,

This parser wasn't my design, but I've talked to the author and originally the Goods Movement was done before the production confirmation because of some possible reversals that might be done in the transaction. Those reversals were never implemented but the logic was never changed (Order of submissions). That being said, this transaction was extensively tested and had no problem so the design was never altered.

Here's the REALLY wierd part, its not just confirmations that are being done twice, We have a SQL insert in this transaction that is ALSO being run twice. Within the same second. In the last case, 200 milliseconds apart (according to the log file) This should not happen. Its as if action blocks of the transaction run twice occasionally. SQL blocks, log entries, JCO calls, all seem to be trying to run concurrently.

[agentry_src · Accepted solution]

Hi Doug,

Thanks for the explanation. That makes sense.

And there is nothing so permanent as a temporary fix.

I recall this happening a few months ago. There were two java.exe instances running in task manager . Can you check to see if there is another instance running in task manager?

Regards,

Mike

Found a couple of threads with the same issue.

[duplicate emails|]

[duplicate inserts|]

Edited by: Michael Appleby on Feb 16, 2009 5:34 PM

[Former Member · Accepted solution]

I found two instances of java.exe I killed them both and restarted servletexec lets see.

Is there a way to keep this from happening? When it does it causes some serious issue with the customer.

Edited by: Doug Holtke on Feb 16, 2009 7:43 PM

[agentry_src · Accepted solution]

Hi Doug,

I asked that same question. What I was told (and don't presume that I am expert in this area) was that there was an incomplete shutdown of the servlet which did not shutdown the jave.exe service. When it was restarted, it didn't take notice of the extra java.exe and started a new one. I was also told this is based upon the functioning of the Atlanta servlet and not something that MII has any control over.

I suspect that the restart procedures need to be reviewed and perhaps modified to ensure that the active java.exe services are shutdown completely prior to restarting the service. Again, this is not my area of expertise. I would suggest opening another message to see if anyone else has come up with a programmatic method of preventing recurrences.

Good luck,

Mike