PB & multithreading & loops = not responding

[Former Member]

I am using PB12.5

I created a pbl that has a window and it creates 10 threads each of which connects to the database and starts a timer which sends a simple select to the database.  It creates traffic over the network.

I can see the traffic when I go to resmon (windows resource monitor) and go to overview tab and click on PB125.exe

then go to the network tab and under TCP connections I right click on title and make sure I can see: send receive total

I will see each of the 10 threads and the traffic it is sending.

What doesn't work is when I have a long loop in the application.  (click on LOOP button) Then the threads stop sending data.

On the Resource Monitor it will dwindle down to 0 traffic.  Which means the threads are not running

When I put a yield in the loop it works but it is slow and it is multithreaded then I shouldn't have to do that.

Maybe I didn't set it up correctly -  I attached the pbl.

I can send the pbl.  i

Window creates NVO (main) which adds an array of 10 NVOs (TR) with transaction objects.

Each NVO (TR) creates a thread which sends out network traffic every N seconds.

In the Window there is a LOOP button and when that is click it just goes into a loop (doesn't do anything)

pbl 8 objects

muti: application object opens window

w_multi: window has open event which creates the NVO (main) and executes a function which adds 10 connections

n_trnvo_main: NVO main has one fx that creates 10 NVO transactions

n_trnvo NVO transactions connects to the database and creates a thread passing itself to the subthread

nv_shared: shared NVO creates a timer which call ue_wakeup every 5 secs
n_timer: calls ue_wakeup in nv_shared

nv_arg: executes SQL

n_tr: transaction object

ue_wakeup event executes subthread ue_wakeup event which executes the NVO transaction to send SQL

the code is here:  http://www99.zippyshare.com/v/9VwYwOog/file.html

Click the ORANGE download button the other buttons are advertisements

I really appreciate any help.  This has me stumped.  TIA

[Former Member · Accepted solution]

Hi John;

   You probably need to put a YIELD() command in the looping code in order for the other threads to get dispatched.

Regards ... Chris

[Former Member · Accepted solution]

It does work when I put YIELD in the app but that slows the app down.

If it is multithreaded then it should not stop if one of the threads or the app itself is in a loop.

It was working so well up to that point.

[Former Member · Accepted solution]

Is the Loop in the main thread?

FWIW: your basically emulating a tight CPU loop - which would not be the normal processing profile you would build in a real world MT application.

[Former Member · Accepted solution]

Yes the loop is in the main thread.

And it is based on a real world MT FAT CLIENT application.  I just created this as an example of the main parts.

The real world application has 10 connections and they will get disconnected if there is NO network traffic after N minutes.

So I created threads to make sure there is dummy traffic.  This works well.  UNTIL...

It has a LOOP in the main app that is loading a datawindow - about a thousand rows.

That is all the LOOP does is move data from a flat file to a datawindow.  It isn't a tight CPU loop.  It executes functions/events...  within the loop...  and still it doesn't allow the threads to process until the loop is over. 

While it is in a LOOP I can see the network traffic dwindle down to nothing and this external app then disconnects the other transactions and the app fails when it gets out of the LOOP because the connection has been dropped. 

If I put a YIELD in the LOOP then it slows down the app.  Which will have to work -  I was hoping I was just doing something wrong when I created the multithreaded app.

[Former Member · Accepted solution]

That's usually the MT application's downfall when you code processing logic like that in the main thread. All the main thread should do is sleep, wake up, pole the sub-thread's for completed work, assign new work and then go back to sleep (nothing else).

For the DW processing, that should all be encapsulated in a sub-thread within a DS. When the sub-thread  completes, it can either post the main thread or wait to be polled for a completion status. Once the sub-thread has the DS result set loaded, the main thread can just do a ShareData() to populate the DC.

I would also encourage the DS processing thread to connect asynchronously and still issue a Yield() if a liopy is required to process the DS buffers after the Retrieve().

FWIW: Also, instantiate your transaction objects in the NVUO sub-thread as an instance variable. I would not share any work areas with the main thread and never use any global variables in the sub-threads.

HTH

[Former Member]

Hi John,

I checked your code. The reason for your problem is conceptional. Your database calls are all in main thread. Only the timers are multithreaded.

You create n_trnvo_main in main thread and this creates the worker objects of type n_trnvo with create. So this worker objects are also in main thread and its ue_wakeup event, thats run the sql, is also in main thread.

Only the instances of nv_shared are in separate threads. But in its ue_wakeup you use a callback  (using inv_arg) to the instance of n_trnvo. So you have a callback to the main thread.

If you now have a loop in your main thread without a yield() the posted callback events are waiting in the event queue until the loop ends. If you use yield() the loop stops running to look for queued events.

But this is not want you want.

If you really want multithreaded requests you have to do the selects in an object that is created using the SharedObject functionality or in a object that is created from such object.

For a simple test: Put your code for database connect and for database select from n_trnvo to nv_shared and it should work.

Change your concept!

HTH

[Former Member]

Yes that works but as soon as I want to pass something back to the main app using the callback it waits because the main app is in a loop.

Like Chris said, I shouldn't have the MAIN app do much processing.

The main app can have processes and it puts in a queue and the subthreads can do work and add to it's own queue.  Then when it interacts though the callback it puts those requests at end of the main queue. 

BUT when the main app is in a LOOP or is waiting for a LONG stored procedure it waits for that work to be finished (the loop has a mini queue it uses) and it does not look at the main queue until it is finished with the LOOP or Stored Procedure.  Which makes sense. 

I have a two hour process which loads a flatfile processes it and sends it to several databases then calls several stored procs and sends it back to the client and processes it again and sends it off to another flatfile.  Thoughout the whole process I can see the subthreads adding to the main apps queue and it share's it's transaction objects and sends commands to the database.  BUT when the main app is in a LOOP it never looks at the main queue until it is finished (unless I put a YIELD in the code).  When it is finished with a loop it goes back to the main queue and continues processing the callback and other queued events. 

with that said this is an old app and I just wanted to make all the database connections have traffic so the external app won't kill the connections.  And without writing the whole logic I will just use YIELDS  it works with YIELDS in the LOOPS.  ie: for every 100 rows yield and check the main queue.

thanks so much Chris and Rene for helping me understand multithreading.