Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
> Hi all!
>
>
> My program is working fine when thread::send don't use the -async
> option. When it does, all of those created threads exit prematurely.
>
> The pseudo-code I have is this:

Working code that you've tested to exhibit the bug you see is
preferable, and your code was *very* close....

> ===== main file
>
> while 1 {
> ...
> while {nr_live_threads < nr_max_threads} {

This will error out as a syntax error. You want to both initialize
these variables before you use them, and to interpolate them using $
above.

> set tid [thread::create $init_script]
> thread::send -async $tid [list sourceFiles ....]
> }
> after 10000
> }

You never increment nr_live_threads, so this loop above will (assuming
the variables were initalized, and referenced, correctly) simply loop
forever, creating new threads. At least until the whole process is
killed for using all free memory up.

> ===== oo.tcl
> namespace eval ns0 {
> proc runAnsible {...} {
> Parse new ...
> vwait ::exit_flag
> }
> }

You never signal to the master thread that this thread has exited, so
the master (as written here) will never launch a new thread when an
existing one finishes.

> This comprises the important parts of the script, I think.
> When thread::send does not use `-async`, the `vwait ::exit_flag` works and
> the thread is run until the end.
> With `-async`, the thread exits shortly after the `thread::send` command.

Something must be different in the "psudeo" code vs. your real code
then.

> I've read about `thread::preserve` and `thread::release`, but interpreted
> it as necessary when threads have to be orchestrated and some may be
> dependent on the results of others.

No, those are to do reference counting for thread cleanup.

> What I want is really to have several threads launched in the same moment,
> at each run of the while loop that checks if the number of active threads
> is less than the nr_max_threads.
> How can that be accomplished?

Well, first, you have to communicate the exit of a child thread back to
he main thread, and have that comm path decrement "nr_active" (and you
also need to increment nr_active when you launch a new thread).

Syntax cleaned up -- and simplified version of your original code, that
*actually runs*:

thread-test:
#!/usr/bin/tclsh

package require Thread

set nr_max_threads 4
set nr_live_threads 0

set init_script {
puts stderr "Thread: [thread::id] Init: creating sourceFiles"
proc sourceFiles {args} {
source oo.tl
ns0::runAnsible $args
}
puts stderr "Thread: [thread::id] waiting"
thread::wait
puts stderr "Thread: [thread::id] out of wait"
}

while 1 {
while {$nr_live_threads < $nr_max_threads} {
puts stderr "Main: live=$nr_live_threads max=$nr_max_threads"
set tid [thread::create $init_script]
incr nr_live_threads
thread::send -async $tid [list sourceFiles ....]
}
puts stderr "Main: sleeping for 10s"
after 10000
puts stderr "Main: awake"
}

oo.tl:
namespace eval ns0 {
proc runAnsible {...} {
puts stderr "Thread [thread::id]: executing parse new"
Parse new ...
puts stderr "Thread [thread::id]: vwait begin"
vwait ::exit_flag
puts stderr "Thread [thread::id]: vwait complete"
}
}
oo::class create Parse {
constructor {...} {
set random [expr {int(rand()*20000)}]
puts stderr "Thread [thread::id]: object constructor - sleeping for $random"
after $random [list set ::exit_flag 1]
}
}

Sample run of the above:

$ ./thread-test
Main: live=0 max=4
Thread: tid0x7fbfa7b6c640 Init: creating sourceFiles
Thread: tid0x7fbfa7b6c640 waiting
Main: live=1 max=4
Thread tid0x7fbfa7b6c640: executing parse new
Thread tid0x7fbfa7b6c640: object constructor - sleeping for 8476
Thread tid0x7fbfa7b6c640: vwait begin
Thread: tid0x7fbfa6b6a640 Init: creating sourceFiles
Thread: tid0x7fbfa6b6a640 waiting
Main: live=2 max=4
Thread tid0x7fbfa6b6a640: executing parse new
Thread tid0x7fbfa6b6a640: object constructor - sleeping for 16806
Thread tid0x7fbfa6b6a640: vwait begin
Thread: tid0x7fbfa6369640 Init: creating sourceFiles
Thread: tid0x7fbfa6369640 waiting
Main: live=3 max=4
Thread tid0x7fbfa6369640: executing parse new
Thread tid0x7fbfa6369640: object constructor - sleeping for 11225
Thread tid0x7fbfa6369640: vwait begin
Thread: tid0x7fbfa5b68640 Init: creating sourceFiles
Thread: tid0x7fbfa5b68640 waiting
Main: sleeping for 10s
Thread tid0x7fbfa5b68640: executing parse new
Thread tid0x7fbfa5b68640: object constructor - sleeping for 5573
Thread tid0x7fbfa5b68640: vwait begin
Thread tid0x7fbfa5b68640: vwait complete
Thread tid0x7fbfa7b6c640: vwait complete
Main: awake
Main: sleeping for 10s
Thread tid0x7fbfa6369640: vwait complete
Thread tid0x7fbfa6b6a640: vwait complete
Main: awake
Main: sleeping for 10s
Main: awake
Main: sleeping for 10s

And, it will continue to loop saying 'awake' and 'sleeping' since the
exit of the children is never communicated to the master.

You need to master to become aware that one of the children has exited,
so it knows to relaunch another child. One way is to use the
additional result variable for -async threads and vwait on that
variable in the master.

Here is the 'diff' necessary to have the master monitor children
exiting and to launch a new child when that happens:

--- thread-test.v1 2024-06-19 10:42:34.359605931 -0400
+++ thread-test 2024-06-19 11:00:01.433949725 -0400
@@ -4,6 +4,7 @@

set nr_max_threads 4
set nr_live_threads 0
+set sync 0

set init_script {
puts stderr "Thread: [thread::id] Init: creating sourceFiles"
@@ -21,10 +22,10 @@
puts stderr "Main: live=$nr_live_threads max=$nr_max_threads"
set tid [thread::create $init_script]
incr nr_live_threads
- thread::send -async $tid [list sourceFiles ....]
+ thread::send -async $tid [list sourceFiles ....] sync
}
- puts stderr "Main: sleeping for 10s"
- after 10000
- puts stderr "Main: awake"
+ puts stderr "Main: waiting for a child to exit"
+ vwait sync
+ puts stderr "Main: a child exited"
+ incr nr_live_threads -1
}

And a sample run:

$ ./thread-test
Main: live=0 max=4
Thread: tid0x7f2d21a4b640 Init: creating sourceFiles
Thread: tid0x7f2d21a4b640 waiting
Main: live=1 max=4
Thread tid0x7f2d21a4b640: executing parse new
Thread tid0x7f2d21a4b640: object constructor - sleeping for 19992
Thread tid0x7f2d21a4b640: vwait begin
Thread: tid0x7f2d20a49640 Init: creating sourceFiles
Thread: tid0x7f2d20a49640 waiting
Main: live=2 max=4
Thread tid0x7f2d20a49640: executing parse new
Thread tid0x7f2d20a49640: object constructor - sleeping for 8316
Thread tid0x7f2d20a49640: vwait begin
Thread: tid0x7f2d1bfff640 Init: creating sourceFiles
Thread: tid0x7f2d1bfff640 waiting
Main: live=3 max=4
Thread tid0x7f2d1bfff640: executing parse new
Thread tid0x7f2d1bfff640: object constructor - sleeping for 17902
Thread tid0x7f2d1bfff640: vwait begin
Thread: tid0x7f2d1b7fe640 Init: creating sourceFiles
Thread: tid0x7f2d1b7fe640 waiting
Main: waiting for a child to exit
Thread tid0x7f2d1b7fe640: executing parse new
Thread tid0x7f2d1b7fe640: object constructor - sleeping for 12322
Thread tid0x7f2d1b7fe640: vwait begin
Thread tid0x7f2d20a49640: vwait complete
Main: a child exited
Main: live=3 max=4
Thread: tid0x7f2d1affd640 Init: creating sourceFiles
Thread: tid0x7f2d1affd640 waiting
Main: waiting for a child to exit
Thread tid0x7f2d1affd640: executing parse new
Thread tid0x7f2d1affd640: object constructor - sleeping for 7521
Thread tid0x7f2d1affd640: vwait begin
Thread tid0x7f2d1b7fe640: vwait complete
Main: a child exited
Main: live=3 max=4
Thread: tid0x7f2d1a7fc640 Init: creating sourceFiles
Thread: tid0x7f2d1a7fc640 waiting
Main: waiting for a child to exit
Thread tid0x7f2d1a7fc640: executing parse new
Thread tid0x7f2d1a7fc640: object constructor - sleeping for 9508
Thread tid0x7f2d1a7fc640: vwait begin
Thread tid0x7f2d1affd640: vwait complete
Main: a child exited
Main: live=3 max=4
Thread: tid0x7f2d19ffb640 Init: creating sourceFiles
Thread: tid0x7f2d19ffb640 waiting
Main: waiting for a child to exit
Thread tid0x7f2d19ffb640: executing parse new
Thread tid0x7f2d19ffb640: object constructor - sleeping for 13232
Thread tid0x7f2d19ffb640: vwait begin