In article <677c7a1b$0$28501$426a74cc@news.free.fr>,
Nicolas George <nicolas$george@salle-s.org> wrote:
>Kalevi Kolttonen, dans le message <vlh5ag$1nruu$1@dont-email.me>, a
> �crit�:
>> I am no expert, but I guess if you need to do async programming
>> on UNIX/Linux userspace, your best is to use POSIX Threads.
>
>Very common misconception.

This is correct. Threads are a concurrency mechanism. You use
them to write (more or less) sequential execution flows that run
concurrently with respect to one another. They are not needed
to do "async programming UNIX/Linux userspace". Indeed, in some
sense they're sort of the antithesis of async programming.

>The communication mechanisms between POSIX
>threads and Unix I/O are completely alien to each-other: it is not possible
>to poll() on a thread condition, nor is it it possible to set up a condition
>to be woken by data on a file descriptor. As a result, anybody who tries to
>use threads to solve problems of I/O concurrency ends up having to implement
>a poll() or equivalent loop in each thread, defeating the purpose.

This, however, does not follow. I don't see why "poll" is
strictly required for IO concurrency.

In addition to signalfd etc, there is no reason you cannot, say,
have a signal handler that broadcasts on a condition variable
after an asynchronous IO operation completes, thus waking up a
thread.

More generally, in the context of a multithreaded program,
(assuming the POSIX threading model) the "condition to be woken
by data on a file descriptor" is that the IO operation on which
the thread was blocked has completed or been interrupted. In a
system with kernel scheduled threads, this simply means that the
thread becomes runnable again, and will be resumed at some
point; this is exactly analogous to the sort of state
transitions a process goes through in a traditional Unix-style
kernel when blocked on IO.

Moreover, nothing prevents a program from, say, sharing a
producer/consumer queue between threads where one extracts
entries and performs IO on them while the other prepares IO
operations to be performed by the worker. For that matter, one
can have multiple IO threads pushing work into a queue that's
executed by a pool of worker threads.

>POSIX threads are good to improve on computation concurrency, but they do
>not make I/O concurrency simpler, quite the opposite.
>
>The same might not be true for other kinds of threads.

A nice property of POSIX threads is that, in _most_
implementations, if you go off and initiate some blocking
operation, such as performing (blocking) IO on a file descriptor
of some sort, other runnable threads in the system can continue
executing. Thus, it follows that you can have multiple IO
operations in separate threads pending at one time. In that
sense, one can leverage threads for IO concurrency (indeed, many
programs do just that). Threads can block independently without
impeding execution of the overall program.

Of course, not all POSIX thread implementations are capable of
doing this: consider a purely userspace "green" thread
implementation; executing a single blocking system call blocks
the whole program. In those, in many cases, blocking IO is
emulated at the library level (which translates "normal" IO
operations into their non-blocking counterparts) to give the
illusion of sequential program flow; of course, the simulacrum
is inexact at best, which is why most systems push thread
management up into the kernel.

- Dan C.

On Tue, 7 Jan 2025 13:18:54 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>In article <vlip2c$24ccb$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>On Mon, 06 Jan 2025 16:46:56 GMT
>>ITYF it is VERY widely shared and having a signal safe API function is only
>>step 2 - plenty of the functions in the program itself or 3rd party library
>>functions are probably not re-entrant safe and even if they are, having
>>code stomp over itself - eg if in the middle of writing a log message then a
>>signal is generated which tried to write a log message itself - is a very
>>poor way to write code.
>
>So don't write code that way. It does not follow that the only
>thing you can do in a signal handler is an some atomic flag
>somewhere.

Just because you can doesn't mean you should. C lets you do a lot of things
that are a Bad Idea.

In article <vlioum$24bqm$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Mon, 6 Jan 2025 16:39:49 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vlgun1$1minf$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>On Mon, 6 Jan 2025 15:22:51 -0000 (UTC)
>>>Multiplexing is not asychronous, its simply offloading status checking to
>>>the kernel.
>>
>>Of course. It's a means to allow a program to respond to
>>asynchronous events.
>
>Thats not the same as the program itself being asynch.

Isn't it? The point is that the program kicks off multiple
asynchronous operations; the issue comes when figuring out what
to do when they complete. In general, there are only really two
choices here: either poll their completion status, or expect to
be notified by some kind of an event. In a POSIX-y environment,
`poll`/`select` etc give you the former; signals give you the
latter.

>>>The program using is still very much sequential , at least at
>>>that point.
>>
>>But the events are not. That's the point. This allows a
>>program to initiate a non-blocking IO operation (like, say,
>>establishing a TCP connection using the sockets API), go do
>>something else, and check it's status later.
>
>Thats not proper asych, its still sequential. Proper asynch is when the
>program execution path is directly modified by external events. Otherwise
>you could claim simply using the standard file I/O system is asynchronous
>programming as there's no guarantee that any data has been written to the disk
>before write(), fprintf() etc return.

This is conflating multiple things. Most IO operations dealing
with the actual hardware _are_ asynchronous (this is what
McIlroy meant in the quote I posted earlier). The system call
interface gives the program the illusion of those happening
sequentially, but that's not how the devices really work.

It turns out the simple model of early research Unix was
insufficient for handling all sorts of important use cases,
hence why interfaces like `select` and `poll` were added.

>>>Posix AIO is not asynch in the strict sense , its more "ok kernel, go do this
>>>and I'll check how you're doing later". Proper asynch where the program
>>>execution path gets bounced around between various callbacks is something
>>>else entirely.
>>
>>The POSIX AIO interface allows the kernel to generate a signal
>>to inform the program that an IO operation has completed, e.g.,
>>by setting up the `aio_sigevent` and `SIGEV_SIGNAL`. It doesn't
>>get much more asynchronous than that.
>
>Sure, but as I've said before, signals should only set flags to be processed
>later.

You said that, but that flies in the face of 50 years of
evidence to the contrary and the letter of the standard. This
doesn't mean that you should do arbitrary amounts of work in a
signal handler, but there's no reason that, say, one couldn't
push an event onto a queue and signal a condition variable or
something similar.

- Dan C.

In article <vljcbk$27v6l$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Tue, 7 Jan 2025 13:18:54 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vlip2c$24ccb$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>On Mon, 06 Jan 2025 16:46:56 GMT
>>>ITYF it is VERY widely shared and having a signal safe API function is only
>>>step 2 - plenty of the functions in the program itself or 3rd party library
>>>functions are probably not re-entrant safe and even if they are, having
>>>code stomp over itself - eg if in the middle of writing a log message then a
>>>signal is generated which tried to write a log message itself - is a very
>>>poor way to write code.
>>
>>So don't write code that way. It does not follow that the only
>>thing you can do in a signal handler is an some atomic flag
>>somewhere.
>
>Just because you can doesn't mean you should. C lets you do a lot of things
>that are a Bad Idea.

I have to ask at this point: have you ever written a concurrent
program under Unix? One that used signals? For that matter,
have you ever written a program that used `fork()` and caught a
`SIGCHLD`?

- Dan C.

Nicolas George <nicolas$george@salle-s.org> writes:
>Lawrence D'Oliveiro , dans le message <vli2lj$1t3lt$8@dont-email.me>, a
> �crit�:
>> Linux offers signalfd, so you can indeed use poll(2) in a thread to be
>> woken up by any file descriptor, including a signal one (and that includes
>> POSIX real-time signals).
>
>Proving my point that you need to use poll() even when doing threads.

It "proves" no such thing.

An I/O thread can coordinate other threads using
posix condition variables, system V semaphores, pipes, etc.

On Tue, 7 Jan 2025 14:13:29 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>In article <vlioum$24bqm$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>On Mon, 6 Jan 2025 16:39:49 -0000 (UTC)
>>Thats not the same as the program itself being asynch.
>
>Isn't it? The point is that the program kicks off multiple

Not, it isn't. Using operating system facilities is standard programming, its
not asych programming whereby the program execution will be automatically
be diverted when an operation completes.

>>Thats not proper asych, its still sequential. Proper asynch is when the
>>program execution path is directly modified by external events. Otherwise
>>you could claim simply using the standard file I/O system is asynchronous
>>programming as there's no guarantee that any data has been written to the
>disk
>>before write(), fprintf() etc return.
>
>This is conflating multiple things. Most IO operations dealing
>with the actual hardware _are_ asynchronous (this is what
>McIlroy meant in the quote I posted earlier). The system call
>interface gives the program the illusion of those happening
>sequentially, but that's not how the devices really work.

And? By your definition its still asynchronous programming.

>>Sure, but as I've said before, signals should only set flags to be processed
>>later.
>
>You said that, but that flies in the face of 50 years of
>evidence to the contrary and the letter of the standard. This

Please don't just make stuff up.

On Tue, 7 Jan 2025 14:14:38 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>In article <vljcbk$27v6l$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>On Tue, 7 Jan 2025 13:18:54 -0000 (UTC)
>>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>>In article <vlip2c$24ccb$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>>On Mon, 06 Jan 2025 16:46:56 GMT
>>>>ITYF it is VERY widely shared and having a signal safe API function is only
>>>>step 2 - plenty of the functions in the program itself or 3rd party library
>>>>functions are probably not re-entrant safe and even if they are, having
>>>>code stomp over itself - eg if in the middle of writing a log message then
>a
>>>>signal is generated which tried to write a log message itself - is a very
>>>>poor way to write code.
>>>
>>>So don't write code that way. It does not follow that the only
>>>thing you can do in a signal handler is an some atomic flag
>>>somewhere.
>>
>>Just because you can doesn't mean you should. C lets you do a lot of things
>>that are a Bad Idea.
>
>I have to ask at this point: have you ever written a concurrent
>program under Unix? One that used signals? For that matter,
>have you ever written a program that used `fork()` and caught a
>`SIGCHLD`?

Is that supposed to be a serious question?

The only thing that should ever be done in a child exit handler is a wait*()
or set a flag.

cross@spitfire.i.gajendra.net (Dan Cross) writes:
>In article <vlioum$24bqm$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:

>>Thats not proper asych, its still sequential. Proper asynch is when the
>>program execution path is directly modified by external events. Otherwise
>>you could claim simply using the standard file I/O system is asynchronous
>>programming as there's no guarantee that any data has been written to the disk
>>before write(), fprintf() etc return.
>
>This is conflating multiple things. Most IO operations dealing
>with the actual hardware _are_ asynchronous (this is what
>McIlroy meant in the quote I posted earlier). The system call
>interface gives the program the illusion of those happening
>sequentially, but that's not how the devices really work.

Indeed, and it was subsequently recognized that more than
the 'sync'[*] system call was required for applications to
ensure data was successfully written to the underlying
device.

Applications in those days (e.g. fsck) would access the
raw character device using the unbuffered read() and
write() system calls rather than using stdio. A key
characteristic of raw devices were that the hardware DMA would
use the application buffer directly rather than copying
the data to the kernel buffer pool first.

[*] I recall using 'sync;sync;sync' from the Sixth Edition
command line more than once, before rebooting.

Subsequently APIs like fdatasync(2) and open flags
such as O_DIRECT and O_DSYNC were added.

>
>It turns out the simple model of early research Unix was
>insufficient for handling all sorts of important use cases,
>hence why interfaces like `select` and `poll` were added.

Although to be fair, select(2) originated with BSD and is
a bit less flexible than poll(2).

In article <vljgbg$28o6f$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Tue, 7 Jan 2025 14:14:38 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vljcbk$27v6l$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>On Tue, 7 Jan 2025 13:18:54 -0000 (UTC)
>>>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>>>In article <vlip2c$24ccb$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>>>On Mon, 06 Jan 2025 16:46:56 GMT
>>>>>ITYF it is VERY widely shared and having a signal safe API function is only
>>>>>step 2 - plenty of the functions in the program itself or 3rd party library
>>>>>functions are probably not re-entrant safe and even if they are, having
>>>>>code stomp over itself - eg if in the middle of writing a log message then
>>a
>>>>>signal is generated which tried to write a log message itself - is a very
>>>>>poor way to write code.
>>>>
>>>>So don't write code that way. It does not follow that the only
>>>>thing you can do in a signal handler is an some atomic flag
>>>>somewhere.
>>>
>>>Just because you can doesn't mean you should. C lets you do a lot of things
>>>that are a Bad Idea.
>>
>>I have to ask at this point: have you ever written a concurrent
>>program under Unix? One that used signals? For that matter,
>>have you ever written a program that used `fork()` and caught a
>>`SIGCHLD`?
>
>Is that supposed to be a serious question?

Yes.

>The only thing that should ever be done in a child exit handler is a wait*()
>or set a flag.

I think perhaps you should try to write some complex programs in
the Unix environment before making such categorial statement.

- Dan C.

On Tue, 7 Jan 2025 15:35:44 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>In article <vljgbg$28o6f$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>On Tue, 7 Jan 2025 14:14:38 -0000 (UTC)
>>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>>In article <vljcbk$27v6l$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>>On Tue, 7 Jan 2025 13:18:54 -0000 (UTC)
>>>>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>>>>In article <vlip2c$24ccb$1@dont-email.me>, <Muttley@DastardlyHQ.org>
>wrote:
>>>>>>On Mon, 06 Jan 2025 16:46:56 GMT
>>>>>>ITYF it is VERY widely shared and having a signal safe API function is
>only
>>>>>>step 2 - plenty of the functions in the program itself or 3rd party
>library
>>>>>>functions are probably not re-entrant safe and even if they are, having
>>>>>>code stomp over itself - eg if in the middle of writing a log message then
>
>>>a
>>>>>>signal is generated which tried to write a log message itself - is a very
>>>>>>poor way to write code.
>>>>>
>>>>>So don't write code that way. It does not follow that the only
>>>>>thing you can do in a signal handler is an some atomic flag
>>>>>somewhere.
>>>>
>>>>Just because you can doesn't mean you should. C lets you do a lot of things
>>>>that are a Bad Idea.
>>>
>>>I have to ask at this point: have you ever written a concurrent
>>>program under Unix? One that used signals? For that matter,
>>>have you ever written a program that used `fork()` and caught a
>>>`SIGCHLD`?
>>
>>Is that supposed to be a serious question?
>
>Yes.
>
>>The only thing that should ever be done in a child exit handler is a wait*()
>>or set a flag.
>
>I think perhaps you should try to write some complex programs in
>the Unix environment before making such categorial statement.

Don't be patronising. I've probably written more unix software in 30 years
than you've had hot dinners including a fully featured telnetd and numerous
other servers for work and play. And in the places I've worked which included
finance/banking, aerospace and government, the advice was almost always NOT to
use signals in the first place unless there was no choice - eg SIGCHLD - but if
you did then do very little in the handler and nothing that could cause any
re-entrancy issues.

I suspect its you who needs a bit more practice at writing large multi process
and multi threaded applications.

Dan Cross, dans le message <vljbvv$gl9$1@reader2.panix.com>, a écrit :
> This, however, does not follow. I don't see why "poll" is
> strictly required for IO concurrency.

Well, try to do implement anything non-trivial involving I/O concurrency,
including timeouts, clients causing other clients to abort, etc., with
the common denominator of POSIX threads and come back telling us how you
managed that.

I tried, and stopped trying using threads for I/O concurrency.

On 07 Jan 2025 15:54:48 GMT
Nicolas George <nicolas$george@salle-s.org> wibbled:
>Dan Cross, dans le message <vljbvv$gl9$1@reader2.panix.com>, a �crit�:
>> This, however, does not follow. I don't see why "poll" is
>> strictly required for IO concurrency.
>
>Well, try to do implement anything non-trivial involving I/O concurrency,
>including timeouts, clients causing other clients to abort, etc., with
>the common denominator of POSIX threads and come back telling us how you
>managed that.
>
>I tried, and stopped trying using threads for I/O concurrency.

For some mad reason it seems to be the way to do it in Windows and also Java
IIRC.

In article <vljg72$28nj0$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Tue, 7 Jan 2025 14:13:29 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vlioum$24bqm$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>On Mon, 6 Jan 2025 16:39:49 -0000 (UTC)
>>>Thats not the same as the program itself being asynch.
>>
>>Isn't it? The point is that the program kicks off multiple
>
>Not, it isn't. Using operating system facilities is standard programming, its
>not asych programming whereby the program execution will be automatically
>be diverted when an operation completes.
>
>>>Thats not proper asych, its still sequential. Proper asynch is when the
>>>program execution path is directly modified by external events. Otherwise
>>>you could claim simply using the standard file I/O system is asynchronous
>>>programming as there's no guarantee that any data has been written to the
>>disk
>>>before write(), fprintf() etc return.
>>
>>This is conflating multiple things. Most IO operations dealing
>>with the actual hardware _are_ asynchronous (this is what
>>McIlroy meant in the quote I posted earlier). The system call
>>interface gives the program the illusion of those happening
>>sequentially, but that's not how the devices really work.
>
>And? By your definition its still asynchronous programming.

In the kernel, it sure is. Unix programmers have been writing
asynchronous programs (using e.g. `fork`) since 1970.

>>>Sure, but as I've said before, signals should only set flags to be processed
>>>later.
>>
>>You said that, but that flies in the face of 50 years of
>>evidence to the contrary and the letter of the standard. This
>
>Please don't just make stuff up.

Hmm. I wonder what shell you use, if you use Unix at all.

Here for example is the signal handler for SIGINT in bash:
https://git.savannah.gnu.org/cgit/bash.git/tree/sig.c?h=devel#n691

You'll notice that it's rather more complex than just "set a
flag or `wait`.

Here's the handler in `zsh`:
https://github.com/zsh-users/zsh/blob/263659acb73d0222e641dfd8d37e48e96582de02/Src/signals.c#L399

You'll again notice that it does rather more than just "set a
flag or wait."

Here's the SIGWINCH handler for good 'ol `script` from
OpenBSD:
https://github.com/openbsd/src/blob/6d253f95424ee0054c798f493d12377911cd3668/usr.bin/script/script.c#L224

Wow, that one does ioctl's modifying PTY state and signals an
entire process group. That's definitely a bit more than just
"setting a flag or wait."

Here's an example from Postgres:
https://github.com/postgres/postgres/blob/5b291d1c9c09d75982c3270bfa61d4e822087b6a/src/backend/storage/ipc/latch.c#L2269

Wow, that one writes to a pipe. Definitely a bit more than just
"setting a flag or wait."

Those are just a few examples. If one cares to look, one will
find many more in non-trivial programs used in production daily.

- Dan C.

In article <vljiml$296n5$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Tue, 7 Jan 2025 15:35:44 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vljgbg$28o6f$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>>[snip]
>>>>
>>>>I have to ask at this point: have you ever written a concurrent
>>>>program under Unix? One that used signals? For that matter,
>>>>have you ever written a program that used `fork()` and caught a
>>>>`SIGCHLD`?
>>>
>>>Is that supposed to be a serious question?
>>
>>Yes.
>>
>>>The only thing that should ever be done in a child exit handler is a wait*()
>>>or set a flag.
>>
>>I think perhaps you should try to write some complex programs in
>>the Unix environment before making such categorial statement.
>
>Don't be patronising.

Wow, that's rich coming from you, my guy.

>I've probably written more unix software in 30 years
>than you've had hot dinners including a fully featured telnetd and numerous
>other servers for work and play. And in the places I've worked which included
>finance/banking, aerospace and government,

"For play" implies things that could be, or are, open source.
So post a link to code, then.

Bluntly, I don't believe that any of this is true. Your posts
here show a distinct lack of relevant experience and knowledge.

>the advice was almost always NOT to
>use signals in the first place unless there was no choice - eg SIGCHLD - but if
>you did then do very little in the handler and nothing that could cause any
>re-entrancy issues.

Earlier you said you should _only_ set a flag in a signal
handler. Then you moved the goal posts to say that you could
call `wait` (presumably after I posted that example). How
you're just saying that you should do "very little that could
cause any re-entrancy issues", which is something I more or
less said a few posts ago, but again moving those pesky goal
posts when it suits you.

So which is it?

>I suspect its you who needs a bit more practice at writing large multi process
>and multi threaded applications.

*shrug* Feel free to look up some things that I've written, if
you like. Perhaps you'll learn something.

- Dan C.

cross@spitfire.i.gajendra.net (Dan Cross) writes:

[...]

> there is no reason you cannot, say, have a signal handler that
> broadcasts on a condition variable after an asynchronous IO operation
> completes, thus waking up a thread.

The pthread_cond_* calls are not async-signal safe and hence, this is
either undefined behaviour (newly introduced with POSIX.1-2024) or
undefined behaviour if the signal handler interrupted something that
isn't async-signal safe (prior to POSIX.1-2024 and still retained in the
current text).

However, POSIX semaphores can safely be used for that.

In article <677d4e48$0$28053$426a74cc@news.free.fr>,
Nicolas George <nicolas$george@salle-s.org> wrote:
>Dan Cross, dans le message <vljbvv$gl9$1@reader2.panix.com>, a �crit�:
>> This, however, does not follow. I don't see why "poll" is
>> strictly required for IO concurrency.
>
>Well, try to do implement anything non-trivial involving I/O concurrency,
>including timeouts, clients causing other clients to abort, etc., with
>the common denominator of POSIX threads and come back telling us how you
>managed that.

Well, this is rather more involved than what you'd originally
said, which was just IO concurrency.

But if you've got a dedicated IO thread with a known tid, I
don't see why you couldn't do this with
`pthread_cond_timedwait` and signals.

But at this point, I'll gladly admit that `poll` et al may be
more convenient, if not required.

>I tried, and stopped trying using threads for I/O concurrency.

I'm not saying it's easy.

- Dan C.

On Tue, 7 Jan 2025 16:02:51 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>In article <vljg72$28nj0$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>On Tue, 7 Jan 2025 14:13:29 -0000 (UTC)
>>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>>This is conflating multiple things. Most IO operations dealing
>>>with the actual hardware _are_ asynchronous (this is what
>>>McIlroy meant in the quote I posted earlier). The system call
>>>interface gives the program the illusion of those happening
>>>sequentially, but that's not how the devices really work.
>>
>>And? By your definition its still asynchronous programming.
>
>In the kernel, it sure is. Unix programmers have been writing
>asynchronous programs (using e.g. `fork`) since 1970.

Thats not what we're discussion here and you know it.

>>Please don't just make stuff up.
>
>Hmm. I wonder what shell you use, if you use Unix at all.

Stupid comments really are your forte arn't they.

>Here for example is the signal handler for SIGINT in bash:
>https://git.savannah.gnu.org/cgit/bash.git/tree/sig.c?h=devel#n691

Basically sets flags.

>Here's the SIGWINCH handler for good 'ol `script` from
>OpenBSD:
>https://github.com/openbsd/src/blob/6d253f95424ee0054c798f493d12377911cd3668/us
>r.bin/script/script.c#L224

Not a clever way to do it because an xterm and other terminal progs can
indirectly cause a whole load of SIGWINCH to be created if someone is resizing
it and only the final one really needs the ioctl call done. Better to set a
flag then manually do a call when appropriate.

>Those are just a few examples. If one cares to look, one will
>find many more in non-trivial programs used in production daily.

There are always exceptions to every rule. You seem to be so desperate to
win this argument I can only assume your fragile ego has been burst by
someone having the temerity to disagree with you. Tough, suck it up.

In article <87ttaa7gay.fsf@doppelsaurus.mobileactivedefense.com>,
Rainer Weikusat <rweikusat@talktalk.net> wrote:
>cross@spitfire.i.gajendra.net (Dan Cross) writes:
>
>[...]
>
>> there is no reason you cannot, say, have a signal handler that
>> broadcasts on a condition variable after an asynchronous IO operation
>> completes, thus waking up a thread.
>
>The pthread_cond_* calls are not async-signal safe and hence, this is
>either undefined behaviour (newly introduced with POSIX.1-2024) or
>undefined behaviour if the signal handler interrupted something that
>isn't async-signal safe (prior to POSIX.1-2024 and still retained in the
>current text).

You are correct; I was wrong about condvars. From:
https://pubs.opengroup.org/onlinepubs/9799919799/functions/pthread_cond_broadcast.html

|It is not safe to use the pthread_cond_signal() function in a
|signal handler that is invoked asynchronously. Even if it were
|safe, there would still be a race between the test of the
|Boolean pthread_cond_wait() that could not be efficiently
|eliminated.
| |Mutexes and condition variables are thus not suitable for
|releasing a waiting thread by signaling from code running in a
|signal handler.

(Curiously they make no mention of `pthread_cond_broadcast`
here; I suppose the same rationale applies.)

>However, POSIX semaphores can safely be used for that.

Another mechanism might be to have a thread blocked in `sigwait`
or `sigtimedwait` and use `pthread_kill` to signal it.

- Dan C.

On Tue, 7 Jan 2025 16:10:55 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>In article <vljiml$296n5$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>On Tue, 7 Jan 2025 15:35:44 -0000 (UTC)
>>>I think perhaps you should try to write some complex programs in
>>>the Unix environment before making such categorial statement.
>>
>>Don't be patronising.
>
>Wow, that's rich coming from you, my guy.

"My guy?" LOL, how old are you, 12? :)

>>I've probably written more unix software in 30 years
>>than you've had hot dinners including a fully featured telnetd and numerous
>>other servers for work and play. And in the places I've worked which included
>>finance/banking, aerospace and government,
>
>"For play" implies things that could be, or are, open source.
>So post a link to code, then.

Nope, I like my relative anonymity here and I don't need to prove anything to
some twat with a chip on his shoulder getting worked up over technical trivia.

>Bluntly, I don't believe that any of this is true. Your posts

Believe what you like, I couldn't give a rats arse.

>here show a distinct lack of relevant experience and knowledge.

Whatever you say genius.

>*shrug* Feel free to look up some things that I've written, if
>you like. Perhaps you'll learn something.

Is this yours?

https://github.com/dancrossnyc

Am I supposed to be impressed?

In article <vljjmf$g76$2@reader2.panix.com>,
Dan Cross <cross@spitfire.i.gajendra.net> wrote:
....
>"For play" implies things that could be, or are, open source.
>So post a link to code, then.
>
>Bluntly, I don't believe that any of this is true. Your posts
>here show a distinct lack of relevant experience and knowledge.

Or dementia. Lot of that going around in the world today.

Many Usenet posters fall into this category. They may have been
smart/accomplished once upon a time, but that was years/decades ago, and
now they're just living on memories.

Seriously, I've seen a lot of this over the years, and we should cut this
guy some slack.

--
After 4 years of disastrous screwups, Trump now favors 3 policies that I support:
1) $2K/pp stimulus money. Who doesn't want more money?
2) Water pressure. My shower doesn't work very well; I want Donnie to come fix it.
3) Repeal of Section 230. This will lead to the demise of Face/Twit/Gram. Yey!

In article <vljmc6$29tkd$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Tue, 7 Jan 2025 16:02:51 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vljg72$28nj0$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>On Tue, 7 Jan 2025 14:13:29 -0000 (UTC)
>>>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>>>This is conflating multiple things. Most IO operations dealing
>>>>with the actual hardware _are_ asynchronous (this is what
>>>>McIlroy meant in the quote I posted earlier). The system call
>>>>interface gives the program the illusion of those happening
>>>>sequentially, but that's not how the devices really work.
>>>
>>>And? By your definition its still asynchronous programming.
>>
>>In the kernel, it sure is. Unix programmers have been writing
>>asynchronous programs (using e.g. `fork`) since 1970.
>
>Thats not what we're discussion here and you know it.

Actually, it is.

>>>Please don't just make stuff up.
>>
>>Hmm. I wonder what shell you use, if you use Unix at all.
>
>Stupid comments really are your forte arn't they.

I see that you can't support your argument.

>>Here for example is the signal handler for SIGINT in bash:
>>https://git.savannah.gnu.org/cgit/bash.git/tree/sig.c?h=devel#n691
>
>Basically sets flags.

Did you actually read and understand any of that code?

>>Here's the SIGWINCH handler for good 'ol `script` from
>>OpenBSD:
>>https://github.com/openbsd/src/blob/6d253f95424ee0054c798f493d12377911cd3668/us
>>r.bin/script/script.c#L224
>
>Not a clever way to do it because an xterm and other terminal progs can
>indirectly cause a whole load of SIGWINCH to be created if someone is resizing
>it and only the final one really needs the ioctl call done. Better to set a
>flag then manually do a call when appropriate.

Ok. You may even be right! But tell me: where would you check
those flags?

Regardless, here you are, again, moving the goalposts in the
face of evidence that contradicted your earlier position.

>>Those are just a few examples. If one cares to look, one will
>>find many more in non-trivial programs used in production daily.
>
>There are always exceptions to every rule. You seem to be so desperate to
>win this argument I can only assume your fragile ego has been burst by
>someone having the temerity to disagree with you. Tough, suck it up.

Ah, here we go. The classic attempt at an insult.

Look, you made categorical, definitive statements. Those
statements were factually incorrect. I pointed that out. You
seem to be pretty upset about that and want to argument the
point, no matter how much evidence to the contrary you are
presented with.

Perhaps I am not the one with the fragile ego that needs to suck
it up when disagreed with.

- Dan C.

In article <vljmlt$29vt3$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>On Tue, 7 Jan 2025 16:10:55 -0000 (UTC)
>cross@spitfire.i.gajendra.net (Dan Cross) wibbled:
>>In article <vljiml$296n5$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>>>On Tue, 7 Jan 2025 15:35:44 -0000 (UTC)
>>>>I think perhaps you should try to write some complex programs in
>>>>the Unix environment before making such categorial statement.
>>>
>>>Don't be patronising.
>>
>>Wow, that's rich coming from you, my guy.
>
>"My guy?" LOL, how old are you, 12? :)
>
>>>I've probably written more unix software in 30 years
>>>than you've had hot dinners including a fully featured telnetd and numerous
>>>other servers for work and play. And in the places I've worked which included
>>>finance/banking, aerospace and government,
>>
>>"For play" implies things that could be, or are, open source.
>>So post a link to code, then.
>
>Nope, I like my relative anonymity here and I don't need to prove anything to
>some twat with a chip on his shoulder getting worked up over technical trivia.

Ok. So we're just supposed to take your word for it, I guess.
Got it.

>>Bluntly, I don't believe that any of this is true. Your posts
>
>Believe what you like, I couldn't give a rats arse.

You also have no evidence to back up your claims, it seems.

>>here show a distinct lack of relevant experience and knowledge.
>
>Whatever you say genius.
>
>>*shrug* Feel free to look up some things that I've written, if
>>you like. Perhaps you'll learn something.
>
>Is this yours?
>
>https://github.com/dancrossnyc

Yup, that's me.

>Am I supposed to be impressed?

*shrug* I think my credentials speak for themselves. I really
don't care whether you're impressed or not.

- Dan C.

In article <vGbfP.54357$XfF8.7280@fx04.iad>,
Scott Lurndal <slp53@pacbell.net> wrote:
>cross@spitfire.i.gajendra.net (Dan Cross) writes:
>>In article <vlioum$24bqm$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:
>
>>>Thats not proper asych, its still sequential. Proper asynch is when the
>>>program execution path is directly modified by external events. Otherwise
>>>you could claim simply using the standard file I/O system is asynchronous
>>>programming as there's no guarantee that any data has been written to the disk
>>>before write(), fprintf() etc return.
>>
>>This is conflating multiple things. Most IO operations dealing
>>with the actual hardware _are_ asynchronous (this is what
>>McIlroy meant in the quote I posted earlier). The system call
>>interface gives the program the illusion of those happening
>>sequentially, but that's not how the devices really work.
>
>Indeed, and it was subsequently recognized that more than
>the 'sync'[*] system call was required for applications to
>ensure data was successfully written to the underlying
>device.

Yup.

>Applications in those days (e.g. fsck) would access the
>raw character device using the unbuffered read() and
>write() system calls rather than using stdio. A key
>characteristic of raw devices were that the hardware DMA would
>use the application buffer directly rather than copying
>the data to the kernel buffer pool first.

They still do!

>[*] I recall using 'sync;sync;sync' from the Sixth Edition
>command line more than once, before rebooting.

I've always felt the triple-sync thing was kind of superstition.

But this makes some sense when one considers that a sync would
trigger additional IO requests that would be scheduled, but
not necessarily completed.

Still, once disk drivers started optimizing write order and
thus divorcing ordering of requests to the device from the
chronological order the requests arrived in, all bets were out
the window.

>Subsequently APIs like fdatasync(2) and open flags
>such as O_DIRECT and O_DSYNC were added.
>
>>It turns out the simple model of early research Unix was
>>insufficient for handling all sorts of important use cases,
>>hence why interfaces like `select` and `poll` were added.
>
>Although to be fair, select(2) originated with BSD and is
>a bit less flexible than poll(2).

This is fair.

- Dan C.

In article <vljnhd$2mih3$1@news.xmission.com>,
Kenny McCormack <gazelle@shell.xmission.com> wrote:
>In article <vljjmf$g76$2@reader2.panix.com>,
>Dan Cross <cross@spitfire.i.gajendra.net> wrote:
>...
>>"For play" implies things that could be, or are, open source.
>>So post a link to code, then.
>>
>>Bluntly, I don't believe that any of this is true. Your posts
>>here show a distinct lack of relevant experience and knowledge.
>
>Or dementia. Lot of that going around in the world today.
>
>Many Usenet posters fall into this category. They may have been
>smart/accomplished once upon a time, but that was years/decades ago, and
>now they're just living on memories.
>
>Seriously, I've seen a lot of this over the years, and we should cut this
>guy some slack.

A lot of USENET folks talk the talk without ever having walked
the walk, but you could be right.

- Dan C.

cross@spitfire.i.gajendra.net (Dan Cross) writes:
>In article <vGbfP.54357$XfF8.7280@fx04.iad>,
>Scott Lurndal <slp53@pacbell.net> wrote:
>>cross@spitfire.i.gajendra.net (Dan Cross) writes:
>>>In article <vlioum$24bqm$1@dont-email.me>, <Muttley@DastardlyHQ.org> wrote:

>>Applications in those days (e.g. fsck) would access the
>>raw character device using the unbuffered read() and
>>write() system calls rather than using stdio. A key
>>characteristic of raw devices were that the hardware DMA would
>>use the application buffer directly rather than copying
>>the data to the kernel buffer pool first.
>
>They still do!

Well, not on linux. Even O_DIRECT still goes through
the file cache, last I checked.

I submitted a 'raw device' patch to the linux mailing
list in the late 90s while at SGI. It wasn't accepted because
O_DIRECT was considered sufficient, even with the spurious
copy. The overhead of pinning the user space pages was
considered onerous.