On 11/16/24 1:16 AM, rbowman wrote:
> On Sat, 16 Nov 2024 00:37:35 -0500, 186282@ud0s4.net wrote:
>
>> IF you are doing multi-threaded/process style pgms then you almost
>> HAVE to use them if you want to xmit more than a few bytes between
>> parents/children. A few years back I was experimenting with TCP/UDP
>> servers and found that it could be very useful to transmit a few
>> dozen bytes worth of active data between parents/children.
>>
>> SO ... learn yer pipes !
>
> Message queues and shared memory are handy for IPC too. I don't think
> Microsoft ever figured either out. What they call message queues are
> completely different.

Whatever M$ does ... well ....... always seems
to be FUCKED somehow.

Anyway, pipes are VERY useful. Anybody getting into 'C',
esp multi-threaded/process, needs to get very familiar
with them. Even in 'C' they're still kinda awkward
alas. Somebody had The Idea, but nobody seems to have
made it smooth and consistent across multiple languages
and systems.

My old server experiments, each child returned stats
back to the root parent while it was running. Under
certain conditions the parent would do things - maybe
terminate the child or connection or do a timer
process. Did work up to a fairly compact UDP bi-di
pre-threaded server. Never found a great use for it,
but it was fun and maybe someday .......

Louis Krupp <lkrupp@invalid.pssw.com.invalid> writes:
> On 11/15/2024 12:49 AM, 186282@ud0s4.net wrote:
>
> On 11/14/24 9:05 PM, Lawrence D'Oliveiro wrote:
>
> On Thu, 14 Nov 2024 16:09:07 -0800, John Ames wrote:
>
> (I think it was a topic shift to applications of *nix pipes...?)
>
> Oh shelly boy, the pipes, the pipes are calling ...
>
> Pipes are good.
>
> But, really, they're just temp files the parent
> process can access.
>
> Pipes *could* be implemented with temporary disk files, at least to an
> extent, but as far as I can tell, they're not.

“Temporary files the parent can access” is not a good model for a couple
of reasons:

* The semantics are different. Bytes read from a regular file are still
there to be read again if you rewind the file; bytes read from a pipe
are gone. Empty/full files behave differently to empty/full pipes.

* Temporary regular files can already be accessed by other processes by
name, by file descriptor inheritance, or file descriptor passing.

--
https://www.greenend.org.uk/rjk/

On 2024-11-16, rbowman <bowman@montana.com> wrote:

> On Sat, 16 Nov 2024 00:37:35 -0500, 186282@ud0s4.net wrote:
>
>> IF you are doing multi-threaded/process style pgms then you almost
>> HAVE to use them if you want to xmit more than a few bytes between
>> parents/children. A few years back I was experimenting with TCP/UDP
>> servers and found that it could be very useful to transmit a few
>> dozen bytes worth of active data between parents/children.
>>
>> SO ... learn yer pipes !

Our software has two modules: one runs 24/7 collecting data, while the
other handles the number crunching and user interface. The original
MS-DOS version ran the collector as a TSR. When I ported it to SCO
Unix I used named pipes.

(BTW MS-DOS and Windows implement command-line pipes using work files.)

> Message queues and shared memory are handy for IPC too. I don't think
> Microsoft ever figured either out. What they call message queues are
> completely different.

I originally played around with those methods when doing the Windows
version, but there were various drawbacks with whatever method I tried.
The DDEML version could take down the supposedly unkillable NT supervisor.

But then sockets came along, and life has been good ever since.

--
/~\ Charlie Gibbs | Growth for the sake of
\ / <cgibbs@kltpzyxm.invalid> | growth is the ideology
X I'm really at ac.dekanfrus | of the cancer cell.
/ \ if you read it the right way. | -- Edward Abbey

On 2024-11-16, 186282@ud0s4.net <186283@ud0s4.net> wrote:

> Whatever M$ does ... well ....... always seems
> to be FUCKED somehow.

"The day Microsoft makes a product that doesn't suck
is the day they make vacuum cleaners."

--
/~\ Charlie Gibbs | Growth for the sake of
\ / <cgibbs@kltpzyxm.invalid> | growth is the ideology
X I'm really at ac.dekanfrus | of the cancer cell.
/ \ if you read it the right way. | -- Edward Abbey

On 11/16/24 1:14 PM, Charlie Gibbs wrote:
> On 2024-11-16, 186282@ud0s4.net <186283@ud0s4.net> wrote:
>
>> Whatever M$ does ... well ....... always seems
>> to be FUCKED somehow.
>
> "The day Microsoft makes a product that doesn't suck
> is the day they make vacuum cleaners."

LIKE that one !!! :-)

On 11/16/24 5:31 AM, Richard Kettlewell wrote:
> Louis Krupp <lkrupp@invalid.pssw.com.invalid> writes:
>> On 11/15/2024 12:49 AM, 186282@ud0s4.net wrote:
>>
>> On 11/14/24 9:05 PM, Lawrence D'Oliveiro wrote:
>>
>> On Thu, 14 Nov 2024 16:09:07 -0800, John Ames wrote:
>>
>> (I think it was a topic shift to applications of *nix pipes...?)
>>
>> Oh shelly boy, the pipes, the pipes are calling ...
>>
>> Pipes are good.
>>
>> But, really, they're just temp files the parent
>> process can access.
>>
>> Pipes *could* be implemented with temporary disk files, at least to an
>> extent, but as far as I can tell, they're not.
>
> “Temporary files the parent can access” is not a good model for a couple
> of reasons:
>
> * The semantics are different. Bytes read from a regular file are still
> there to be read again if you rewind the file; bytes read from a pipe
> are gone. Empty/full files behave differently to empty/full pipes.
>
> * Temporary regular files can already be accessed by other processes by
> name, by file descriptor inheritance, or file descriptor passing.

Didn't say temp disk files were necessarily SUPERIOR, just
that they CAN do most of the stuff pipes are used for and
in an easier user/code-friendly fashion. SIZE is almost
unlimited too.

Anyway, I'd made some TCP/UDP servers but the parent needed
to know the status of the children in a little more detail
than just a tiny return code - might want to terminate the
child if it'd been idle for awhile, might want to know how
much bandwidth was being used, might want to pick a good
time to do some maint routines, might want to know WHO the
connections were from. Pipes were the (relatively) easy
'C'-approved method and did the job very well.

So, for most, use pipes. Python and FPC also do pipes.
They're INTENDED for inter-process communications after
all so generally GO that way.

The Bi-Di server was the most fun ... think a "chat" app
but with the potential for more than just text - closer
to 'remote terminal'/SSH. ALMOST slipped into re-creating
an FTP app until I realized where it was going. Too much
enthusiasm. Commented-out the remote root system call
function :-)

Amazing how LITTLE code is required for such things.
Even the pre-threaded version (max cap) wasn't huge
at all (albeit the hardest to 'hold in yer head' as
to how it worked). Base examples in several langs easily
found on the net - just build-on as necessary and
then party on.

On Sat, 16 Nov 2024 23:59:41 -0500, 186282@ud0s4.net wrote:

> On 11/16/24 5:31 AM, Richard Kettlewell wrote:
>>
>> “Temporary files the parent can access” is not a good model for a
>> couple of reasons:
>>
>> * The semantics are different. Bytes read from a regular file are still
>> there to be read again if you rewind the file; bytes read from a
>> pipe are gone. Empty/full files behave differently to empty/full
>> pipes.
>>
>> * Temporary regular files can already be accessed by other processes by
>> name, by file descriptor inheritance, or file descriptor passing.
>
> Didn't say temp disk files were necessarily SUPERIOR, just that they
> CAN do most of the stuff pipes are used for and in an easier
> user/code-friendly fashion.

Not really, no.

> SIZE is almost unlimited too.

That is another reason they don’t work.

On 11/17/24 12:30 AM, Lawrence D'Oliveiro wrote:
> On Sat, 16 Nov 2024 23:59:41 -0500, 186282@ud0s4.net wrote:
>
>> On 11/16/24 5:31 AM, Richard Kettlewell wrote:
>>>
>>> “Temporary files the parent can access” is not a good model for a
>>> couple of reasons:
>>>
>>> * The semantics are different. Bytes read from a regular file are still
>>> there to be read again if you rewind the file; bytes read from a
>>> pipe are gone. Empty/full files behave differently to empty/full
>>> pipes.
>>>
>>> * Temporary regular files can already be accessed by other processes by
>>> name, by file descriptor inheritance, or file descriptor passing.
>>
>> Didn't say temp disk files were necessarily SUPERIOR, just that they
>> CAN do most of the stuff pipes are used for and in an easier
>> user/code-friendly fashion.
>
> Not really, no.
>
>> SIZE is almost unlimited too.
>
> That is another reason they don’t work.

It's a block of info, just by one means over another.
any sub-process/thread can write to a file as easily
as a pipe. Just need a good naming scheme.

But, as said, better to stick to pipes for most needs.

If you're passing multi-megabyte pages of text/images
or whatever back to the parent then you're doing
something wrong. Don't hassle yer parents ! :-)

On Sun, 17 Nov 2024 01:05:20 -0500, 186282@ud0s4.net wrote:

> On 11/17/24 12:30 AM, Lawrence D'Oliveiro wrote:
>> On Sat, 16 Nov 2024 23:59:41 -0500, 186282@ud0s4.net wrote:
>>
>>> On 11/16/24 5:31 AM, Richard Kettlewell wrote:
>>>>
>>>> “Temporary files the parent can access” is not a good model for a
>>>> couple of reasons:
>>>>
>>>> * The semantics are different. Bytes read from a regular file are
>>>> still there to be read again if you rewind the file; bytes read
>>>> from a pipe are gone. Empty/full files behave differently to
>>>> empty/full pipes.
>>>>
>>>> * Temporary regular files can already be accessed by other processes
>>>> by name, by file descriptor inheritance, or file descriptor passing.
>>>
>>> Didn't say temp disk files were necessarily SUPERIOR, just that they
>>> CAN do most of the stuff pipes are used for and in an easier
>>> user/code-friendly fashion.
>>
>> Not really, no.
>>
>>> SIZE is almost unlimited too.
>>
>> That is another reason they don’t work.
>
> It's a block of info, just by one means over another.

You’re forgetting the sequencing and synchronization aspects.

On 11/17/24 2:59 AM, Lawrence D'Oliveiro wrote:
> On Sun, 17 Nov 2024 01:05:20 -0500, 186282@ud0s4.net wrote:
>
>> On 11/17/24 12:30 AM, Lawrence D'Oliveiro wrote:
>>> On Sat, 16 Nov 2024 23:59:41 -0500, 186282@ud0s4.net wrote:
>>>
>>>> On 11/16/24 5:31 AM, Richard Kettlewell wrote:
>>>>>
>>>>> “Temporary files the parent can access” is not a good model for a
>>>>> couple of reasons:
>>>>>
>>>>> * The semantics are different. Bytes read from a regular file are
>>>>> still there to be read again if you rewind the file; bytes read
>>>>> from a pipe are gone. Empty/full files behave differently to
>>>>> empty/full pipes.
>>>>>
>>>>> * Temporary regular files can already be accessed by other processes
>>>>> by name, by file descriptor inheritance, or file descriptor passing.
>>>>
>>>> Didn't say temp disk files were necessarily SUPERIOR, just that they
>>>> CAN do most of the stuff pipes are used for and in an easier
>>>> user/code-friendly fashion.
>>>
>>> Not really, no.
>>>
>>>> SIZE is almost unlimited too.
>>>
>>> That is another reason they don’t work.
>>
>> It's a block of info, just by one means over another.
>
> You’re forgetting the sequencing and synchronization aspects.

What ? Can't figure that out ? :-)

Anyway, again as said, I agree that pipes ARE
generally much better for these sorts of
purposes.

You seem to be making a war here where none
actually exists.

"186282@ud0s4.net" <186283@ud0s4.net> writes:
> On 11/16/24 5:31 AM, Richard Kettlewell wrote:
>> Louis Krupp <lkrupp@invalid.pssw.com.invalid> writes:

>>>> But, really, they're just temp files the parent process can access.
>>>
>>> Pipes *could* be implemented with temporary disk files, at least to an
>>> extent, but as far as I can tell, they're not.
>>
>> “Temporary files the parent can access” is not a good model for a
>> couple of reasons:
>>
>> * The semantics are different. Bytes read from a regular file are
>> still there to be read again if you rewind the file; bytes read from
>> a pipe are gone. Empty/full files behave differently to empty/full
>> pipes.
>>
>> * Temporary regular files can already be accessed by other processes
>> by name, by file descriptor inheritance, or file descriptor passing.
>
> Didn't say temp disk files were necessarily SUPERIOR,

Indeed. You said they’re really the same, which they’re not.

> just that they CAN do most of the stuff pipes are used for and in an
> easier user/code-friendly fashion.

There’s certainly an overlap in the end goals you can meet with them,
but even within that overlap they’re used in slightly different ways.

> SIZE is almost unlimited too.

Capacity is one of the ways in which they are very different, already
alluded to above.

> Anyway, I'd made some TCP/UDP servers but the parent needed
> to know the status of the children in a little more detail
> than just a tiny return code - might want to terminate the
> child if it'd been idle for awhile, might want to know how
> much bandwidth was being used, might want to pick a good
> time to do some maint routines, might want to know WHO the
> connections were from. Pipes were the (relatively) easy
> 'C'-approved method and did the job very well.

I’ve no idea what you think “’C’-approved” means. C is a programming
language, not a set of guidelines. Pipes are an OS concept, not
appearing in the C language standard.

--
https://www.greenend.org.uk/rjk/

On 11/17/24 3:42 AM, Richard Kettlewell wrote:
> "186282@ud0s4.net" <186283@ud0s4.net> writes:
>> On 11/16/24 5:31 AM, Richard Kettlewell wrote:
>>> Louis Krupp <lkrupp@invalid.pssw.com.invalid> writes:
>
>>>>> But, really, they're just temp files the parent process can access.
>>>>
>>>> Pipes *could* be implemented with temporary disk files, at least to an
>>>> extent, but as far as I can tell, they're not.
>>>
>>> “Temporary files the parent can access” is not a good model for a
>>> couple of reasons:
>>>
>>> * The semantics are different. Bytes read from a regular file are
>>> still there to be read again if you rewind the file; bytes read from
>>> a pipe are gone. Empty/full files behave differently to empty/full
>>> pipes.
>>>
>>> * Temporary regular files can already be accessed by other processes
>>> by name, by file descriptor inheritance, or file descriptor passing.
>>
>> Didn't say temp disk files were necessarily SUPERIOR,
>
> Indeed. You said they’re really the same, which they’re not.

Umm ... they're all "blocks of information", however
stored/accessed. The little details are, well, the
little details.

>> just that they CAN do most of the stuff pipes are used for and in an
>> easier user/code-friendly fashion.
>
> There’s certainly an overlap in the end goals you can meet with them,
> but even within that overlap they’re used in slightly different ways.
>
>> SIZE is almost unlimited too.
>
> Capacity is one of the ways in which they are very different, already
> alluded to above.

As I said somewhere, if your child processes are sending
megabytes back to the parent you're DOING SOMETHING WRONG.
Using files -vs- pipes you CAN xmit mass quantities of
data between parent/child and or child/child but that's
probably not the best way to write your app. Leave the
parent to do 'parent stuff' mostly.

But somebody, somewhere, for some reason, MIGHT wanna
do it differently ... and maybe, in context, it's for
a good reason. I'm not gonna piss all over them.

>> Anyway, I'd made some TCP/UDP servers but the parent needed
>> to know the status of the children in a little more detail
>> than just a tiny return code - might want to terminate the
>> child if it'd been idle for awhile, might want to know how
>> much bandwidth was being used, might want to pick a good
>> time to do some maint routines, might want to know WHO the
>> connections were from. Pipes were the (relatively) easy
>> 'C'-approved method and did the job very well.
>
> I’ve no idea what you think “’C’-approved” means. C is a programming
> language, not a set of guidelines. Pipes are an OS concept, not
> appearing in the C language standard.

The 'C' Way IS a sort of 'philosophy' I think ... an
approach to programming and data-use. Most every lang
is that way. The Algol/Pascal/Modula approach is a bit
different, another 'look and feel' to all the issues.
Everybody has what they think is "The Best Way" - of
thinking/doing. Otherwise we shoulda all stuck with ASM.
No FORTRAN, no COBOL, no LISP, no Algol, no 'C' ....

The way 'C' does interprocess comms was writ in, by and
for 'C'. As such if you're doing 'C' you will have the
best luck and least angst going with the flow.

You can also send/receive between parent/child and
such by passing socket info. See :

https://stackoverflow.com/questions/14427898/how-can-i-pass-a-socket-from-parent-to-child-processes

On Sun, 17 Nov 2024 22:50:41 -0500, 186282@ud0s4.net wrote:

> As I said somewhere, if your child processes are sending megabytes
> back to the parent you're DOING SOMETHING WRONG.

Says the one who has no clue how to write real-world programs.

On 11/18/24 1:06 AM, Lawrence D'Oliveiro wrote:
> On Sun, 17 Nov 2024 22:50:41 -0500, 186282@ud0s4.net wrote:
>
>> As I said somewhere, if your child processes are sending megabytes
>> back to the parent you're DOING SOMETHING WRONG.
>
> Says the one who has no clue how to write real-world programs.

Am I supposed to say something horrible about
you now ?

Nope. Won't. I'm sick of 'The Wars'.

Clue - I've writ 'real world programs' and
got paid well for them from the punch-card
days on. I did it my way, for my reasons,
my 'vision'.

And so did you.

On 11/18/24 06:51, 186282@ud0s4.net wrote:
> On 11/18/24 1:06 AM, Lawrence D'Oliveiro wrote:
>> On Sun, 17 Nov 2024 22:50:41 -0500, 186282@ud0s4.net wrote:
>>
>>> As I said somewhere, if your child processes are sending megabytes
>>> back to the parent you're DOING SOMETHING WRONG.
>>
>> Says the one who has no clue how to write real-world programs.
>
>   Am I supposed to say something horrible about
>   you now ?
>
>   Nope. Won't. I'm sick of 'The Wars'.
>
>   Clue - I've writ 'real world programs' and
>   got paid well for them from the punch-card
>   days on. I did it my way, for my reasons,
>   my 'vision'.
>
>   And so did you.

I mainly did as I was told. In large political organisations following
your own vision can be problematic. You have to fight for a vision.

I've no idea why using IPC to send megabytes of data between different
processes is wrong. That is what many service-orientated architectures
do. The moment you have a persistence service it is likely to happen.

Although to be fair we very rarely used pipes, directly, almost never.
It was always something like REST or message queues.

On Mon, 18 Nov 2024 09:20:23 +0000, Pancho wrote:

> I've no idea why using IPC to send megabytes of data between different
> processes is wrong.

It is something I have done. It’s perfectly commonplace.

> Although to be fair we very rarely used pipes, directly, almost never.

I have used pipes, I have used Unix sockets, I have used network sockets.

If you are running a Linux GUI, then almost certainly it is built on D-Bus
as a high-level IPC mechanism that is used as a core component. That is
designed to run over Unix sockets. It is not itself designed for high-
bandwidth data transfers; if you want to do that, you can exchange your
own D-Bus messages to set up custom pipe or Unix socket connections
between bus peers.

> It was always something like REST or message queues.

Message queues are an OS-provided primitive, but REST is not -- that is a
protocol, not a transport. What transport(s) did you use for that? I would
assume network connections.

On 11/18/24 09:36, Lawrence D'Oliveiro wrote:
> On Mon, 18 Nov 2024 09:20:23 +0000, Pancho wrote:
>
>> I've no idea why using IPC to send megabytes of data between different
>> processes is wrong.
>
> It is something I have done. It’s perfectly commonplace.
>
>> Although to be fair we very rarely used pipes, directly, almost never.
>
> I have used pipes, I have used Unix sockets, I have used network sockets.
>
> If you are running a Linux GUI, then almost certainly it is built on D-Bus
> as a high-level IPC mechanism that is used as a core component. That is
> designed to run over Unix sockets. It is not itself designed for high-
> bandwidth data transfers; if you want to do that, you can exchange your
> own D-Bus messages to set up custom pipe or Unix socket connections
> between bus peers.
>
>> It was always something like REST or message queues.
>
> Message queues are an OS-provided primitive, but REST is not -- that is a
> protocol, not a transport. What transport(s) did you use for that? I would
> assume network connections.

I don't know if Linux provides message queues as a primitive, or not. I
meant I used message queue protocols (MQTT,zeroMQ, ActiveMQ).

I guess I was saying as a software developer I only cared about the
application layer, I never went down to the level of using pipes
directly. I would have thought my experience was reasonably normal, for
the last few decades. The period after software developers needed to
roll everything themselves.

I do remember looking at named pipes and deciding it was appropriate for
some task, but I have no memory of actually implementing it. Which could
mean it just worked and I never thought about it again, or it could mean
I didn't do it.

On 18/11/2024 11:05, Pancho wrote:
> I do remember looking at named pipes and deciding it was appropriate for
> some task, but I have no memory of actually implementing it. Which could
> mean it just worked and I never thought about it again, or it could mean
> I didn't do it.

Similar here.
I've tended to use ramdisk files ... for interprocess communication. Not
idiot proof, but can be made fit for purpose.

--
No Apple devices were knowingly used in the preparation of this post.

On 11/18/24 13:45, The Natural Philosopher wrote:
> On 18/11/2024 11:05, Pancho wrote:
>> I do remember looking at named pipes and deciding it was appropriate
>> for some task, but I have no memory of actually implementing it. Which
>> could mean it just worked and I never thought about it again, or it
>> could mean I didn't do it.
>
> Similar here.
> I've tended to use ramdisk files ... for interprocess communication. Not
> idiot proof, but can be made fit for purpose.
>

I guess simpler than shared memory sections.

One of my golden rules of programming was - show me some RAM and I will
squander it on a cache. :-) Although, my crime was mainly in memory
databases.

On 18/11/2024 17:04, Pancho wrote:
> On 11/18/24 13:45, The Natural Philosopher wrote:
>> On 18/11/2024 11:05, Pancho wrote:
>>> I do remember looking at named pipes and deciding it was appropriate
>>> for some task, but I have no memory of actually implementing it.
>>> Which could mean it just worked and I never thought about it again,
>>> or it could mean I didn't do it.
>>
>> Similar here.
>> I've tended to use ramdisk files ... for interprocess communication.
>> Not idiot proof, but can be made fit for purpose.
>>
>
> I guess simpler than shared memory sections.

A ram disk IS effectively a shared memory section.
But with well known ways to access it - and well known issues about race
conditions if two+ processes try to write to it
In the end the issues are just the same, only the coding and sometimes
access speed is different.

>
> One of my golden rules of programming was - show me some RAM and I will
> squander it on a cache. :-) Although, my crime was mainly in memory
> databases.
>
RAM space has never ever been as problem for me when using low level
languages.

>

--
Renewable energy: Expensive solutions that don't work to a problem that
doesn't exist instituted by self legalising protection rackets that
don't protect, masquerading as public servants who don't serve the public.

On Mon, 18 Nov 2024 11:05:03 +0000, Pancho wrote:

> I don't know if Linux provides message queues as a primitive, or not. I
> meant I used message queue protocols (MQTT,zeroMQ, ActiveMQ).

https://www.softprayog.in/programming/interprocess-communication-using-
system-v-message-queues-in-linux

https://www.softprayog.in/programming/interprocess-communication-using-
posix-message-queues-in-linux

I've only used System V queues. Neither work like ActiveMQ.

On Mon, 18 Nov 2024 17:04:35 +0000, Pancho wrote:

> I guess simpler than shared memory sections.
>
> One of my golden rules of programming was - show me some RAM and I will
> squander it on a cache. Although, my crime was mainly in memory
> databases.

https://www.softprayog.in/programming/interprocess-communication-using-
system-v-shared-memory-in-linux

Shared memory may be faster.

On 11/18/24 4:20 AM, Pancho wrote:
> On 11/18/24 06:51, 186282@ud0s4.net wrote:
>> On 11/18/24 1:06 AM, Lawrence D'Oliveiro wrote:
>>> On Sun, 17 Nov 2024 22:50:41 -0500, 186282@ud0s4.net wrote:
>>>
>>>> As I said somewhere, if your child processes are sending megabytes
>>>> back to the parent you're DOING SOMETHING WRONG.
>>>
>>> Says the one who has no clue how to write real-world programs.
>>
>>    Am I supposed to say something horrible about
>>    you now ?
>>
>>    Nope. Won't. I'm sick of 'The Wars'.
>>
>>    Clue - I've writ 'real world programs' and
>>    got paid well for them from the punch-card
>>    days on. I did it my way, for my reasons,
>>    my 'vision'.
>>
>>    And so did you.
>
> I mainly did as I was told. In large political organisations following
> your own vision can be problematic. You have to fight for a vision.

I very intentionally stayed away from such orgs. A little
less money, but the happiness factor more than compensated.
Got to pursue many esoteric projects - some of which WERE
useful - and the org paid for most of it (except the odd
Radio Shack parts).

> I've no idea why using IPC to send megabytes of data between different
> processes is wrong. That is what many service-orientated architectures
> do. The moment you have a persistence service it is likely to happen.

Really you DON'T want to send megabytes as IPC ... but
somebody somewhere MAY wanna. So, what's the best way
for them to do that ? Don't necessarily piss on 'em but
help 'em out instead.

> Although to be fair we very rarely used pipes, directly, almost never.
> It was always something like REST or message queues.

Again though, just "blocks of information". MANY ways
to do that. Each developer has to figure out what's
best for THEIR particular app.

I mentioned a server style app I made, but I wanted the
child processes/threads (tried both ways) to return
some active stat data to the parent. Not a huge amount
of info, but useful. Pipes were the best way. The
parent was allowed to xmit a 'kill' to a child in
trouble, via the pipe, so it could shut down in an
orderly fashion.

Pre-threaded servers seem to have the highest capacity
and throughput. The parent allocates maybe ten threads
to start off with and then can add new blocks of ten
as-needed. It's a little harder to manage than some
other kinds of servers, and sometimes a bit harder
to visualize how it's working, but they're good things.
Once you've got it you're SET.

On 11/18/24 8:45 AM, The Natural Philosopher wrote:
> On 18/11/2024 11:05, Pancho wrote:
>> I do remember looking at named pipes and deciding it was appropriate
>> for some task, but I have no memory of actually implementing it. Which
>> could mean it just worked and I never thought about it again, or it
>> could mean I didn't do it.
>
> Similar here.
> I've tended to use ramdisk files ... for interprocess communication. Not
> idiot proof, but can be made fit for purpose.

As I was saying all along ... yer passing "blocks
of information" between processes. CAN be done
in a large number of ways and they'll all WORK.
Temporary files on a RAMdisk are fine (but slower).
Pipes are fine, the "normal 'C' way". MQs will
work. In any case it'll get done.

Dunno why everybody seems so keen to slag anybody
who doesn't do it/see it exactly like some high-holy
manual from 1982 says you're supposed to. That's
no fun !

Hmmm ... can you pass IPC data using paper tape ?
It'd be cool ! :-)

On 2024-11-18, 186282@ud0s4.net <186283@ud0s4.net> wrote:
>...
>
> As I said somewhere, if your child processes are sending
> megabytes back to the parent you're DOING SOMETHING WRONG.

Oh?

Let's take my example of an older Raspberry Pi model 1 with less
than 1GB of RAM and only 4GB of SD card "disk", and process a
video stream several GB in total length, from an mplayer process
that will write only to a file and refuses to write to stdout,
where the stream needs to be sent over the LAN to a machine with
enough spinning rust to store the stream.

Would temporary files work? Not really. A named pipe let
mplayer write to a named file. Even if having two processes
shuffling the data wasn't the most efficient, the Pi handled it
quite well.

--
Robert Riches
spamtrap42@jacob21819.net
(Yes, that is one of my email addresses.)