Rocksolid Light

News from da outaworlds

mail  files  register  groups  login

Message-ID:  

BOFH excuse #2: solar flares


comp / comp.os.linux.misc / Re: Named pipes vs. Unix sockets

SubjectAuthor
* Are We Back to the "Wars" Now ?186282@ud0s4.net
+* Re: Are We Back to the "Wars" Now ?D
|`* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
| `- Re: Are We Back to the "Wars" Now ?D
+- Re: Are We Back to the "Wars" Now ?Pancho
+- Re: Are We Back to the "Wars" Now ?Richard Kettlewell
+* Re: Are We Back to the "Wars" Now ?Rich
|`* Re: Are We Back to the "Wars" Now ?Phillip Frabott
| `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|  `* Re: Are We Back to the "Wars" Now ?Phillip Frabott
|   `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|    `* Re: Are We Back to the "Wars" Now ?Phillip Frabott
|     +* Re: Are We Back to the "Wars" Now ?Richard Kettlewell
|     |`* Re: Are We Back to the "Wars" Now ?Phillip Frabott
|     | +* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|     | |`* Re: Are We Back to the "Wars" Now ?vallor
|     | | +* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|     | | |`* Re: Are We Back to the "Wars" Now ?vallor
|     | | | `* Named pipes vs. Unix sockets (was: Re: Are We Back to the "Wars" Now ?)vallor
|     | | |  `* Re: Named pipes vs. Unix socketsvallor
|     | | |   +- Re: Named pipes vs. Unix socketsLawrence D'Oliveiro
|     | | |   `* Re: Named pipes vs. Unix socketsPhillip Frabott
|     | | |    +* Re: Named pipes vs. Unix socketsLawrence D'Oliveiro
|     | | |    |`* Re: Named pipes vs. Unix socketsPhillip Frabott
|     | | |    | `- Re: Named pipes vs. Unix socketsLawrence D'Oliveiro
|     | | |    `* Re: Named pipes vs. Unix socketsRichard L. Hamilton
|     | | |     `* Re: Named pipes vs. Unix socketsLawrence D'Oliveiro
|     | | |      `- Re: Named pipes vs. Unix socketsRichard L. Hamilton
|     | | `* Re: Are We Back to the "Wars" Now ?Rich
|     | |  +* Re: Are We Back to the "Wars" Now ?vallor
|     | |  |+- Re: Are We Back to the "Wars" Now ?Rich
|     | |  |`- Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|     | |  `- Pipes v. FIFOs (was: Are We Back to the "Wars" Now ?)Geoff Clare
|     | `- Re: Are We Back to the "Wars" Now ?Richard Kettlewell
|     `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|      `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
|       `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|        `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
|         `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|          `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
|           +* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|           |`* Re: Are We Back to the "Wars" Now ?Carlos E.R.
|           | +* Re: Are We Back to the "Wars" Now ?Fritz Wuehler
|           | |`- Re: Are We Back to the "Wars" Now ?Carlos E.R.
|           | `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|           |  `- Re: Are We Back to the "Wars" Now ?Carlos E.R.
|           `* Re: Are We Back to the "Wars" Now ?The Natural Philosopher
|            `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|             `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
|              +- Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|              `* Re: Are We Back to the "Wars" Now ?Rich
|               `* Re: Are We Back to the "Wars" Now ?Lew Pitcher
|                +- Re: Are We Back to the "Wars" Now ?Rich
|                `* Re: Are We Back to the "Wars" Now ?Richard Kettlewell
|                 +- Re: Are We Back to the "Wars" Now ?Carlos E.R.
|                 +- Re: Are We Back to the "Wars" Now ?Rich
|                 `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
|                  +- Re: Are We Back to the "Wars" Now ?Richard Kettlewell
|                  +- Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
|                  `- Re: Are We Back to the "Wars" Now ?Jerry Peters
`* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
 `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
  +* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
  |`* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
  | `- Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
  `* Re: Are We Back to the "Wars" Now ?Richard L. Hamilton
   `* Re: Are We Back to the "Wars" Now ?Pancho
    `* Re: Are We Back to the "Wars" Now ?root
     `* Re: Are We Back to the "Wars" Now ?The Natural Philosopher
      `* Re: Are We Back to the "Wars" Now ?Geoff Clare
       `* Re: Are We Back to the "Wars" Now ?Lawrence D'Oliveiro
        `* Re: Are We Back to the "Wars" Now ?186282@ud0s4.net
         `* Re: Are We Back to the "Wars" Now ?Rich
          +* Re: Are We Back to the "Wars" Now ?Robert Riches
          |`* Re: Are We Back to the "Wars" Now ?Rich
          | `* Re: Are We Back to the "Wars" Now ?Robert Riches
          |  `* Re: Are We Back to the "Wars" Now ?Rich
          |   `- Re: Are We Back to the "Wars" Now ?D
          `* Re: Are We Back to the "Wars" Now ?Richard Kettlewell
           `* Re: Are We Back to the "Wars" Now ?Rich
            `* Re: Are We Back to the "Wars" Now ?John Ames
             +- Re: Are We Back to the "Wars" Now ?Rich
             `- Re: Are We Back to the "Wars" Now ?Richard Kettlewell

Pages:1234
Subject: Re: Are We Back to the "Wars" Now ?
From: Richard Kettlewell
Newsgroups: comp.os.linux.misc
Organization: terraraq NNTP server
Date: Sun, 24 Nov 2024 19:14 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Path: eternal-september.org!news.eternal-september.org!feeder2.eternal-september.org!tncsrv06.tnetconsulting.net!nntp.terraraq.uk!.POSTED.tunnel.sfere.anjou.terraraq.org.uk!not-for-mail
From: invalid@invalid.invalid (Richard Kettlewell)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Sun, 24 Nov 2024 19:14:59 +0000
Organization: terraraq NNTP server
Message-ID: <wwvldx8qw98.fsf@LkoBDZeT.terraraq.uk>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhmm4c$hnbj$1@dont-email.me> <vhmn2t$hv8i$3@dont-email.me>
<vhnikj$me7m$1@dont-email.me> <vhoaau$qjkl$1@dont-email.me>
<lIqcnTlkpf_Jtd36nZ2dnZfqn_udnZ2d@earthlink.com>
<vhp9hs$12qaa$2@dont-email.me>
<v4adnSK1nItcwNz6nZ2dnZfqnPWdnZ2d@earthlink.com>
<vhrolu$1jata$1@dont-email.me>
<Oo6dnZb3z-NT6Nz6nZ2dnZfqnPednZ2d@earthlink.com>
<vhsdbi$1m6qu$1@dont-email.me> <vhth7o$1s5d5$4@dont-email.me>
<heednZLHO-zE69_6nZ2dnZfqn_idnZ2d@earthlink.com>
<vhvd06$2900d$1@dont-email.me> <vhvht1$28nn4$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Injection-Info: innmantic.terraraq.uk; posting-host="tunnel.sfere.anjou.terraraq.org.uk:172.17.207.6";
logging-data="143974"; mail-complaints-to="usenet@innmantic.terraraq.uk"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
Cancel-Lock: sha1:gmC8DLKazz7lQP4rgn4a98na1RU=
X-Face: h[Hh-7npe<<b4/eW[]sat,I3O`t8A`(ej.H!F4\8|;ih)`7{@:A~/j1}gTt4e7-n*F?.Rl^
F<\{jehn7.KrO{!7=:(@J~]<.[{>v9!1<qZY,{EJxg6?Er4Y7Ng2\Ft>Z&W?r\c.!4DXH5PWpga"ha
+r0NzP?vnz:e/knOY)PI-
X-Boydie: NO
View all headers

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
> On Sun, 24 Nov 2024 14:25:10 +0000, Rich wrote:
>> 186282@ud0s4.net <186283@ud0s4.net> wrote:
>>> On 11/23/24 4:25 PM, Lawrence D'Oliveiro wrote:
>>>> That’s irrelevant. Pipes originated on the earliest Unix machine,
>>>> which was a PDP-11 with only a 64kiB address space. They work great
>>>> for pumping around gigabytes of data, but you don’t need
>>>> gigabyte-sized memory buffers to do that.
>>>
>>> It all has to be SOMEWHERE ... if not in RAM then
>>> on a mass storage device.
>>
>> Nope, at least not with pipes.
>
> Hold on a sec.... pipes are /buffered/ in RAM, so there's at least
> a small bit of ram set aside for each open pipe.

The word ‘all’ isn’t just decoration. The claim was ‘it all has to be
somewhere’, and Rich’s point (as I understand it) is that it does not
all have to be somewhere.

For example,

head -c $((1024*1024*1024)) /dev/urandom | sha256sum

puts a gigabyte of data through a pipe, but at no point does anything
allocate anywhere close to a gigabyte of storage of any kind.

--
https://www.greenend.org.uk/rjk/

Subject: Re: Are We Back to the "Wars" Now ?
From: Carlos E.R.
Newsgroups: comp.os.linux.misc
Date: Sun, 24 Nov 2024 19:32 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Path: eternal-september.org!news.eternal-september.org!feeder2.eternal-september.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: robin_listas@es.invalid (Carlos E.R.)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Sun, 24 Nov 2024 20:32:46 +0100
Lines: 12
Message-ID: <ujbc1lxtjt.ln2@Telcontar.valinor>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhmm4c$hnbj$1@dont-email.me> <vhmn2t$hv8i$3@dont-email.me>
<vhnikj$me7m$1@dont-email.me> <vhoaau$qjkl$1@dont-email.me>
<lIqcnTlkpf_Jtd36nZ2dnZfqn_udnZ2d@earthlink.com>
<vhp9hs$12qaa$2@dont-email.me>
<v4adnSK1nItcwNz6nZ2dnZfqnPWdnZ2d@earthlink.com>
<vhrolu$1jata$1@dont-email.me>
<Oo6dnZb3z-NT6Nz6nZ2dnZfqnPednZ2d@earthlink.com>
<vhsdbi$1m6qu$1@dont-email.me> <vhth7o$1s5d5$4@dont-email.me>
<heednZLHO-zE69_6nZ2dnZfqn_idnZ2d@earthlink.com>
<vhvd06$2900d$1@dont-email.me> <vhvht1$28nn4$1@dont-email.me>
<wwvldx8qw98.fsf@LkoBDZeT.terraraq.uk>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Trace: individual.net Eju4taXQWjf/IgQ32UDx8AomqsB6lbsnxKlrizr3tU7aliQb6O
X-Orig-Path: Telcontar.valinor!not-for-mail
Cancel-Lock: sha1:2Ir1hedt82o1IW3bdVFcXxZwX30= sha256:3rUcrrbrb5AWkJBFLqzHFYdPmYvR7OJJlZm05TlqHzg=
User-Agent: Mozilla Thunderbird
Content-Language: es-ES, en-CA
In-Reply-To: <wwvldx8qw98.fsf@LkoBDZeT.terraraq.uk>
View all headers

On 2024-11-24 20:14, Richard Kettlewell wrote:
> For example,
>
> head -c $((1024*1024*1024)) /dev/urandom | sha256sum
>
> puts a gigabyte of data through a pipe, but at no point does anything
> allocate anywhere close to a gigabyte of storage of any kind.

With "dd .... -bs=1G" the program size is 1G in ram. Not a pipe, though.

--
Cheers, Carlos.

Subject: Re: Are We Back to the "Wars" Now ?
From: Rich
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Sun, 24 Nov 2024 19:36 UTC
References: 1 2 3 4 5 6 7 8 9 10 11
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: rich@example.invalid (Rich)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Sun, 24 Nov 2024 19:36:57 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 30
Message-ID: <vhvv8p$2c45s$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com> <lIqcnTlkpf_Jtd36nZ2dnZfqn_udnZ2d@earthlink.com> <vhp9hs$12qaa$2@dont-email.me> <v4adnSK1nItcwNz6nZ2dnZfqnPWdnZ2d@earthlink.com> <vhrolu$1jata$1@dont-email.me> <Oo6dnZb3z-NT6Nz6nZ2dnZfqnPednZ2d@earthlink.com> <vhsdbi$1m6qu$1@dont-email.me> <vhth7o$1s5d5$4@dont-email.me> <heednZLHO-zE69_6nZ2dnZfqn_idnZ2d@earthlink.com> <vhvd06$2900d$1@dont-email.me> <vhvht1$28nn4$1@dont-email.me> <wwvldx8qw98.fsf@LkoBDZeT.terraraq.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 24 Nov 2024 20:36:57 +0100 (CET)
Injection-Info: dont-email.me; posting-host="f9f41132f628b1fe2d858245900693aa";
logging-data="2494652"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/QHuzMUiYSmfVbq20RxNSV"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.139 (x86_64))
Cancel-Lock: sha1:7NlJH/RikqAtc7hBoyfCgR+E+hE=
View all headers

Richard Kettlewell <invalid@invalid.invalid> wrote:
> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>> On Sun, 24 Nov 2024 14:25:10 +0000, Rich wrote:
>>> 186282@ud0s4.net <186283@ud0s4.net> wrote:
>>>> On 11/23/24 4:25 PM, Lawrence D'Oliveiro wrote:
>>>>> That’s irrelevant. Pipes originated on the earliest Unix machine,
>>>>> which was a PDP-11 with only a 64kiB address space. They work great
>>>>> for pumping around gigabytes of data, but you don’t need
>>>>> gigabyte-sized memory buffers to do that.
>>>>
>>>> It all has to be SOMEWHERE ... if not in RAM then
>>>> on a mass storage device.
>>>
>>> Nope, at least not with pipes.
>>
>> Hold on a sec.... pipes are /buffered/ in RAM, so there's at least
>> a small bit of ram set aside for each open pipe.
>
> The word ‘all’ isn’t just decoration. The claim was ‘it all has to be
> somewhere’, and Rich’s point (as I understand it) is that it does not
> all have to be somewhere.
>
> For example,
>
> head -c $((1024*1024*1024)) /dev/urandom | sha256sum
>
> puts a gigabyte of data through a pipe, but at no point does anything
> allocate anywhere close to a gigabyte of storage of any kind.

Exactly.

Subject: Pipes v. FIFOs (was: Are We Back to the "Wars" Now ?)
From: Geoff Clare
Newsgroups: comp.os.linux.misc
Date: Tue, 26 Nov 2024 14:21 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12
Path: eternal-september.org!news.eternal-september.org!feeder2.eternal-september.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: geoff@clare.See-My-Signature.invalid (Geoff Clare)
Newsgroups: comp.os.linux.misc
Subject: Pipes v. FIFOs (was: Are We Back to the "Wars" Now ?)
Date: Tue, 26 Nov 2024 14:21:21 +0000
Lines: 14
Message-ID: <142h1l-92h.ln1@ID-313840.user.individual.net>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhf5ts$16rpr$1@dont-email.me> <vhfjtj$19ijm$1@dont-email.me>
<vhja6j$23f5e$4@dont-email.me> <vhmm4c$hnbj$1@dont-email.me>
<vhmn2t$hv8i$3@dont-email.me> <vhnikj$me7m$1@dont-email.me>
<wwvcyio8m9p.fsf@LkoBDZeT.terraraq.uk> <vhorrb$t48o$1@dont-email.me>
<vhosra$1171f$1@dont-email.me> <lqalg1F7fi9U2@mid.individual.net>
<vhql0r$1a0ch$2@dont-email.me>
Reply-To: netnews@gclare.org.uk
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Trace: individual.net Fqdm3BENRbt6MT3QxqQjBQ926xKqNz25dW3h5mdmF4HgDPtOW3
X-Orig-Path: ID-313840.user.individual.net!not-for-mail
Cancel-Lock: sha1:Wuv4SjvZ0Dv0MuqxZKfW0rivvAE= sha256:LHSktMsYCZ5FcA+ojeKYYfCqdMTWDOWr1B40QD90tLA=
User-Agent: Pan/0.154 (Izium; 517acf4)
View all headers

Rich wrote:

> The only 'filesystem' access for named pipes is during the
> open() call to look up the name from the filesystem. Once you get the
> file descriptor back, it is the exact same in-memory FIFO queue as an
> anonymous pipe created via pipe() (at least on Linux).

POSIX requires that pipes and FIFOs behave identically once you have a
file descriptor. This is how it defines the term "pipe":

An object identical to a FIFO which has no links in the file hierarchy.

--
Geoff Clare <netnews@gclare.org.uk>

Subject: Re: Named pipes vs. Unix sockets
From: Richard L. Hamilton
Newsgroups: comp.os.linux.misc
Organization: Timetravellers Anonymous
Date: Tue, 3 Dec 2024 05:23 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx17.iad.POSTED!not-for-mail
Mime-Version: 1.0
X-Newsreader: knews 1.0b.1
Reply-To: rlhamil@smart.net
Organization: Timetravellers Anonymous
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhf5ts$16rpr$1@dont-email.me> <vhfjtj$19ijm$1@dont-email.me>
<vhja6j$23f5e$4@dont-email.me> <vhmm4c$hnbj$1@dont-email.me>
<vhmn2t$hv8i$3@dont-email.me> <vhnikj$me7m$1@dont-email.me>
<wwvcyio8m9p.fsf@LkoBDZeT.terraraq.uk> <vhorrb$t48o$1@dont-email.me>
<vhosra$1171f$1@dont-email.me> <lqalg1F7fi9U2@mid.individual.net>
<vhp8qi$12m83$2@dont-email.me> <lqaoknF8btnU2@mid.individual.net>
<lqaq6cF8btnU3@mid.individual.net> <lqau7sF8btnU4@mid.individual.net>
<vhq6ag$176f0$1@dont-email.me>
From: rlhamil@smart.net (Richard L. Hamilton)
Subject: Re: Named pipes vs. Unix sockets
Newsgroups: comp.os.linux.misc
Content-Type: text/plain; charset=us-ascii
Lines: 46
Message-ID: <7Bw3P.41602$vLg2.8549@fx17.iad>
X-Complaints-To: abuse(at)newshosting.com
NNTP-Posting-Date: Tue, 03 Dec 2024 05:23:15 UTC
Date: Tue, 03 Dec 2024 05:23:15 GMT
X-Received-Bytes: 3626
View all headers

In article <vhq6ag$176f0$1@dont-email.me>,
Phillip Frabott <nntp@fulltermprivacy.com> writes:
> On 11/22/2024 03:38, vallor wrote:
>> On 22 Nov 2024 07:29:16 GMT, vallor <vallor@cultnix.org> wrote in
>> <lqaq6cF8btnU3@mid.individual.net>:
>>
>>> However, the speed appears to be limited by dd in my examples -- setting
>>> a block size to fill the pipe/packets seems to increase throughput:
>>>
>>> $ nc -l -U -N /tmp/socket > /dev/null & time -p ( dd if=/dev/zero
>>> count=$[1024*1024*4] bs=1024 | nc -U -N /tmp/socket > /dev/null )
>>
>> Realized the bottleneck would be the pipe between dd and nc, so wrote
>> a program to connect to /tmp/socket and spew data at it -- it sends
>> 46950 212992-byte buffers (9999974400 bytes) in 2.41 seconds.
>> (4149366971 bytes/second, or 4.1GB/s).
>>
>> (The default "MTU" for a Linux Unix socket connection
>> is 212992 bytes. Default pipe size is 8*512 bytes.)
>>
>
> Yeah, this has been the experience of our testing as well. We can also
> bump the MTU up a bit to get sockets to do better and we've also
> modified some of the socket code to our environment so it's able to do
> around 12GB/s. We think we can tune it further to get about 15GB/s from
> sockets. At the moment we have to do 4.6GB per 0.5sec to keep up, and
> 7GB per 0.5sec to stay ahead. We've done everything we can to squeeze
> out as much performance from pipes as we could but there just isn't much
> we can do anymore. But yeah, sockets are way more capable of large data
> intake vs pipes, and this shows as well in a lot of research that's been
> done. We've been testing sockets for about the last year and the results
> have shown we are keeping up better then we are with pipes right now.
>
> And for tribute to pipes, we've been using pipes in this project for the
> last 30 years. That's a pretty great track record for something to have
> been useful for such a long time and not be replaced with something else
> until now. Sadly sockets will likely bottom out in the next 10-15 years
> and we'll have to switch to something else. But for now, it's working.
>

Shared memory and a semaphore to coordinate access? Should be crazy fast
given a large enough hunk of shared memory. More changes on both ends, of
course; whereas on Solaris, I swapped (unnamed) pipes (STREAMS based, there)
with socketpair() using an LD_PRELOADable wrapper, so no changes on either end.
Just to see if I could; I didn't run any timing tests.

Subject: Re: Are We Back to the "Wars" Now ?
From: 186282@ud0s4.net
Newsgroups: comp.os.linux.misc
Organization: wokiesux
Date: Tue, 3 Dec 2024 06:19 UTC
References: 1 2
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!border-2.nntp.ord.giganews.com!nntp.giganews.com!Xl.tags.giganews.com!local-4.nntp.ord.giganews.com!nntp.earthlink.com!news.earthlink.com.POSTED!not-for-mail
NNTP-Posting-Date: Tue, 03 Dec 2024 06:19:54 +0000
Subject: Re: Are We Back to the "Wars" Now ?
Newsgroups: comp.os.linux.misc
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
From: 186283@ud0s4.net (186282@ud0s4.net)
Organization: wokiesux
Date: Tue, 3 Dec 2024 01:19:53 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <vhja4p$23f5e$3@dont-email.me>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Message-ID: <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
Lines: 20
X-Usenet-Provider: http://www.giganews.com
NNTP-Posting-Host: 99.101.150.97
X-Trace: sv3-Y7P9zCl2B3FMCXZ2wXn4EQAU7uTw4f7Z0XyszYlJ6yZn5SJAJVxeE1jSdita+gl1wnw84HJfYLCXkgq!u3oms1pqcRzuSbKAiwOqGUw+kKz7mDQGD4wpFcx75HgjSfNDXl2lgD8naUD8SrwpB8hi9fEIcxjX!PaUUNDX0a5ML4BsNMg6p
X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
X-Postfilter: 1.3.40
View all headers

On 11/19/24 7:22 PM, Lawrence D'Oliveiro wrote:
> On Mon, 18 Nov 2024 01:20:53 -0500, 186282@ud0s4.net wrote:
>
>> Hey, if I feel the need to use files instead of 'pipes' then I WANT TO
>> USE FILES INSTEAD OF PIPES.
>
> Except ... you were trying to argue that there was no fundamental
> difference between pipes and files anyway. That you could somehow do
> everything you could do with pipes by using temporary files.

Yep.

But I *just may not WANT to* :-)

Files DO have a hidden advantage - many largely
dis-related programs can ACCESS them. This can
give you stats, insight, 'intelligence'. Pipes
are basically restricted to the original parent
and children. Good reasons for that, sometimes,
but not *always*.

Subject: Re: Named pipes vs. Unix sockets
From: Lawrence D'Oliv
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Tue, 3 Dec 2024 06:20 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ldo@nz.invalid (Lawrence D'Oliveiro)
Newsgroups: comp.os.linux.misc
Subject: Re: Named pipes vs. Unix sockets
Date: Tue, 3 Dec 2024 06:20:01 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 9
Message-ID: <vim7ug$3t1l3$3@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhf5ts$16rpr$1@dont-email.me> <vhfjtj$19ijm$1@dont-email.me>
<vhja6j$23f5e$4@dont-email.me> <vhmm4c$hnbj$1@dont-email.me>
<vhmn2t$hv8i$3@dont-email.me> <vhnikj$me7m$1@dont-email.me>
<wwvcyio8m9p.fsf@LkoBDZeT.terraraq.uk> <vhorrb$t48o$1@dont-email.me>
<vhosra$1171f$1@dont-email.me> <lqalg1F7fi9U2@mid.individual.net>
<vhp8qi$12m83$2@dont-email.me> <lqaoknF8btnU2@mid.individual.net>
<lqaq6cF8btnU3@mid.individual.net> <lqau7sF8btnU4@mid.individual.net>
<vhq6ag$176f0$1@dont-email.me> <7Bw3P.41602$vLg2.8549@fx17.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 03 Dec 2024 07:20:01 +0100 (CET)
Injection-Info: dont-email.me; posting-host="c96be26192a45ce8d8c08f341d719685";
logging-data="4097699"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/GoVRAES4+iDSNkcqF6mKD"
User-Agent: Pan/0.161 (Chasiv Yar; )
Cancel-Lock: sha1:Ex0W0Cb0/qDRdYuv287eksY+Axw=
View all headers

On Tue, 03 Dec 2024 05:23:15 GMT, Richard L. Hamilton wrote:

> ... whereas on Solaris, I swapped (unnamed) pipes (STREAMS based,
> there) with socketpair() using an LD_PRELOADable wrapper, so no changes
> on either end.

STREAMS made that easy to do, didn’t it. Yet the whole idea was never that
popular; both Steve Jobs and Linus Torvalds were less than enthusiastic
about it. Were there performance downsides?

Subject: Re: Are We Back to the "Wars" Now ?
From: Lawrence D'Oliv
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Tue, 3 Dec 2024 06:49 UTC
References: 1 2 3
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ldo@nz.invalid (Lawrence D'Oliveiro)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Tue, 3 Dec 2024 06:49:41 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 6
Message-ID: <vim9m5$3th66$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 03 Dec 2024 07:49:41 +0100 (CET)
Injection-Info: dont-email.me; posting-host="c96be26192a45ce8d8c08f341d719685";
logging-data="4113606"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+2kgGcLVZupOS14qvLaCle"
User-Agent: Pan/0.161 (Chasiv Yar; )
Cancel-Lock: sha1:uELm+8xlTkCZy84A1y0/7lJfjk4=
View all headers

On Tue, 3 Dec 2024 01:19:53 -0500, 186282@ud0s4.net wrote:

> Files DO have a hidden advantage - many largely dis-related programs
> can ACCESS them.

On *nix systems, pipes can be accessed just as easily.

Subject: Re: Are We Back to the "Wars" Now ?
From: 186282@ud0s4.net
Newsgroups: comp.os.linux.misc
Organization: wokiesux
Date: Wed, 4 Dec 2024 06:10 UTC
References: 1 2 3 4
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!border-1.nntp.ord.giganews.com!nntp.giganews.com!local-3.nntp.ord.giganews.com!Xl.tags.giganews.com!local-4.nntp.ord.giganews.com!nntp.earthlink.com!news.earthlink.com.POSTED!not-for-mail
NNTP-Posting-Date: Wed, 04 Dec 2024 06:10:56 +0000
Subject: Re: Are We Back to the "Wars" Now ?
Newsgroups: comp.os.linux.misc
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<vim9m5$3th66$1@dont-email.me>
From: 186283@ud0s4.net (186282@ud0s4.net)
Organization: wokiesux
Date: Wed, 4 Dec 2024 01:10:25 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <vim9m5$3th66$1@dont-email.me>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Message-ID: <Y_SdnQQNbertb9L6nZ2dnZfqnPWdnZ2d@earthlink.com>
Lines: 16
X-Usenet-Provider: http://www.giganews.com
NNTP-Posting-Host: 99.101.150.97
X-Trace: sv3-0gmEQFl9jQvay44FGSxt1HVmO/nHet0105FyQOYzEAlMAWP+wGd4regBrV+RDmV61c8QqOgLgaimvK8!8EvWfL0wBwimfo5x5dUux4c+XCKUdVlltpiB1jxqCFtsnvLiGITMPTWD5uOBBPokm9vFjXjBQV3d!BKgVa7zCP3w7atj1VTN8
X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
X-Postfilter: 1.3.40
View all headers

On 12/3/24 1:49 AM, Lawrence D'Oliveiro wrote:
> On Tue, 3 Dec 2024 01:19:53 -0500, 186282@ud0s4.net wrote:
>
>> Files DO have a hidden advantage - many largely dis-related programs
>> can ACCESS them.
>
> On *nix systems, pipes can be accessed just as easily.

Well ... do it your way :-)

I'll stay more flexible, whatever inspires.

I can't really argue over pipes/files ... they
can both get the same thing done. Files CAN be
a bit more flexible, even non-pipe langs can
access them very easily.

Subject: Re: Are We Back to the "Wars" Now ?
From: Lawrence D'Oliv
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 4 Dec 2024 06:45 UTC
References: 1 2 3 4 5
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ldo@nz.invalid (Lawrence D'Oliveiro)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 4 Dec 2024 06:45:39 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 24
Message-ID: <viotqi$n3vg$4@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<vim9m5$3th66$1@dont-email.me>
<Y_SdnQQNbertb9L6nZ2dnZfqnPWdnZ2d@earthlink.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 04 Dec 2024 07:45:39 +0100 (CET)
Injection-Info: dont-email.me; posting-host="49d5f9553a2964b815f257dd85f9d35a";
logging-data="757744"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/oL908Hk1SlEbvlg54a7Jw"
User-Agent: Pan/0.161 (Chasiv Yar; )
Cancel-Lock: sha1:dqiAUfDdu2xD/BIjWlCAMI1PrzM=
View all headers

On Wed, 4 Dec 2024 01:10:25 -0500, 186282@ud0s4.net wrote:

> On 12/3/24 1:49 AM, Lawrence D'Oliveiro wrote:
>>
>> On Tue, 3 Dec 2024 01:19:53 -0500, 186282@ud0s4.net wrote:
>>
>>> Files DO have a hidden advantage - many largely dis-related programs
>>> can ACCESS them.
>>
>> On *nix systems, pipes can be accessed just as easily.
>
> I can't really argue over pipes/files ...

Consider this:

cat <(echo first line) <(echo second line) <(echo third line)

Output:

first line
second line
third line

What’s going on here?

Subject: Re: Named pipes vs. Unix sockets
From: Richard L. Hamilton
Newsgroups: comp.os.linux.misc
Organization: Timetravellers Anonymous
Date: Sat, 14 Dec 2024 07:58 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!tncsrv06.tnetconsulting.net!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx16.iad.POSTED!not-for-mail
Mime-Version: 1.0
X-Newsreader: knews 1.0b.1
Reply-To: rlhamil@smart.net
Organization: Timetravellers Anonymous
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhf5ts$16rpr$1@dont-email.me> <vhfjtj$19ijm$1@dont-email.me>
<vhja6j$23f5e$4@dont-email.me> <vhmm4c$hnbj$1@dont-email.me>
<vhmn2t$hv8i$3@dont-email.me> <vhnikj$me7m$1@dont-email.me>
<wwvcyio8m9p.fsf@LkoBDZeT.terraraq.uk> <vhorrb$t48o$1@dont-email.me>
<vhosra$1171f$1@dont-email.me> <lqalg1F7fi9U2@mid.individual.net>
<vhp8qi$12m83$2@dont-email.me> <lqaoknF8btnU2@mid.individual.net>
<lqaq6cF8btnU3@mid.individual.net> <lqau7sF8btnU4@mid.individual.net>
<vhq6ag$176f0$1@dont-email.me> <7Bw3P.41602$vLg2.8549@fx17.iad>
<vim7ug$3t1l3$3@dont-email.me>
From: rlhamil@smart.net (Richard L. Hamilton)
Subject: Re: Named pipes vs. Unix sockets
Newsgroups: comp.os.linux.misc
Content-Type: text/plain; charset=iso8859-1
Content-Transfer-Encoding: 8bit
Lines: 29
Message-ID: <lUa7P.64694$oR74.14893@fx16.iad>
X-Complaints-To: abuse(at)newshosting.com
NNTP-Posting-Date: Sat, 14 Dec 2024 07:58:09 UTC
Date: Sat, 14 Dec 2024 07:58:09 GMT
X-Received-Bytes: 2672
View all headers

In article <vim7ug$3t1l3$3@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
> On Tue, 03 Dec 2024 05:23:15 GMT, Richard L. Hamilton wrote:
>
>> ... whereas on Solaris, I swapped (unnamed) pipes (STREAMS based,
>> there) with socketpair() using an LD_PRELOADable wrapper, so no changes
>> on either end.
>
> STREAMS made that easy to do, didn’t it. Yet the whole idea was never that
> popular; both Steve Jobs and Linus Torvalds were less than enthusiastic
> about it. Were there performance downsides?

What I did didn't depend on STREAMS, only that the pipe implementation
was bidirectional like a socketpair() is, so that the behavior is
similar enough for most purposes. So I didn't write a STREAMS module.
The difference between pipe() and socketpair() does change some of the
tricky things that can be done; for instance, both STREAMS pipes and
socketpair() could allow passing a file descriptor, but how that's
done is quite different.

Solaris still uses STREAMS, but to get TCP/IP up to full speed, the
connection between the two isn't STREAMS any more (so you can't put
a STREAMS module between them) but some faster approach at
handing off packets. Somewhere in there is also a fast packet classifier,
to decide what listeners get what packets as quickly as possible.

Not sure how they did SCTP, haven't looked at that code.

So yes, the full generality of STREAMS can have performance limitations.

Subject: Re: Are We Back to the "Wars" Now ?
From: Richard L. Hamilton
Newsgroups: comp.os.linux.misc
Organization: Timetravellers Anonymous
Date: Sat, 14 Dec 2024 08:06 UTC
References: 1 2 3
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer02.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx16.iad.POSTED!not-for-mail
Mime-Version: 1.0
X-Newsreader: knews 1.0b.1
Reply-To: rlhamil@smart.net
Organization: Timetravellers Anonymous
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
From: rlhamil@smart.net (Richard L. Hamilton)
Subject: Re: Are We Back to the "Wars" Now ?
Newsgroups: comp.os.linux.misc
Content-Type: text/plain; charset=us-ascii
Lines: 39
Message-ID: <w0b7P.64695$oR74.19157@fx16.iad>
X-Complaints-To: abuse(at)newshosting.com
NNTP-Posting-Date: Sat, 14 Dec 2024 08:06:52 UTC
Date: Sat, 14 Dec 2024 08:06:52 GMT
X-Received-Bytes: 2503
View all headers

In article <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>,
"186282@ud0s4.net" <186283@ud0s4.net> writes:
> On 11/19/24 7:22 PM, Lawrence D'Oliveiro wrote:
>> On Mon, 18 Nov 2024 01:20:53 -0500, 186282@ud0s4.net wrote:
>>
>>> Hey, if I feel the need to use files instead of 'pipes' then I WANT TO
>>> USE FILES INSTEAD OF PIPES.
>>
>> Except ... you were trying to argue that there was no fundamental
>> difference between pipes and files anyway. That you could somehow do
>> everything you could do with pipes by using temporary files.
>
> Yep.
>
> But I *just may not WANT to* :-)
>
> Files DO have a hidden advantage - many largely
> dis-related programs can ACCESS them. This can
> give you stats, insight, 'intelligence'. Pipes
> are basically restricted to the original parent
> and children. Good reasons for that, sometimes,
> but not *always*.

Named pipes can allow communication between unrelated processes.

Using files means there has to be locking or some coordination, so
that the receiver only reads the file when the contents are in a
consistent state. Renaming a file (on the same filesystem, where it's
not a copy and delete) is atomic, so if the file is created in one
directory and moved to a parallel directory when complete, the
receiving program can just grab it from there, perhaps after being
signalled to wake up and scan the directory. That works somewhat
efficiently even without modern locking or filesystem change
notification mechanisms.

A file has the advantage that one can seek on it, which may simplify
some things; for example if a header has a checksum over the following
data, it's easier to seek back and fill that field in with its final
value. Otherwise one may have to use a temporary file internally anyway.

Subject: Re: Are We Back to the "Wars" Now ?
From: Pancho
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Sat, 14 Dec 2024 10:10 UTC
References: 1 2 3 4
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Pancho.Jones@proton.me (Pancho)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Sat, 14 Dec 2024 10:10:57 +0000
Organization: A noiseless patient Spider
Lines: 53
Message-ID: <vjjljh$3unfq$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Sat, 14 Dec 2024 11:10:57 +0100 (CET)
Injection-Info: dont-email.me; posting-host="74b736f9eb3d83cc1dcc839794a0408f";
logging-data="4152826"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+sbPpj8MAEfwZEY1oroIzMqsCknG9y3cQ="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:YUJe1guO79yB9UccncGaqR2C+DQ=
In-Reply-To: <w0b7P.64695$oR74.19157@fx16.iad>
Content-Language: en-GB
View all headers

On 12/14/24 08:06, Richard L. Hamilton wrote:
> In article <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>,
> "186282@ud0s4.net" <186283@ud0s4.net> writes:
>> On 11/19/24 7:22 PM, Lawrence D'Oliveiro wrote:
>>> On Mon, 18 Nov 2024 01:20:53 -0500, 186282@ud0s4.net wrote:
>>>
>>>> Hey, if I feel the need to use files instead of 'pipes' then I WANT TO
>>>> USE FILES INSTEAD OF PIPES.
>>>
>>> Except ... you were trying to argue that there was no fundamental
>>> difference between pipes and files anyway. That you could somehow do
>>> everything you could do with pipes by using temporary files.
>>
>> Yep.
>>
>> But I *just may not WANT to* :-)
>>
>> Files DO have a hidden advantage - many largely
>> dis-related programs can ACCESS them. This can
>> give you stats, insight, 'intelligence'. Pipes
>> are basically restricted to the original parent
>> and children. Good reasons for that, sometimes,
>> but not *always*.
>
> Named pipes can allow communication between unrelated processes.
>
> Using files means there has to be locking or some coordination, so
> that the receiver only reads the file when the contents are in a
> consistent state.

I don't really know, but I would be surprised if that were always true.

> Renaming a file (on the same filesystem, where it's
> not a copy and delete) is atomic, so if the file is created in one
> directory and moved to a parallel directory when complete, the
> receiving program can just grab it from there, perhaps after being
> signalled to wake up and scan the directory. That works somewhat
> efficiently even without modern locking or filesystem change
> notification mechanisms.
>

I suspect you are mixing up sensible programming techniques for handling
file concurrency, with the way things need to be. i.e. I think it is
perfectly possible to have concurrent programs read and write to the
same file, it is just that there might be a few gotchas. Whereas
renaming is a very safe and simple technique, avoiding many potential
problems.

> A file has the advantage that one can seek on it, which may simplify
> some things; for example if a header has a checksum over the following
> data, it's easier to seek back and fill that field in with its final
> value. Otherwise one may have to use a temporary file internally anyway.

Subject: Re: Are We Back to the "Wars" Now ?
From: root
Newsgroups: comp.os.linux.misc
Organization: Linux Advocacy
Date: Sat, 14 Dec 2024 15:54 UTC
References: 1 2 3 4 5
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: NoEMail@home.org (root)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Sat, 14 Dec 2024 15:54:30 -0000 (UTC)
Organization: Linux Advocacy
Lines: 3
Message-ID: <vjk9nm$2chi$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
Injection-Date: Sat, 14 Dec 2024 16:54:31 +0100 (CET)
Injection-Info: dont-email.me; posting-host="24dd4e08120658f4f3b771f47cc62dfc";
logging-data="78386"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+ZIvNbv6LldhN4t0qnIzlsV7soKmx8FzI="
User-Agent: slrn/1.0.2 (Linux)
Cancel-Lock: sha1:+cPrUsDq3KU6n92R42cmrD5mMY8=
View all headers

If you create a tmpfs /ram directory then pipes or named
files would both work in ram, and there would be no
functional difference.

Subject: Re: Are We Back to the "Wars" Now ?
From: The Natural Philosop
Newsgroups: comp.os.linux.misc
Organization: A little, after lunch
Date: Sat, 14 Dec 2024 16:06 UTC
References: 1 2 3 4 5 6
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: tnp@invalid.invalid (The Natural Philosopher)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Sat, 14 Dec 2024 16:06:33 +0000
Organization: A little, after lunch
Lines: 19
Message-ID: <vjkae9$1uri$11@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 14 Dec 2024 17:06:34 +0100 (CET)
Injection-Info: dont-email.me; posting-host="63747a19b95c63ec9864040b731c8f1c";
logging-data="64370"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+l1kvSrxqT667RZ877Oo05OEFGizEHiQ8="
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:EWnEywqQ1N6i6/fxK96N4+uly4U=
Content-Language: en-GB
In-Reply-To: <vjk9nm$2chi$1@dont-email.me>
View all headers

On 14/12/2024 15:54, root wrote:
> If you create a tmpfs /ram directory then pipes or named
> files would both work in ram, and there would be no
> functional difference.

I am not sure how pipes work as to be certain of that. There are
overheads associated with files and with pipes.

I am a fan of ramdisk files because I understand files, but not really
pipes.
But I cant say they are as good. They are, for me. merely 'good enough'

--
“The fundamental cause of the trouble in the modern world today is that
the stupid are cocksure while the intelligent are full of doubt."

- Bertrand Russell

Subject: Re: Are We Back to the "Wars" Now ?
From: Geoff Clare
Newsgroups: comp.os.linux.misc
Date: Tue, 17 Dec 2024 13:34 UTC
References: 1 2 3 4 5 6 7
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: geoff@clare.See-My-Signature.invalid (Geoff Clare)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Tue, 17 Dec 2024 13:34:30 +0000
Lines: 28
Message-ID: <68b83l-jtd.ln1@ID-313840.user.individual.net>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me>
Reply-To: netnews@gclare.org.uk
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Trace: individual.net w17NefBDHJ/JqVlDzaVSVQ8ub9Bfk1MUzFCd+CuFSQfKtZ4emP
X-Orig-Path: ID-313840.user.individual.net!not-for-mail
Cancel-Lock: sha1:YuWqyYFZ+2lzL1ts8Lib06RakYE= sha256:b3GKR235ZPRYbvtwL/w+d7ZfZ4Qrb7hiBVz6TTMoO+k=
User-Agent: Pan/0.154 (Izium; 517acf4)
View all headers

The Natural Philosopher wrote:

> On 14/12/2024 15:54, root wrote:
>> If you create a tmpfs /ram directory then pipes or named
>> files would both work in ram, and there would be no
>> functional difference.
>
> I am not sure how pipes work as to be certain of that.

In fact quite the opposite is true; there is a very real and
easily noticeable functional difference.

If you use a regular file in a tmpfs to transfer data between two
processes, the amount of data you can transfer is limited to the
free space on the tmpfs, unless you go to great lengths to put
logic into the two processes to coordinate the reading and writing
such that the file never grows larger than a certain size and is
overwritten as necessary. You also need a way for the writer to
indicate to the reader that all of the data has been written.

With a pipe or FIFO, you just use simple read and write operations
and the system handles all the messy stuff for you. If the pipe
reaches capacity, write blocks until there is room to write some
more; if the pipe becomes empty, read blocks until there is more
data available; when read returns EOF that's the end of the data.

--
Geoff Clare <netnews@gclare.org.uk>

Subject: Re: Are We Back to the "Wars" Now ?
From: Lawrence D'Oliv
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 18 Dec 2024 01:23 UTC
References: 1 2 3 4 5 6 7 8
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ldo@nz.invalid (Lawrence D'Oliveiro)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 01:23:18 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 14
Message-ID: <vjt866$20222$3@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me>
<68b83l-jtd.ln1@ID-313840.user.individual.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 18 Dec 2024 02:23:18 +0100 (CET)
Injection-Info: dont-email.me; posting-host="9ed5d2960e91ad91a8c092c667dd3279";
logging-data="2099266"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19t5wq2TKT5JFtqa1bY43Uw"
User-Agent: Pan/0.161 (Chasiv Yar; )
Cancel-Lock: sha1:yENo+5S7N+cTwT+bwmJyotP+wbQ=
View all headers

On Tue, 17 Dec 2024 13:34:30 +0000, Geoff Clare wrote:

> With a pipe or FIFO, you just use simple read and write operations and
> the system handles all the messy stuff for you. If the pipe reaches
> capacity, write blocks until there is room to write some more; if the
> pipe becomes empty, read blocks until there is more data available; when
> read returns EOF that's the end of the data.

Yup. Furthermore:

* When the last writer closes its end, any remaining read attempts get
EOF.
* When the last reader closes its end, any remaining write attempts get
“broken pipe”.

Subject: Re: Are We Back to the "Wars" Now ?
From: 186282@ud0s4.net
Newsgroups: comp.os.linux.misc
Organization: wokiesux
Date: Wed, 18 Dec 2024 04:25 UTC
References: 1 2 3 4 5 6 7 8 9
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!border-2.nntp.ord.giganews.com!nntp.giganews.com!Xl.tags.giganews.com!local-3.nntp.ord.giganews.com!nntp.earthlink.com!news.earthlink.com.POSTED!not-for-mail
NNTP-Posting-Date: Wed, 18 Dec 2024 04:25:26 +0000
Subject: Re: Are We Back to the "Wars" Now ?
Newsgroups: comp.os.linux.misc
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me>
<68b83l-jtd.ln1@ID-313840.user.individual.net> <vjt866$20222$3@dont-email.me>
From: 186283@ud0s4.net (186282@ud0s4.net)
Organization: wokiesux
Date: Tue, 17 Dec 2024 23:25:25 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
MIME-Version: 1.0
In-Reply-To: <vjt866$20222$3@dont-email.me>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Message-ID: <KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com>
Lines: 36
X-Usenet-Provider: http://www.giganews.com
NNTP-Posting-Host: 99.101.150.97
X-Trace: sv3-C3lj1qktbJ5Y8ej6bTvr8YU8Bj2NjdCKrpWmAmRGZIPo++FVOa7JZV7frQJXYnLkOv7p+BL6AjzUvfH!VrljyH0R+paZFJF4rPp4WbIq1AG+tXw6A6hQsjazxnKNJlHRCXaG2YdThqHW0bnXNVHjyrOi4I9P!ODqq+0hPLs8Ov0BbDDCR
X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
X-Postfilter: 1.3.40
View all headers

On 12/17/24 8:23 PM, Lawrence D'Oliveiro wrote:
> On Tue, 17 Dec 2024 13:34:30 +0000, Geoff Clare wrote:
>
>> With a pipe or FIFO, you just use simple read and write operations and
>> the system handles all the messy stuff for you. If the pipe reaches
>> capacity, write blocks until there is room to write some more; if the
>> pipe becomes empty, read blocks until there is more data available; when
>> read returns EOF that's the end of the data.
>
> Yup. Furthermore:
>
> * When the last writer closes its end, any remaining read attempts get
> EOF.
> * When the last reader closes its end, any remaining write attempts get
> “broken pipe”.

But you're still limited to the amount of RAM the
system can access.

These days that's probably a LOT - but might NOT be,
esp for 'embedded' type boards like the older PIs,
BBBs and such. Never assume the user has essentially
infinite RAM.

For such platforms you may need to sacrifice speed
and some convenience and instead put such data on a
mass-storage device. Not THAT hard to do - but
remember to use locks.

Know your target audience - yet TRY to accommodate all.

I'll still rec making interprocess comms relatively
terse and handle 'big data' in some other manner
(as extensively covered in this thread). An 8k raw
frame grab is about 146mb - and let's not even speak
of 8k video segments.

Subject: Re: Are We Back to the "Wars" Now ?
From: Rich
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 18 Dec 2024 05:01 UTC
References: 1 2 3 4 5 6 7 8 9 10
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: rich@example.invalid (Rich)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 05:01:15 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 33
Message-ID: <vjtkuq$25l55$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com> <vhja4p$23f5e$3@dont-email.me> <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com> <w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me> <vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me> <68b83l-jtd.ln1@ID-313840.user.individual.net> <vjt866$20222$3@dont-email.me> <KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 18 Dec 2024 06:01:16 +0100 (CET)
Injection-Info: dont-email.me; posting-host="0a4897d36eecd2c0c7c91823a39c9faf";
logging-data="2282661"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1++4hcdJe6MRZQhYZHpjUSw"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.139 (x86_64))
Cancel-Lock: sha1:kflEJQ3zpLuPN+gVQJHD0xBAUM8=
View all headers

186282@ud0s4.net <186283@ud0s4.net> wrote:
> On 12/17/24 8:23 PM, Lawrence D'Oliveiro wrote:
>> On Tue, 17 Dec 2024 13:34:30 +0000, Geoff Clare wrote:
>>
>>> With a pipe or FIFO, you just use simple read and write operations
>>> and the system handles all the messy stuff for you. If the pipe
>>> reaches capacity, write blocks until there is room to write some
>>> more; if the pipe becomes empty, read blocks until there is more
>>> data available; when read returns EOF that's the end of the data.
>>
>> Yup. Furthermore:
>>
>> * When the last writer closes its end, any remaining read attempts
>> get EOF.
>> * When the last reader closes its end, any remaining write attempts
>> get “broken pipe”.
>
> But you're still limited to the amount of RAM the system can
> access.

Not with a pipe or FIFO, which is what is being discussed above.

The amount of data you can transfer over a pipe is not in any way
limited by system memory size or any other system imposed limits.

> These days that's probably a LOT - but might NOT be,
> esp for 'embedded' type boards like the older PIs,
> BBBs and such. Never assume the user has essentially
> infinite RAM.

The system will not have infinite RAM. You can transfer infinite data
over a pipe (although it will take a while to reach infinity).

Subject: Re: Are We Back to the "Wars" Now ?
From: Robert Riches
Newsgroups: comp.os.linux.misc
Organization: none-at-all
Date: Wed, 18 Dec 2024 05:12 UTC
References: 1 2 3 4 5 6 7 8 9 10 11
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: spamtrap42@jacob21819.net (Robert Riches)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: 18 Dec 2024 05:12:33 GMT
Organization: none-at-all
Lines: 49
Message-ID: <slrnvm4me1.662.spamtrap42@one.localnet>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me>
<68b83l-jtd.ln1@ID-313840.user.individual.net>
<vjt866$20222$3@dont-email.me>
<KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com>
<vjtkuq$25l55$1@dont-email.me>
Reply-To: spamtrap42@jacob21819.net
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Trace: individual.net X9kKFL01cQ9jN4bycg35IwBz5cZUWALYeKYDiF264qfpZ/ECYC
Cancel-Lock: sha1:yKG+VdVe32/aG8q+Y3lXva0MDjM= sha256:hwmXQWf6rxetkOWp4V4RKDpamHTC8wDcx9WNu6xU3pQ=
User-Agent: slrn/1.0.3 (Linux)
View all headers

On 2024-12-18, Rich <rich@example.invalid> wrote:
> 186282@ud0s4.net <186283@ud0s4.net> wrote:
>> On 12/17/24 8:23 PM, Lawrence D'Oliveiro wrote:
>>> On Tue, 17 Dec 2024 13:34:30 +0000, Geoff Clare wrote:
>>>
>>>> With a pipe or FIFO, you just use simple read and write operations
>>>> and the system handles all the messy stuff for you. If the pipe
>>>> reaches capacity, write blocks until there is room to write some
>>>> more; if the pipe becomes empty, read blocks until there is more
>>>> data available; when read returns EOF that's the end of the data.
>>>
>>> Yup. Furthermore:
>>>
>>> * When the last writer closes its end, any remaining read attempts
>>> get EOF.
>>> * When the last reader closes its end, any remaining write attempts
>>> get “broken pipe”.
>>
>> But you're still limited to the amount of RAM the system can
>> access.
>
> Not with a pipe or FIFO, which is what is being discussed above.
>
> The amount of data you can transfer over a pipe is not in any way
> limited by system memory size or any other system imposed limits.
>
>> These days that's probably a LOT - but might NOT be,
>> esp for 'embedded' type boards like the older PIs,
>> BBBs and such. Never assume the user has essentially
>> infinite RAM.
>
> The system will not have infinite RAM. You can transfer infinite data
> over a pipe (although it will take a while to reach infinity).

A pipe is _NOT_ limited to system RAM!

Using a named pipe on a Raspberry Pi model 1 with a _half_ GB of
total RAM, I would routinely transfer _several_ GB in a single
stream from an mplayer process to a netcat process. The only
reason that's not currently happening every night these days is
the amplified TV antenna lost too much gain due to age, attic
heat, etc.

HTH

--
Robert Riches
spamtrap42@jacob21819.net
(Yes, that is one of my email addresses.)

Subject: Re: Are We Back to the "Wars" Now ?
From: Richard Kettlewell
Newsgroups: comp.os.linux.misc
Organization: terraraq NNTP server
Date: Wed, 18 Dec 2024 08:27 UTC
References: 1 2 3 4 5 6 7 8 9 10 11
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!news.gegeweb.eu!gegeweb.org!nntp.terraraq.uk!.POSTED.tunnel.sfere.anjou.terraraq.org.uk!not-for-mail
From: invalid@invalid.invalid (Richard Kettlewell)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 08:27:25 +0000
Organization: terraraq NNTP server
Message-ID: <wwvy10d5rbm.fsf@LkoBDZeT.terraraq.uk>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me>
<68b83l-jtd.ln1@ID-313840.user.individual.net>
<vjt866$20222$3@dont-email.me>
<KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com>
<vjtkuq$25l55$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Injection-Info: innmantic.terraraq.uk; posting-host="tunnel.sfere.anjou.terraraq.org.uk:172.17.207.6";
logging-data="60032"; mail-complaints-to="usenet@innmantic.terraraq.uk"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux)
Cancel-Lock: sha1:htNiUS23ym9LO9Y5fvBhVedryA8=
X-Face: h[Hh-7npe<<b4/eW[]sat,I3O`t8A`(ej.H!F4\8|;ih)`7{@:A~/j1}gTt4e7-n*F?.Rl^
F<\{jehn7.KrO{!7=:(@J~]<.[{>v9!1<qZY,{EJxg6?Er4Y7Ng2\Ft>Z&W?r\c.!4DXH5PWpga"ha
+r0NzP?vnz:e/knOY)PI-
X-Boydie: NO
View all headers

Rich <rich@example.invalid> writes:
> 186282@ud0s4.net <186283@ud0s4.net> wrote:
>> But you're still limited to the amount of RAM the system can
>> access.
>
> Not with a pipe or FIFO, which is what is being discussed above.
>
> The amount of data you can transfer over a pipe is not in any way
> limited by system memory size or any other system imposed limits.

Quite. I’m not sure why this discussion has restarted but it was clear
from last time round that some of the participants don’t know what a
pipe is, and aren’t particularly interested in finding out.

>> These days that's probably a LOT - but might NOT be,
>> esp for 'embedded' type boards like the older PIs,
>> BBBs and such. Never assume the user has essentially
>> infinite RAM.
>
> The system will not have infinite RAM. You can transfer infinite data
> over a pipe (although it will take a while to reach infinity).

I think I’d use a slightly weaker term than ‘infinite’, something will
put an upper bound on it, even if it’s the heat death of the universe.

--
https://www.greenend.org.uk/rjk/

Subject: Re: Are We Back to the "Wars" Now ?
From: Rich
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 18 Dec 2024 14:02 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: rich@example.invalid (Rich)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 14:02:52 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 30
Message-ID: <vjukmc$2aoep$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com> <vhja4p$23f5e$3@dont-email.me> <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com> <w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me> <vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me> <68b83l-jtd.ln1@ID-313840.user.individual.net> <vjt866$20222$3@dont-email.me> <KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com> <vjtkuq$25l55$1@dont-email.me> <wwvy10d5rbm.fsf@LkoBDZeT.terraraq.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 18 Dec 2024 15:02:52 +0100 (CET)
Injection-Info: dont-email.me; posting-host="0a4897d36eecd2c0c7c91823a39c9faf";
logging-data="2449881"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19zJDOWBmAW4iaGVUBiKNFb"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.139 (x86_64))
Cancel-Lock: sha1:eNxZeSTWihqPeR7097XKUg+2X3Q=
View all headers

Richard Kettlewell <invalid@invalid.invalid> wrote:
> Rich <rich@example.invalid> writes:
>> 186282@ud0s4.net <186283@ud0s4.net> wrote:
>>> But you're still limited to the amount of RAM the system can
>>> access.
>>
>> Not with a pipe or FIFO, which is what is being discussed above.
>>
>> The amount of data you can transfer over a pipe is not in any way
>> limited by system memory size or any other system imposed limits.
>
> Quite. I’m not sure why this discussion has restarted but it was clear
> from last time round that some of the participants don’t know what a
> pipe is, and aren’t particularly interested in finding out.

Yes, our local nymshift troll seems to clearly not know what a pipe is,
nor care to learn either.

>>> These days that's probably a LOT - but might NOT be, esp for
>>> 'embedded' type boards like the older PIs, BBBs and such. Never
>>> assume the user has essentially infinite RAM.
>>
>> The system will not have infinite RAM. You can transfer infinite data
>> over a pipe (although it will take a while to reach infinity).
>
> I think I’d use a slightly weaker term than ‘infinite’, something will
> put an upper bound on it, even if it’s the heat death of the universe.

That was the point of the subtle hint of "will take a while to reach
infinity". Although I suppose it was too subtle there.

Subject: Re: Are We Back to the "Wars" Now ?
From: Rich
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 18 Dec 2024 14:03 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: rich@example.invalid (Rich)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 14:03:53 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 46
Message-ID: <vjuko9$2aoep$2@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com> <vhja4p$23f5e$3@dont-email.me> <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com> <w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me> <vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me> <68b83l-jtd.ln1@ID-313840.user.individual.net> <vjt866$20222$3@dont-email.me> <KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com> <vjtkuq$25l55$1@dont-email.me> <slrnvm4me1.662.spamtrap42@one.localnet>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 18 Dec 2024 15:03:53 +0100 (CET)
Injection-Info: dont-email.me; posting-host="0a4897d36eecd2c0c7c91823a39c9faf";
logging-data="2449881"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18wQwscBuQEVAvkcxP+YTcr"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.139 (x86_64))
Cancel-Lock: sha1:82N+K91zNsUOb8Cym5tArZiBNzo=
View all headers

Robert Riches <spamtrap42@jacob21819.net> wrote:
> On 2024-12-18, Rich <rich@example.invalid> wrote:
>> 186282@ud0s4.net <186283@ud0s4.net> wrote:
>>> On 12/17/24 8:23 PM, Lawrence D'Oliveiro wrote:
>>>> On Tue, 17 Dec 2024 13:34:30 +0000, Geoff Clare wrote:
>>>>
>>>>> With a pipe or FIFO, you just use simple read and write operations
>>>>> and the system handles all the messy stuff for you. If the pipe
>>>>> reaches capacity, write blocks until there is room to write some
>>>>> more; if the pipe becomes empty, read blocks until there is more
>>>>> data available; when read returns EOF that's the end of the data.
>>>>
>>>> Yup. Furthermore:
>>>>
>>>> * When the last writer closes its end, any remaining read attempts
>>>> get EOF.
>>>> * When the last reader closes its end, any remaining write attempts
>>>> get “broken pipe”.
>>>
>>> But you're still limited to the amount of RAM the system can
>>> access.
>>
>> Not with a pipe or FIFO, which is what is being discussed above.
>>
>> The amount of data you can transfer over a pipe is not in any way
>> limited by system memory size or any other system imposed limits.
>>
>>> These days that's probably a LOT - but might NOT be,
>>> esp for 'embedded' type boards like the older PIs,
>>> BBBs and such. Never assume the user has essentially
>>> infinite RAM.
>>
>> The system will not have infinite RAM. You can transfer infinite data
>> over a pipe (although it will take a while to reach infinity).
>
> A pipe is _NOT_ limited to system RAM!
>
> Using a named pipe on a Raspberry Pi model 1 with a _half_ GB of
> total RAM, I would routinely transfer _several_ GB in a single
> stream from an mplayer process to a netcat process. The only
> reason that's not currently happening every night these days is
> the amplified TV antenna lost too much gain due to age, attic
> heat, etc.

While you are correct, you responded to the wrong post. I pointed out
to the nymshift troll the exact statement you made to me.

Subject: Re: Are We Back to the "Wars" Now ?
From: John Ames
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 18 Dec 2024 16:11 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12 13
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: commodorejohn@gmail.com (John Ames)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 08:11:04 -0800
Organization: A noiseless patient Spider
Lines: 28
Message-ID: <20241218081104.00007add@gmail.com>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com>
<vhja4p$23f5e$3@dont-email.me>
<lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com>
<w0b7P.64695$oR74.19157@fx16.iad>
<vjjljh$3unfq$1@dont-email.me>
<vjk9nm$2chi$1@dont-email.me>
<vjkae9$1uri$11@dont-email.me>
<68b83l-jtd.ln1@ID-313840.user.individual.net>
<vjt866$20222$3@dont-email.me>
<KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com>
<vjtkuq$25l55$1@dont-email.me>
<wwvy10d5rbm.fsf@LkoBDZeT.terraraq.uk>
<vjukmc$2aoep$1@dont-email.me>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Injection-Date: Wed, 18 Dec 2024 17:11:08 +0100 (CET)
Injection-Info: dont-email.me; posting-host="2db07abe362f67b06a44632bfa900db0";
logging-data="2488284"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/Wmu+WtJb8yVerFgOFQcEbrT0pB5kyTEM="
Cancel-Lock: sha1:dgB97xc099UDWEP0aF2iIbq39+0=
X-Newsreader: Claws Mail 4.3.0 (GTK 3.24.42; x86_64-w64-mingw32)
View all headers

On Wed, 18 Dec 2024 14:02:52 -0000 (UTC)
Rich <rich@example.invalid> wrote:

> >> The amount of data you can transfer over a pipe is not in any way
> >> limited by system memory size or any other system imposed limits.
> >
> > Quite. I’m not sure why this discussion has restarted but it was
> > clear from last time round that some of the participants don’t know
> > what a pipe is, and aren’t particularly interested in finding out.
>
> Yes, our local nymshift troll seems to clearly not know what a pipe
> is, nor care to learn either.

I *think* what he's meaning to say is this: while you can transfer any
arbitrary amount of data *through* a pipe, there is an upper limit to
how much you can have *in* a pipe at any one time; eventually, you hit
either *A.* an OS-imposed limit on buffer size, at which point things
start blocking as already discussed, or *B.* the upper bounds of system
memory, at which point the system will either start swapping (in which
case you lose any speed advantage) or blocking (as with limited buffer
size.)

That said, what probably shouldn't need saying here is that if you're
filling up all available space in a pipe such that you're regularly
hitting these limits, you're probably doing pipes wrong.

Subject: Re: Are We Back to the "Wars" Now ?
From: Rich
Newsgroups: comp.os.linux.misc
Organization: A noiseless patient Spider
Date: Wed, 18 Dec 2024 16:51 UTC
References: 1 2 3 4 5 6 7 8 9 10 11 12
Path: eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: rich@example.invalid (Rich)
Newsgroups: comp.os.linux.misc
Subject: Re: Are We Back to the "Wars" Now ?
Date: Wed, 18 Dec 2024 16:51:50 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 46
Message-ID: <vjuuj6$2cer0$1@dont-email.me>
References: <Sp-cnSz8UupYQaf6nZ2dnZfqnPednZ2d@earthlink.com> <lNycnZghasCXPtP6nZ2dnZfqnPSdnZ2d@earthlink.com> <w0b7P.64695$oR74.19157@fx16.iad> <vjjljh$3unfq$1@dont-email.me> <vjk9nm$2chi$1@dont-email.me> <vjkae9$1uri$11@dont-email.me> <68b83l-jtd.ln1@ID-313840.user.individual.net> <vjt866$20222$3@dont-email.me> <KSGdnbSzt70r0__6nZ2dnZfqn_SdnZ2d@earthlink.com> <vjtkuq$25l55$1@dont-email.me> <wwvy10d5rbm.fsf@LkoBDZeT.terraraq.uk> <vjukmc$2aoep$1@dont-email.me> <20241218081104.00007add@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 18 Dec 2024 17:51:51 +0100 (CET)
Injection-Info: dont-email.me; posting-host="0a4897d36eecd2c0c7c91823a39c9faf";
logging-data="2505568"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18RmQ566XkmIx1t22oJs48h"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.139 (x86_64))
Cancel-Lock: sha1:dbvS0ufeXO01YrJ1qxeD6rWfsDE=
View all headers

John Ames <commodorejohn@gmail.com> wrote:
> On Wed, 18 Dec 2024 14:02:52 -0000 (UTC)
> Rich <rich@example.invalid> wrote:
>
>> >> The amount of data you can transfer over a pipe is not in any way
>> >> limited by system memory size or any other system imposed limits.
>> >
>> > Quite. I’m not sure why this discussion has restarted but it was
>> > clear from last time round that some of the participants don’t know
>> > what a pipe is, and aren’t particularly interested in finding out.
>>
>> Yes, our local nymshift troll seems to clearly not know what a pipe
>> is, nor care to learn either.
>
> I *think* what he's meaning to say is this: while you can transfer any
> arbitrary amount of data *through* a pipe, there is an upper limit to
> how much you can have *in* a pipe at any one time; eventually, you hit
> either *A.* an OS-imposed limit on buffer size, at which point things
> start blocking as already discussed, or *B.* the upper bounds of system
> memory, at which point the system will either start swapping (in which
> case you lose any speed advantage) or blocking (as with limited buffer
> size.)

Even if that is what the nymshift troll means, the second half is still
wrong when discussing pipes. There is a pipe buffer size, when you hit
that then the OS begins blocking the pipe writer to stop the flow until
the pipe reader removes some of the data.

But you never exhaust system memory via data in a pipe, the pipe API
simply does not work that way.

Where the nymshift troll's idea of pipes seems to be stuck is MSDOS 2.0
and command.com's fake pipes that were really temporary disk files.
This seems to be his concept of a pipe and he's never learned anything
nor updated his belief since MSDOS 2.0.

> That said, what probably shouldn't need saying here is that if you're
> filling up all available space in a pipe such that you're regularly
> hitting these limits, you're probably doing pipes wrong.

You can't fill system memory via data buffered inside a pipe. The pipe
will only hold whatever the system buffer space is (recent Linux's seem
to be 64k buffer size within the pipe). You'd need to create a
significant number of pipes (and never close them) to exhaust system
memory, in which case yes, you are "doing pipes wrong".

Pages:1234

rocksolid light 0.9.8
clearnet tor