Hi there !

Consider the following commands:

$ var1=`uname -sr`
$ echo $var1
Darwin 24.0.0
$ read var2 <<< `uname -sr`
$ echo $var2
Darwin 24.0.0
$ uname -sr | read var3
$ echo $var3

$ uname -sr | read -p var3
$ echo $var3

$

While the first two ones behave like expected, I wonder why the latter
ones fail. What's the difference behind the scenes?

And even more confusing, why does this familiar one work anyway?

$ sw_vers | while read line; do echo $line; done
ProductName: macOS
ProductVersion: 15.0.1
BuildVersion: 24A348
$

I've been using commands like that one for a very long time and that's
why I tried the simple "read" above - with no success.

How can I do such an assignment at the end of a command instead of the
beginning? Any ideas?

TIA

fw

On 2024-10-11 2:11 p.m., Frank Winkler wrote:
> Hi there !
>
> Consider the following commands:
>
>
> $ var1=`uname -sr`
> $ echo $var1
> Darwin 24.0.0
> $ read var2 <<< `uname -sr`
> $ echo $var2
> Darwin 24.0.0
> $ uname -sr | read var3
> $ echo $var3
>
> $ uname -sr | read -p var3
> $ echo $var3
>
> $
>
> While the first two ones behave like expected, I wonder why the latter
> ones fail. What's the difference behind the scenes?

I don't know about other shells, but in Bash each command in a pipeline
is run in a subshell. (See the "Pipelines" section of the Bash man
page.) Thus you're doing the 'read var3' part in a different shell than
where 'echo $var3' runs. That's why it is empty when you echo it.

> And even more confusing, why does this familiar one work anyway?
>
> $ sw_vers | while read line; do echo $line; done
> ProductName: macOS
> ProductVersion: 15.0.1
> BuildVersion: 24A348

Here the subshell runs everything between 'while' and 'done' so the read
and echo commands are in the same (sub)shell this time.

On 11.10.2024 20:27, John-Paul Stewart wrote:

>I don't know about other shells, but in Bash each command in a pipeline
>is run in a subshell. (See the "Pipelines" section of the Bash man
>page.) Thus you're doing the 'read var3' part in a different shell than
>where 'echo $var3' runs. That's why it is empty when you echo it.

That sounds very plausible - thanks for enlighting! :)
So this is not a "read" issue but rather a matter of shell instance and
hence there's no way to do the assignment at the end?

Regards

fw

Frank Winkler <usenet@f.winkler-ka.de>:
> Hi there !
>
>
> Consider the following commands:
>
>
>
> $ var1=`uname -sr`
> $ echo $var1
> Darwin 24.0.0
> $ read var2 <<< `uname -sr`
> $ echo $var2
> Darwin 24.0.0
> $ uname -sr | read var3
> $ echo $var3
>
> $ uname -sr | read -p var3
> $ echo $var3
>
> $
>
> While the first two ones behave like expected, I wonder why the
> latter ones fail. What's the difference behind the scenes?
>
>
> And even more confusing, why does this familiar one work anyway?
>
>
> $ sw_vers | while read line; do echo $line; done
> ProductName: macOS
> ProductVersion: 15.0.1
> BuildVersion: 24A348
> $
>
> I've been using commands like that one for a very long time and
> that's why I tried the simple "read" above - with no success.
>
>
> How can I do such an assignment at the end of a command instead
> of the beginning? Any ideas?
>

uname -sr | { read var3 ; echo $var3 ; }

On 11.10.2024 22:20, Helmut Waitzmann wrote:

> uname -sr | { read var3 ; echo $var3 ; }

That's exactly how I just wanted to confirm John-Paul's answer ...

$ uname -sr | ( read var3; echo $var3 )
Darwin 24.0.0
$

.... but it still doesn't solve the issue that I need the result to be
visible in the parent shell.

Regards

fw

On Fri, 11 Oct 2024 22:50:10 +0200, Frank Winkler wrote:

> ... but it still doesn't solve the issue that I need the result to be
> visible in the parent shell.

coproc { uname -sr; }
read -u ${COPROC[0]} var3
wait $COPROC_PID
echo $var3

On 2024-10-11, Frank Winkler <usenet@f.winkler-ka.de> wrote:

> $ uname -sr | ( read var3; echo $var3 )
> Darwin 24.0.0
> $
>
> ... but it still doesn't solve the issue that I need the result to be
> visible in the parent shell.

read var3 <<EOF
$(uname -sr)
EOF
echo "$var3"

--
Christian "naddy" Weisgerber naddy@mips.inka.de

On 11.10.2024 20:45, Frank Winkler wrote:
> On 11.10.2024 20:27, John-Paul Stewart wrote:
>
> >I don't know about other shells, but in Bash each command in a pipeline
> >is run in a subshell. (See the "Pipelines" section of the Bash man
> >page.) Thus you're doing the 'read var3' part in a different shell than
> >where 'echo $var3' runs. That's why it is empty when you echo it.
>
> That sounds very plausible - thanks for enlighting! :)

> So this is not a "read" issue but rather a matter of shell instance and
> hence there's no way to do the assignment at the end?

It depends on your shell. If you choose Kornshell you won't have that
issue and can write it as you've done with the output as you'd expect.

$ uname -a | read var
$ echo "$var"
Linux [...snip...]

The reason is that the last command in a pipeline will (in Kornshell)
be executed in the "current" shell context.

(In other shells you have to work around the issue as demonstrated in
other answers to your post. Some workaround are more clumsy some less.
A shorter variant of the here-document posted elsethread can be using
here-strings

$ read var <<< $(uname -a)

another method is using process substitution and redirection

$ read var < <(uname -a)

Both supported by shells like ksh, bash, zsh, but non-standard as are
some other workaround proposals that use bash-specifics like 'coproc',
that doesn't work as widely as using '<<<' or '<(...)' do.)

Janis

On Sat, 12 Oct 2024 03:59:49 +0200, Janis Papanagnou wrote:

> ... use bash-specifics like 'coproc' ...

It isn’t bash-specific.

On 12.10.2024 03:59, Janis Papanagnou wrote:

>It depends on your shell. If you choose Kornshell you won't have that
>issue and can write it as you've done with the output as you'd expect.
>
>$ uname -a | read var
>$ echo "$var"
>Linux [...snip...]
>
>The reason is that the last command in a pipeline will (in Kornshell)
>be executed in the "current" shell context.

Interesting hint! I wasn't aware that there are such differences between
the shells. And indeed, some simple tests seem to work in an interactive
ksh.

Let's see the whole story. For historical reasons, I'm actually using
ksh for almost all scripts instead of bash for interactive use - not
knowing about the fact above.

There, I run command A which is producing output and which is calling
sub-command B, also producing output. This works fine.
What I want to achieve is to grab some parts of the output and store it
in a variable but without changing the output on the screen.

So I tried something like

tty=`tty`
A | tee $tty | ... | read var

"tee `tty`" inside the command fails, so I do it outside. The output of
A is still there but B's is gone (because B doesn't know anything about
the "tee"?) and the whole thing doesn't seem to be still working. $var
is empty, though this is a ksh script and the stuff behind "tee" is also
working.
To my understanding, the default B can be changed with an option but
when I set it to "B | tee $tty", there's still no output.

AFAIR, "var=`...`" works better but as the primary job is the command
itself and the variable is just a spin-off product, I'd prefer to do the
assignment at the end. I believe it looks better then ;) ...

Probably it would also be feasible with some temp files but I try to
avoid them wherever possible.

Happy week-end!

fw

On Sat, 12 Oct 2024 09:42:22 +0200, Frank Winkler wrote:

> AFAIR, "var=`...`" works ...

POSIX also allows

$(«cmd»)

as a nicer alternative to

`«cmd»`

On 12.10.2024 09:42, Frank Winkler wrote:
> On 12.10.2024 03:59, Janis Papanagnou wrote:
>
> >It depends on your shell. If you choose Kornshell you won't have that
> >issue and can write it as you've done with the output as you'd expect.
> >
> >$ uname -a | read var
> >$ echo "$var"
> >Linux [...snip...]
> >
> >The reason is that the last command in a pipeline will (in Kornshell)
> >be executed in the "current" shell context.
>
> Interesting hint! I wasn't aware that there are such differences between
> the shells.

Yes, it's a subtle difference, but an important one; many shell users
wondered about the behavior of other shells, and you need to know the
internal information about subprocesses used in pipes to understand
why it doesn't work (in other shells), or why it does in ksh.

> And indeed, some simple tests seem to work in an interactive
> ksh.

You can rely on it with any official ksh (since ksh88) and branches of
them. (But don't count on it if you're using some inferior ksh clone.)

>
> Let's see the whole story. For historical reasons, I'm actually using
> ksh for almost all scripts instead of bash for interactive use - not
> knowing about the fact above.

From your description below I'm not sure you still have a question or
whether you're just describing what you intended to do and are fine
with ksh's pipeline behavior. (Specifically I'm unsure about the 'B'
and what you mean by "sub-command B" in your example(s).)

If something is still not working for you, please clarify.

>
> There, I run command A which is producing output and which is calling
> sub-command B, also producing output. This works fine.
> What I want to achieve is to grab some parts of the output and store it
> in a variable but without changing the output on the screen.
>
> So I tried something like
>
> tty=`tty`
> A | tee $tty | ... | read var

$ man tty
tty - print the file name of the terminal connected to standard input

The 'tty' command in your 'tee' pipeline command has no tty attached;
it reads from the pipe. That's why A | tee $(tty) | ... | read var
doesn't work as you expected. You have to grab the tty information
outside the pipeline (as you've experienced).

Janis

>
> "tee `tty`" inside the command fails, so I do it outside. The output of
> A is still there but B's is gone (because B doesn't know anything about
> the "tee"?) and the whole thing doesn't seem to be still working. $var
> is empty, though this is a ksh script and the stuff behind "tee" is also
> working.
> To my understanding, the default B can be changed with an option but
> when I set it to "B | tee $tty", there's still no output.
>
> AFAIR, "var=`...`" works better but as the primary job is the command
> itself and the variable is just a spin-off product, I'd prefer to do the
> assignment at the end. I believe it looks better then ;) ...
>
> Probably it would also be feasible with some temp files but I try to
> avoid them wherever possible.
>
> Happy week-end!
>
> fw
>

Il Sat, 12 Oct 2024 13:08:48 +0200, Janis Papanagnou ha scritto:

> (Specifically I'm unsure about the 'B' and what you mean by "sub-command
> B" in your example(s).)

Waiting for his clarification, I'm thinking that he sees that something
like:

$ input=$(tty) && echo 123456789 | tee $input | grep -o 456 | \
> tee $input | read myvar && echo "myvar is $myvar"; input= ; myvar=
123456789
456
myvar is

doesn't work, while of course:

$ input=$(tty) && read myvar < <( echo 123456789 | tee $input | \
> grep -o 456 | tee $input ) && echo "myvar is $myvar"; input= ; myvar=
123456789
456
myvar is 456

does work (BTW: here I'm in Bash).
--
Bye, Lem

On 12.10.2024 13:08, Janis Papanagnou wrote:

>From your description below I'm not sure you still have a question or
>whether you're just describing what you intended to do and are fine
>with ksh's pipeline behavior. (Specifically I'm unsure about the 'B'
>and what you mean by "sub-command B" in your example(s).)

A spawns B and grabbing the desired information from the common output
still doesn't work.

>The 'tty' command in your 'tee' pipeline command has no tty attached;
>it reads from the pipe. That's why A | tee $(tty) | ... | read var
>doesn't work as you expected. You have to grab the tty information
>outside the pipeline (as you've experienced).

That was also my understanding after the experience ;) ... but thanks
for confirmation.

Regards

fw

On 12.10.2024 14:01, Lem Novantotto wrote:

>Waiting for his clarification, I'm thinking that he sees that something
>like:
>
>$ input=$(tty) && echo 123456789 | tee $input | grep -o 456 | \
> tee $input | read myvar && echo "myvar is $myvar"; input= ; myvar=
>123456789
>456
>myvar is
>
>doesn't work

I think you're right. But why doesn't it work in ksh?

> while of course:
>
>$ input=$(tty) && read myvar < <( echo 123456789 | tee $input | \
> grep -o 456 | tee $input ) && echo "myvar is $myvar"; input= ; myvar=
>123456789
>456
>myvar is 456
>
>does work (BTW: here I'm in Bash).

I'm still thinking about the difference between "< <(...)" and "<<< `...`"

And I would be happy with this one if there was a notation "the other
way round" ;) ... something like "... >>> $var"

Regards

fw

Il Sat, 12 Oct 2024 17:57:16 +0200, Frank Winkler ha scritto:

> I think you're right. But why doesn't it work in ksh?

Uhm... It should work. Here it works, at least.

And sometimes something like that *may* work in bash, too, provided
we set:
$ shopt -s lastpipe

But my command doesn't work instead, in bash. That's why:

| lastpipe
| If set, and job control is not active, the shell runs the last command
| of a pipeline not executed in the background in the current shell
| environment.
--
Bye, Lem

On Sat, 12 Oct 2024 17:57:16 +0200, Frank Winkler wrote:

> I'm still thinking about the difference between "< <(...)" and "<<<
> `...`"

Not sure about the extra “<”, but “<(«cmd»)” gets substituted with the
name of a file (i.e. not stdin) that the process can open and read to get
the output of «cmd». Similarly “>(«cmd»)” gets substituted with the name
of a file (i.e. not stdout) that the process can open and write to feed
input to waiting «cmd».

“<<” and “<<<”, on the other hand, are for specifying alternative sources
of inline data for stdin, or you can use “«fd»<<” and “«fd»<<<” to specify
an alternative «fd» that the process will expect to find already open for
reading, to obtain that data.

On 12.10.2024 23:32, Lawrence D'Oliveiro wrote:
> On Sat, 12 Oct 2024 17:57:16 +0200, Frank Winkler wrote:
>
>> I'm still thinking about the difference between "< <(...)" and "<<<
>> `...`"
>
> Not sure about the extra “<”,

'<(...)' executes the command indicated by '...' and provides a file
descriptor, something like '/dev/fd/5', which (being effectively a
filename) can be redirected to the 'read' command.

The shell's 'read' command doesn't read from files but from stdin,
so if 'read's input is in a file or is the output of a command (as in
this case) you have to do a redirection; for the first case
read < file
and in case of a process substitution (a file named like /dev/fd/5)
read < <(some command)
which sort of "expands" to something like
read < /dev/fd/5

So it's no "extra" '<', it's just a necessary redirection for a
command that reads from 'stdin' but not from files.

Janis

> but “<(«cmd»)” gets substituted with the
> name of a file (i.e. not stdin) that the process can open and read to get
> the output of «cmd». Similarly “>(«cmd»)” gets substituted with the name
> of a file (i.e. not stdout) that the process can open and write to feed
> input to waiting «cmd».
>
> [...]

On Sat, 12 Oct 2024 23:47:34 +0200, Janis Papanagnou wrote:

> '<(...)' executes the command indicated by '...' and provides a file
> descriptor, something like '/dev/fd/5', which (being effectively a
> filename) can be redirected to the 'read' command.

It’s an actual file name. The process treats it as just another filename
argument. The fact that it encodes a file descriptor is something OS-
specific, that only code that wants to create such names (like the shell)
has to worry about.

> The shell's 'read' command doesn't read from files but from stdin

It can read from any file currently open for reading.

> So it's no "extra" '<' ...

Ah, I see. Instead of “< <(«cmd›)”, you could have just written
“0<(«cmd»)”.

On 12.10.2024 23:50, Lawrence D'Oliveiro wrote:
> On Sat, 12 Oct 2024 23:47:34 +0200, Janis Papanagnou wrote:
>
>> '<(...)' executes the command indicated by '...' and provides a file
>> descriptor, something like '/dev/fd/5', which (being effectively a
>> filename) can be redirected to the 'read' command.
>
> It’s an actual file name. The process treats it as just another filename
> argument.

Exactly. (That's what I was saying.) '/dev/fd/5' acts like a file/path
name (but only on systems that support /dev/fd mechanism).

> The fact that it encodes a file descriptor is something OS-
> specific, that only code that wants to create such names (like the shell)
> has to worry about.
>
>> The shell's 'read' command doesn't read from files but from stdin
>
> It can read from any file currently open for reading.
>
>> So it's no "extra" '<' ...
>
> Ah, I see. Instead of “< <(«cmd›)”, you could have just written
> “0<(«cmd»)”.

No.

(1388)$ read <(cat hello_world)
ksh: read: /dev/fd/3: invalid variable name
(1389)$ read < <(cat hello_world)
(1390)$ echo $REPLY
Hello world!
(1391)$ read 0<(cat hello_world)
ksh: read: /dev/fd/3: invalid variable name

Janis

Il Sat, 12 Oct 2024 17:57:16 +0200, Frank Winkler ha scritto:

> I'm still thinking about the difference between "< <(...)" and "<<<`...`"

As already pointed out by Lawrence:

1) "command2 <<<`command1`": `command2`, equal to the preferable[*]
$(command2), can be seen as the output of command2: a string. So with
<<< you are telling the shell to take this string as input on the
standard input of command2. So: execute command1, then take its output
as the input of command2. This is called *command* substitution:
substitution of a command with its output.

[*] No characters are special between parenthesis: easier.

2) "command2 < <(process_of_command1)": here process, which is the
"active running instance" of command1 - enough: remove the difference,
think of command1 and stop - is run with its output connected to a
named FIFO pipe file (or to some file in /dev/fd).
So <(process_of_command1) is expanded as the name of this file.
The first < is simple input redirection.
So: execute command1, connect the output of its process to a special
file, redirect this special file to input of command2.
This is called *process* output substitution: substitute a
process with the name of its output file.
Process substitution does work only on systems supporting named pipes
or /dev/fd/... so it's less universal than command substitution.

Some practical differences. For example, try:

$ cat <<<$(while true; do echo yes; done)

Nothing: cat is waiting for the other stuff to end. But it won't end,
in this case! Gawsh!

$ cat < <( while true; do echo yes; done)

Rock'n'roll!

> And I would be happy with this one if there was a notation "the other
> way round" ... something like "... >>> $var"

You're more the straightforward type of guy, eh! That's fine! ;)
But, sorry, '>>>' is still to come. In bash, at least. :-)
--
Bye, Lem

Il Sat, 12 Oct 2024 23:07:41 -0000 (UTC), Lem Novantotto ha scritto:

> 1) "command2 <<<`command1`": `command2`, equal to the preferable[*]
> $(command2),

Sorry:

1) "command2 <<<`command1`": `command1`, equal to the preferable[*]
$(command1)
--
Bye, Lem

Il Sat, 12 Oct 2024 23:07:41 -0000 (UTC), Lem Novantotto ha scritto:

> As already pointed out by Lawrence:

And by Janis, of course!
I apologize.
--
Bye, Lem

On 12.10.2024 17:57, I wrote:

>I think you're right. But why doesn't it work in ksh?

BTW: in an interactive ksh, the example does work:

$ input=$(tty) && echo 123456789 | tee $input | grep -o 456 | tee $input
| read myvar && echo "myvar is $myvar"; input= ; myvar=
123456789
456
myvar is 456
$

And it also does when I put this into a ksh script.

The thing in question does

tty=`tty`
sudo openconnect -b ... |\
tee $tty | grep "^Session authentication will expire at" |\
cut -d ' ' -f 7- | read end

and this completely fails. Terminal output is missing, $end is empty and
the whole command doesn't seem to work. Without the last two lines, it's
working perfectly.
"sudo" also doesn't seem to be the problem as simple tests are working.

Regards

fw

In article <vecmp3$pur$1@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>On Sat, 12 Oct 2024 03:59:49 +0200, Janis Papanagnou wrote:
>
>> ... use bash-specifics like 'coproc' ...
>
>It isn't bash-specific.

People on these newsgroups often use phrases like "bash specific" as
synonyms for "Not strictly POSIX" (*), even though the bash feature under
discussion is also found in other shells. In fact, many bash-isms,
including "coproc", came originally from ksh. I'm sure Janis knows this.

--
The key difference between faith and science is that in science, evidence that
doesn't fit the theory tends to weaken the theory (that is, make it less likely to
be believed), whereas in faith, contrary evidence just makes faith stronger (on
the assumption that Satan is testing you - trying to make you abandon your faith).