I came across an issue whose origin I could not track down, neither
by constructing a test sample from scratch nor by reducing the whole
shell program. So I'm asking here just generally whether the observed
effect triggers some memories of the audience to get any hint or some
ideas what might possibly be a source of the issue. The context is...

I'm populating a set of files in an associative array (defined with
'typeset -A a_set') using
a_set["${a_file}"]=
finally I want to remove them with 'rm'. So far so good. The program
creates (for example) for these commands the respective output[*]

printf "rm '%s'\n" "${!a_set[@]}"
=>
rm 'rmd2'
rm 'rmd4'
rm 'rmd9 2'
rm 'rmd9 3'

print "${!a_set[@]}"
=>
rmd2 rmd4 rmd9 2 rmd9 3

rm "${!a_set[@]}"
=>
# no output, and correctly removes the four sample files as expected

Since that's what I actually want I could just use that and be fine.

But being paranoid (with removing files) I wanted to use for a period
of time (to obtain confidence) the individual confirmations 'rm -i'

rm -i "${!a_set[@]}"
=>
rm: remove regular file `rmd2'? rm: remove regular file `rmd4'? rm:
remove regular file `rmd9 2'? rm: remove regular file `rmd9 3'?

The 'rm -i' version is just flushing out the above line on stderr
(and does nothing else)!

(I also tried '/bin/rm -i' to be on the safe side but to no avail.)

If the single shell commands (typeset -A, assignations, rm -i) are
issued interactively the 'mv -i' works as expected; I'm getting asked
for each file whether it shall be removed or not...

(1681)$ typeset -A a_set
(1682)$ a_set["rmd2"]=
(1683)$ a_set["rmd3"]=
(1684)$ a_set["rmd9 2"]=
(1685)$ a_set["rmd9 3"]=
(1686)$ print "${!a_set[@]}"
rmd2 rmd3 rmd9 2 rmd9 3
(1687)$ rm -i "${!a_set[@]}"
rm: remove regular file `rmd2'? n
rm: remove regular file `rmd3'? n
rm: remove regular file `rmd9 2'? n
rm: remove regular file `rmd9 3'? n

Does that effect - interaction of rm -i behavior with non-interactive
ksh, flushing the confirmation messages! - sound familiar to anybody?

(I get the same behavior with 'ksh 93u' and 'ksh 93u+m', BTW.)

Janis

[*] Note: in Kornshell the expression ${!a_set[@]} expands to the
list of keys/indexes of the associative array a_set.

Il Sat, 11 Jan 2025 05:05:48 +0100, Janis Papanagnou ha scritto:

> Does that effect - interaction of rm -i behavior with non-interactive
> ksh, flushing the confirmation messages! - sound familiar to anybody?
>
> (I get the same behavior with 'ksh 93u' and 'ksh 93u+m', BTW.)

Uhmmm... here it works. But maybe I'm missing something...

>11:59:35<lem@biggy:/tmp/prova$ cat mioscript
#!/bin/ksh
unset f_names
typeset -A f_names
f_names["file1"]=
f_names["file2"]=
f_names["file3 3"]=
echo "${!f_names[@]}"
rm -i "${!f_names[@]}"
>11:59:43<lem@biggy:/tmp/prova$ ./mioscript
file1 file2 file3 3
rm: rimuovere file regolare vuoto 'file1'? n
rm: rimuovere file regolare vuoto 'file2'? n
rm: rimuovere file regolare vuoto 'file3 3'? n
>12:00:28<lem@biggy:/tmp/prova$ ksh ./mioscript
file1 file2 file3 3
rm: rimuovere file regolare vuoto 'file1'? n
rm: rimuovere file regolare vuoto 'file2'? n
rm: rimuovere file regolare vuoto 'file3 3'? n
>12:00:38<lem@biggy:/tmp/prova$ ksh --version
version sh (AT&T Research) 93u+m/1.0.8 2024-01-01
--
Bye, Lem
Talis erit dies qualem egeris

On 11.01.2025 12:06, Lem Novantotto wrote:
> Il Sat, 11 Jan 2025 05:05:48 +0100, Janis Papanagnou ha scritto:
>
>> Does that effect - interaction of rm -i behavior with non-interactive
>> ksh, flushing the confirmation messages! - sound familiar to anybody?
>>
>> (I get the same behavior with 'ksh 93u' and 'ksh 93u+m', BTW.)
>
> Uhmmm... here it works. But maybe I'm missing something...

Yes, just these commands in a script work also in my environment.
(That was what I meant when I wrote: "constructing a test sample
from scratch"; it didn't lead me anywhere, since it just wasn't
reproducible in a primitive context like that.)

I think it must have to do with the shell environment that in
some way affects how 'rm -i' behaves. But I have no idea how an
external (shell-environment-)condition could look like that makes
an executed program like 'rm' behave as if all '-i' confirmations
are "magically" considered as being each answered by "no" (without
me typing anything).

Janis

>
>> 11:59:35<lem@biggy:/tmp/prova$ cat mioscript
> #!/bin/ksh
> unset f_names
> typeset -A f_names
> f_names["file1"]=
> f_names["file2"]=
> f_names["file3 3"]=
> echo "${!f_names[@]}"
> rm -i "${!f_names[@]}"
>> 11:59:43<lem@biggy:/tmp/prova$ ./mioscript
> file1 file2 file3 3
> rm: rimuovere file regolare vuoto 'file1'? n
> rm: rimuovere file regolare vuoto 'file2'? n
> rm: rimuovere file regolare vuoto 'file3 3'? n
>> 12:00:28<lem@biggy:/tmp/prova$ ksh ./mioscript
> file1 file2 file3 3
> rm: rimuovere file regolare vuoto 'file1'? n
> rm: rimuovere file regolare vuoto 'file2'? n
> rm: rimuovere file regolare vuoto 'file3 3'? n
>> 12:00:38<lem@biggy:/tmp/prova$ ksh --version
> version sh (AT&T Research) 93u+m/1.0.8 2024-01-01
>

On 2025-01-11, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

> rm -i "${!a_set[@]}"
> =>
> rm: remove regular file `rmd2'? rm: remove regular file `rmd4'? rm:
> remove regular file `rmd9 2'? rm: remove regular file `rmd9 3'?

Well, that's what you get when you redirect stdin to null:

$ rm -i *
remove a? n
remove b? n
remove c? n
$ rm -i * </dev/null
remove a? remove b? remove c? $

--
Christian "naddy" Weisgerber naddy@mips.inka.de

In article <slrnvo4udu.a76.naddy@lorvorc.mips.inka.de>,
Christian Weisgerber <naddy@mips.inka.de> wrote:
>On 2025-01-11, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>
>> rm -i "${!a_set[@]}"
>> =>
>> rm: remove regular file `rmd2'? rm: remove regular file `rmd4'? rm:
>> remove regular file `rmd9 2'? rm: remove regular file `rmd9 3'?
>
>Well, that's what you get when you redirect stdin to null:
>
>$ rm -i *
>remove a? n
>remove b? n
>remove c? n
>$ rm -i * </dev/null
>remove a? remove b? remove c? $

Yeah, that was my first reaction as well. But it seems so obvious, that it
seems unlikely that this particular poster would have fallen into that trap.

Maybe the script is being executed in some non-normal environment, say in
cron, or in an "init" script, where stdin is redirected to /dev/null.

A couple of other comments:

1) I've found out recently that, under certain, as yet undetermined,
conditions, scripts run from .profile have stdin == /dev/null.

2) I think that "rm" should (*) open up /dev/tty to prompt for yes/no,
rather than rely on standard input. It should fail/exit with an error
message if /dev/tty can't be opened (which will happen if the process has
no controlling terminal).

(*) "should" in the sense of does not now, but the world would be better if
it did.

--
"Insisting on perfect safety is for people who don't have the balls to live
in the real world."

- Mary Shafer, NASA Ames Dryden -

On 11.01.2025 15:01, Christian Weisgerber wrote:
> On 2025-01-11, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>
>> rm -i "${!a_set[@]}"
>> =>
>> rm: remove regular file `rmd2'? rm: remove regular file `rmd4'? rm:
>> remove regular file `rmd9 2'? rm: remove regular file `rmd9 3'?
>
> Well, that's what you get when you redirect stdin to null:
>
> $ rm -i *
> remove a? n
> remove b? n
> remove c? n
> $ rm -i * </dev/null
> remove a? remove b? remove c? $
>

Yeah; only that I haven't any [explicit] redirection of stdin (let
alone a redirection to /dev/null).

But your suggestion reminds me to inspect the strace log; maybe I'll
find any hints on that...

Janis

On 11.01.2025 16:16, Kenny McCormack wrote:
> In article <slrnvo4udu.a76.naddy@lorvorc.mips.inka.de>,
> Christian Weisgerber <naddy@mips.inka.de> wrote:
>> On 2025-01-11, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>>
>>> rm -i "${!a_set[@]}"
>>> =>
>>> rm: remove regular file `rmd2'? rm: remove regular file `rmd4'? rm:
>>> remove regular file `rmd9 2'? rm: remove regular file `rmd9 3'?
>>
>> Well, that's what you get when you redirect stdin to null:
>>
>> $ rm -i *
>> remove a? n
>> remove b? n
>> remove c? n
>> $ rm -i * </dev/null
>> remove a? remove b? remove c? $
>
> Yeah, that was my first reaction as well. But it seems so obvious, that it
> seems unlikely that this particular poster would have fallen into that trap.

Sometimes it may be necessary to refresh an old man's brain. ;-)
But you are right, I'd have noticed such suspicious redirections.

>
> Maybe the script is being executed in some non-normal environment, say in
> cron, or in an "init" script, where stdin is redirected to /dev/null.

Yeah, something that I also thought about. - But in this case it's
just a normally run script (tested in ~/bin through PATH search and
with explicit ksh interpreter). - Here I thought more about any fancy
changes in ~/.profile or ~/.kshrc that might have affected behavior.
But yet I haven't spotted anything apparent; environments get overly
complex, though.

>
> A couple of other comments:
>
> 1) I've found out recently that, under certain, as yet undetermined,
> conditions, scripts run from .profile have stdin == /dev/null.

Well, I'd not really expect interactive input in .profile, but if it's
not specified that is or would be bad.

> 2) I think that "rm" should (*) open up /dev/tty to prompt for yes/no,
> rather than rely on standard input. It should fail/exit with an error
> message if /dev/tty can't be opened (which will happen if the process has
> no controlling terminal).

Indeed. - But thinking about that... - using stdin might be considered
advantageous if one wants to connect a controlling tool (say, an AI -
LOL) to judge/control the command. Quite unlikely its usefulness, but
that's the one thing that occurred to me for such a design.

Janis

>
> (*) "should" in the sense of does not now, but the world would be better if
> it did.
>

Christian Weisgerber <naddy@mips.inka.de> writes:
> On 2025-01-11, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> rm -i "${!a_set[@]}"
>> =>
>> rm: remove regular file `rmd2'? rm: remove regular file `rmd4'? rm:
>> remove regular file `rmd9 2'? rm: remove regular file `rmd9 3'?
>
> Well, that's what you get when you redirect stdin to null:
>
> $ rm -i *
> remove a? n
> remove b? n
> remove c? n
> $ rm -i * </dev/null
> remove a? remove b? remove c? $

Which raises the question of what rm -i *should* do if stdin isn't
a tty.

POSIX specifically says that "rm shall write a prompt to standard
error and read a line from the standard input".

Changing the current behavior would (a) violate POSIX, and (b) break
usages like "yes | some_script", where "some_script" invokes rm -i.
The current behavior is well defined, even if it may not be ideal.

I can see the usefulness of, for example, bailing out with an error
message if stdin is not a tty or repeating the prompt if the input is
neither 'y' nor 'n' (or whatever the locale-specific equivalents are),
but if you want that you can program it yourself (or add it as a
non-standard option to rm).

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */

Il Sat, 11 Jan 2025 15:07:05 +0100, Janis Papanagnou ha scritto:

> Yes, just these commands in a script work also in my environment. (That
> was what I meant when I wrote: "constructing a test sample from
> scratch";
> it didn't lead me anywhere, since it just wasn't reproducible in a
> primitive context like that.)
>
> I think it must have to do with the shell environment that in some way
> affects how 'rm -i' behaves. But I have no idea how an external
> (shell-environment-)condition could look like that makes an executed
> program like 'rm' behave as if all '-i' confirmations are "magically"
> considered as being each answered by "no" (without me typing anything).

Oh, I'm sorry for my mistake.

Letting alone the redirection of standard input, or the "cron or whatever"
hypothesis, that didn't seem to fit your situation, and since the command
works well itself in a primitive context, it seem strange to me that the
cause can be found in your ksh configuration, too...

So, I can't think of anything... which doesn't impress me much: it happens
frequently. ;)

However: if you add in the script "< /dev/tty" as in

rm -i file < /dev/tty

does it make any change?

rm -i is called in a pipe? Exec? Subshell?

I know you know. I'm out of ideas.
--
Bye, Lem
Talis erit dies qualem egeris

On 12.01.2025 12:05, Lem Novantotto wrote:
> Il Sat, 11 Jan 2025 15:07:05 +0100, Janis Papanagnou ha scritto:
>
>> Yes, just these commands in a script work also in my environment. (That
>> was what I meant when I wrote: "constructing a test sample from
>> scratch";
>> it didn't lead me anywhere, since it just wasn't reproducible in a
>> primitive context like that.)
>>
>> I think it must have to do with the shell environment that in some way
>> affects how 'rm -i' behaves. But I have no idea how an external
>> (shell-environment-)condition could look like that makes an executed
>> program like 'rm' behave as if all '-i' confirmations are "magically"
>> considered as being each answered by "no" (without me typing anything).
>
> Oh, I'm sorry for my mistake.

No worries. :-)

>
> Letting alone the redirection of standard input, or the "cron or whatever"
> hypothesis, that didn't seem to fit your situation, and since the command
> works well itself in a primitive context, it seem strange to me that the
> cause can be found in your ksh configuration, too...
>
> So, I can't think of anything... which doesn't impress me much: it happens
> frequently. ;)
>
> However: if you add in the script "< /dev/tty" as in
>
> rm -i file < /dev/tty
>
> does it make any change?

Erm, yes. - Then it just works. :-)

I'm now feeling a bit stupid; we were discussing the effect of some
(possibly invisible) redirection without me taking the constructive
approach to just invent that redirection as you propose it. Thanks!

The previously observed effect still unsettles me, but since I want
just a working solution the simple workaround/fix is perfect!

>
> rm -i is called in a pipe? Exec? Subshell?

It's called in a (ksh-)function within a structure like this

function f { ... rm ... }
p1 | p2 | while read ... do ... f ... done
f

>
> I know you know. I'm out of ideas.

One fitting idea suffices :-) - Thanks again!

Janis

In article <vm0ela$15i9h$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
....
>It's called in a (ksh-)function within a structure like this
>
> function f { ... rm ... }
> p1 | p2 | while read ... do ... f ... done
> f

Well, that explains everything.

And you're going to get a dozen responses just like this one, pointing
out the obvious.

When you are in a "... | while read" loop, the stuff inside the while loop
has stdin coming from the pipe. Yes, I've been bitten by this a few times,
and it is one of the (many) reasons I avoid the "... | while read" idiom.
There are always better/cleaner alternatives.

BTW, I was going to suggest that the best way to debug this was, assuming
you're running Linux, to add code just before the "rm" to do a "ls -lsa
/proc/self/fd/0" and you'll then know what fd 0 is. My guess is that that
would show that fd 0 is something like: /proc/self/fd/0 -> pipe:[123456789]

A couple of comments:

1) Keith made some good points about the possibility of changing how
"rm" behaves when stdin is not a tty. Looks like we'll have to get
POSIX changed before we can change the program, although the idea
of adding a new flag (maybe -I instead of (i.e., in addition to) -i)
has merit.

2) I've always thought that adding -i to rm as a safety feature was a
bad idea. This is a total aside, but it seems common for system
admins to setup a global alias that aliases "rm" to "rm -i". It
seems to me that if you want to make "rm" safer, you should alias
it to "rm -v" instead. "rm -i" makes it hard to do large deletes
(like removing a whole directory) - since you get prompted over and
over and over. With "rm -v", the users will know what they've done
and will learn quick to be careful. I.e., the real danger with
"rm" without "-v" is that you can delete stuff without knowing it,
and only find out (much) later.

--
Religion is what keeps the poor from murdering the rich.

- Napoleon Bonaparte -

On 12.01.2025 15:14, Kenny McCormack wrote:
> In article <vm0ela$15i9h$1@dont-email.me>,
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> ...
>> It's called in a (ksh-)function within a structure like this
>>
>> function f { ... rm ... }
>> p1 | p2 | while read ... do ... f ... done
>> f
>
> When you are in a "... | while read" loop, the stuff inside the while loop
> has stdin coming from the pipe.

First I thought you meant bash's sub-shell issue (which isn't present
in ksh). But you are right about stdin; I completely missed that. Doh!
Thanks for spotting that!

> Yes, I've been bitten by this a few times,
> and it is one of the (many) reasons I avoid the "... | while read" idiom.
> There are always better/cleaner alternatives.

(I know that you can circumvent bash's sub-shell issue with while/read
e.g. by using process substitution

while read ... do ... f ... done < <( p1 | p2 )

But - in case you were thinking about that idiom - that would not solve
the redirection issue.)

One way to fix the redirection issue would be to duplicate the 'stdin'
file descriptor; to start the script with
exec 4<&0
and then call the remove command referring explicitly to that other FD
using
0<&4 /bin/rm -i ...

What (other, "better/cleaner") idioms were you thinking of?

Janis

On 12.01.2025 16:39, Janis Papanagnou wrote:
> On 12.01.2025 15:14, Kenny McCormack wrote:
>> In article <vm0ela$15i9h$1@dont-email.me>,
>> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> ...
>>> It's called in a (ksh-)function within a structure like this
>>>
>>> function f { ... rm ... }
>>> p1 | p2 | while read ... do ... f ... done
>>> f
>>
>> When you are in a "... | while read" loop, the stuff inside the while loop
>> has stdin coming from the pipe.

BTW; there's two calls of 'f', one in the loop (without stdin),
the other outside (supposedly having stdin). - I wonder why the
latter also didn't work; it neither showed the flushed data nor
interrogated the interactive confirmation for the final data.
(Am I still missing something?)

Janis

Il Sun, 12 Jan 2025 17:08:03 +0100, Janis Papanagnou ha scritto:

> BTW; there's two calls of 'f', one in the loop (without stdin),
> the other outside (supposedly having stdin). - I wonder why the latter
> also didn't work; it neither showed the flushed data nor interrogated
> the interactive confirmation for the final data.

Edgar Allan Janis Doyle: "Mystery tales of the KSH (Kernel BaskerScript
Hound)", where every function and variable leads us deeper into the
enigmatic labyrinth of the shell's secrets. ;-)

We need a Sherlock Holmes, for sure! :-)
--
Bye, Lem
Talis erit dies qualem egeris

In article <vm0no6$17127$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
....
>What (other, "better/cleaner") idioms were you thinking of?

In general, when I need to iterate over a set (in bash), I use "mapfile".

So, instead of:

... | while read ...

I will do:

mapfile -t < <(...)
for i in "${MAPFILE[@]}"; do
...
done

I think more or less the same is available in ksh.

Oh, and another thing of interest to the original thread. We were talking
about whether the real underlying problem in the "rm" command could or
should be fixed. I did some reading and some testing. "man rm" (on Linux)
does mention stdin being a terminal - implying, somewhat obliquely, that
the -i option checks that and behaves differently if stdin is not a
terminal.

However, the below does in fact remove the file (*), which suggests that the
implementation is not quite up to the man page:

$ touch /tmp/foo
$ echo y > /tmp/bar
$ rm -iv /tmp/foo < /tmp/bar

(*) With the expected weird onscreen display/look.

--
Republican Congressman Matt Gaetz claims that only ugly women want
abortions, which they will never need since no one will impregnate them.

On 13.01.2025 03:54, Kenny McCormack wrote:
>
> In general, when I need to iterate over a set (in bash), I use "mapfile".
> [...]
> I think more or less the same is available in ksh.

Not that I'd know of. - Maybe getting something similar using some
standard mechanisms? - If I understand 'mapfile' correctly it may
(basically) do something like IFS=$'\n' arr=( $( ... ) ) ?

>
> Oh, and another thing of interest to the original thread. We were talking
> about whether the real underlying problem in the "rm" command could or
> should be fixed. I did some reading and some testing. "man rm" (on Linux)
> does mention stdin being a terminal - implying, somewhat obliquely, that
> the -i option checks that and behaves differently if stdin is not a
> terminal.

In case stdin is not a terminal I'd not expect the flushing of the
prompts; in case that condition is even checked I'd rather expect
a warning message. (But I haven't thought about that to any depth.)

Janis

> [...]

Il Mon, 13 Jan 2025 02:54:02 -0000 (UTC), Kenny McCormack ha scritto:

> So, instead of:
>
> ... | while read ...
>
> I will do:
>
> mapfile -t < <(...)
> for i in "${MAPFILE[@]}"; do
> ...
> done

Your solution has its advantages, but IMHO it has many drawbacks too.
It's more memory consuming, less portable, it doesn't react promptly...
And you cannot follow and process in real time a streams, of course.
You cannot do something like:

$ while true; do echo yes; done |while read $w; do echo "$w and no"; done
--
Bye, Lem
Talis erit dies qualem egeris

Il Mon, 13 Jan 2025 20:39:54 -0000 (UTC), Lem Novantotto ha scritto:

some typos

...stream...

...while read w; do echo "$w and no"; done...

Sorry.
--
Bye, Lem
Talis erit dies qualem egeris

Lem Novantotto <Lem@none.invalid> writes:
[...]
> $ while true; do echo yes; done |while read $w; do echo "$w and no"; done

"while true; do echo yes; done" can be equivalently written as "yes yes".

The "yes" command prints its arguments repeatedly. With no argument, it
prints "y" repeatedly.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */

Il Mon, 13 Jan 2025 17:13:06 -0800, Keith Thompson ha scritto:

> "while true; do echo yes; done" can be equivalently written as "yes
> yes".

(true && false || (yes yes | while read line; do echo -n "$line "; count=$((count + 1)); if [ "$count" -eq 2 ]; then echo; break; fi; done | sed -e 's/^./\U&/' -e 's/ /, /'| head -n 1 && echo "is much better." && sleep 1 && seq 1 1 | tail -n 1 > /dev/null && cat /dev/zero | head -c 0 && printf " Thanks. ;-)))\n")) | tr -d '\n'; echo
--
Bye, Lem
Talis erit dies qualem egeris

On 14.01.2025 10:08, Lem Novantotto wrote:
> Il Mon, 13 Jan 2025 17:13:06 -0800, Keith Thompson ha scritto:
>
>> "while true; do echo yes; done" can be equivalently written as "yes
>> yes".
>
> (true && false || (yes yes | while read line; do echo -n "$line "; count=$((count + 1)); if [ "$count" -eq 2 ]; then echo; break; fi; done | sed -e 's/^./\U&/' -e 's/ /, /'| head -n 1 && echo "is much better." && sleep 1 && seq 1 1 | tail -n 1 > /dev/null && cat /dev/zero | head -c 0 && printf " Thanks. ;-)))\n")) | tr -d '\n'; echo

LOL! - It's good I have an interpreter to understand that answer
without the necessity for a code analysis. ;-)

Janis