On Sat, 31 Aug 2024 09:27:44 +0100, Richard Kettlewell wrote:

> (I’ll be disappointed in extreme cases of course, e.g. filesystems that
> permit ‘/’ in filenames, but the scale of the problem can be minimized.)

The nice thing about Unicode is the alternatives it offers: so you can’t
use “/” in a filename, but you can use “∕” instead.

> I think the thing that makes it hard is not the spaces as such, but the
> tooling that makes it inconvenient to handle them, which primarily means
> Bourne shell parsing rules. The problem basically ceases to exist once
> you’re outside the shell ecosystem.
>
> The rest of Unix has evolved substantially since the 1970s but shell is
> still stuck in this particular trap. It’s like we’re still making making
> arrowheads out of flint but everything else from steel.

If you avoid newlines in filenames, Posix shells can cope with anything
else if you set “IFS=$'\n'”.

If you insist on wanting to accept those as well, then I don’t think Posix
is enough, but Bash does have facilities that help you cope.

I wrote:

> If you avoid newlines in filenames, Posix shells can cope with anything
> else if you set “IFS=$'\n'”.

Sorry, no, it looks like the “$'...'” syntax for string literals is not
from Posix, it’s a Bash-ism.

I think it’s still possible to assign a newline to $IFS, it just takes a
bit more work.

On 2024-09-01, Lawrence D'Oliveiro wrote:

> I wrote:
>
>> If you avoid newlines in filenames, Posix shells can cope with anything
>> else if you set “IFS=$'\n'”.
>
> Sorry, no, it looks like the “$'...'” syntax for string literals is not
> from Posix, it’s a Bash-ism.
>
> I think it’s still possible to assign a newline to $IFS, it just takes a
> bit more work.

Perhaps using the printf utility? It is in IEEE 1003.1 (p3049 in -2008).

--
Nuno Silva

Lawrence D'Oliveiro <ldo@nz.invalid>:
> I wrote:
>
>> If you avoid newlines in filenames, Posix shells can cope with
>> anything else if you set “IFS=$'\n'”.
>>
>
> Sorry, no, it looks like the “$'...'” syntax for string literals
> is not from Posix, it’s a Bash-ism.
>
>
> I think it’s still possible to assign a newline to $IFS, it just
> takes a bit more work.
>

newline="$( printf '%b.' '\n' )" && newline="${newline%?}"

As this is a problem with the shell, I suggest Followup-To
"comp.unix.shell"

Lawrence D'Oliveiro <ldo@nz.invalid>:
> On Sat, 31 Aug 2024 09:27:44 +0100, Richard Kettlewell wrote:
>
>> I think the thing that makes it hard is not the spaces as such,
>> but the tooling that makes it inconvenient to handle them,
>> which primarily means Bourne shell parsing rules. The problem
>> basically ceases to exist once you’re outside the shell
>> ecosystem.
>>
>>
>> The rest of Unix has evolved substantially since the 1970s but
>> shell is still stuck in this particular trap. It’s like we’re
>> still making making arrowheads out of flint but everything else
>> from steel.
>>
>
> If you avoid newlines in filenames, Posix shells can cope with
> anything else if you set “IFS=$'\n'”.
>
>
> If you insist on wanting to accept those as well, then I don’t
> think Posix is enough, but Bash does have facilities that help
> you cope.
>

If one wants to cope with any character (including newline
characters) in the POSIX shell's command line, one could make use
of the apostroph quoting mechanism.

If one wants to process arbitrary (may be generated) lists of
filenames, one could make use of the "xargs" utility with option
"-0".

To generate those lists of filenames, there are

find ... -print0

and

printf '%s\0' ...

available.

Followup-To: comp.unix.shell

Lawrence D'Oliveiro <ldo@nz.invalid>:
> I wrote:
>
>> If you avoid newlines in filenames, Posix shells can cope with
>> anything else if you set “IFS=$'\n'”.
>>
>
> Sorry, no, it looks like the “$'...'” syntax for string literals
> is not from Posix, it’s a Bash-ism.
>

Thanks to the newest POSIX standard, this is part of the
standard:

<https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html#tag_19_02_04>

Followup-To: comp.unix.shell

On 9/1/2024 3:03 AM, Lawrence D'Oliveiro wrote:
> I wrote:
>
>> If you avoid newlines in filenames, Posix shells can cope with anything
>> else if you set “IFS=$'\n'”.
>
> Sorry, no, it looks like the “$'...'” syntax for string literals is not
> from Posix, it’s a Bash-ism.

Actually, dollar quotes were added to POSIX as of SUS Issue 8:

"2.2.4 Dollar-Single-Quotes
A sequence of characters starting with a <dollar-sign> immediately followed
by a single-quote ($') shall preserve the literal value of all characters up to
an unescaped terminating single-quote ('), with the exception of certain
<backslash>-escape sequences, as follows:
...."

<https://pubs.opengroup.org/onlinepubs/9799919799/>

--
Wayne

On Tue, 3 Sep 2024 13:56:08 -0400, Wayne wrote:

> On 9/1/2024 3:03 AM, Lawrence D'Oliveiro wrote:
>
>> Sorry, no, it looks like the “$'...'” syntax for string literals is not
>> from Posix, it’s a Bash-ism.
>
> Actually, dollar quotes were added to POSIX as of SUS Issue 8:

Glad to know. I was looking at the man page for dash(1) as my reference
for a vanilla POSIX shell.

<https://manpages.debian.org/1/dash.1.en.html>

On 01.09.2024 09:03, Lawrence D'Oliveiro wrote:
> I wrote:
>
>> If you avoid newlines in filenames, Posix shells can cope with anything
>> else if you set “IFS=$'\n'”.
>
> Sorry, no, it looks like the “$'...'” syntax for string literals is not
> from Posix, it’s a Bash-ism.

Note that it's not a "Bash-ism" - Bash was rarely the _inventor_ of
any shell features, mostly adopted them from other shells, primarily
from Kornshell (still, by far, not catching up).

Kornshell documented that specific feature with its 1993 version.

And we've learned from other replies that it's meanwhile - 30 years
later - already in POSIX. (Good news!)

Janis