I recently replaced all original ksh93u+ by ksh93u+m, and meanwhile
noticed quite some deficiencies, things that worked in original ksh
but not any more in ksh93u+m.

The latest observation is that in interactive mode with 'set -o vi'
I was able to write a command substitution, say,

ls $(date)

and with the cursor positioned (e.g.) at the closing parenthesis I
could expand the argument by Esc-* to obtain

ls 2024\-05\-21

This does not seem to work in ksh93u+m. - Is there a chance to get
original ksh behavior back?

Thanks.

Janis

Op 21-05-2024 om 17:57 schreef Janis Papanagnou:
> I recently replaced all original ksh93u+ by ksh93u+m, and meanwhile
> noticed quite some deficiencies, things that worked in original ksh
> but not any more in ksh93u+m.

I would of course be very interested in learning what deficiencies (other than
the below) you've been observing.

> The latest observation is that in interactive mode with 'set -o vi'
> I was able to write a command substitution, say,
>
> ls $(date)
>
> and with the cursor positioned (e.g.) at the closing parenthesis I
> could expand the argument by Esc-* to obtain
>
> ls 2024\-05\-21
>
> This does not seem to work in ksh93u+m. - Is there a chance to get
> original ksh behavior back?

"Subshell expansion" was never meant to be a thing and is not documented
anywhere. The behaviour was a bug; this was not supposed to happen. It also
behaved inconsistently and crashed the shell in some cases.

For further details and rationale, please see:
https://github.com/ksh93/ksh/commit/7a2d3564
https://github.com/ksh93/ksh/issues/268

The documented behaviour of ESC-* in emacs is:

M-* Attempt pathname expansion on the current word. An asterisk is
appended if the word doesn't match any file or contain any
special pattern characters.

Pathname expansion simply has no business executing code; that's not what it's
for.

I can see how the behaviour as you describe it could come in handy, though. We
could treat it as a feature request for version 93u+m/1.1 and up. It would
have to be implemented properly. If implemented it should probably also be
bound to a new key combination.

--
|| modernish -- harness the shell
|| https://github.com/modernish/modernish
||
|| KornShell lives!
|| https://github.com/ksh93/ksh

Hi Martijn

On 22.06.2024 01:14, Martijn Dekker wrote:
> Op 21-05-2024 om 17:57 schreef Janis Papanagnou:
>> I recently replaced all original ksh93u+ by ksh93u+m, and meanwhile
>> noticed quite some deficiencies, things that worked in original ksh
>> but not any more in ksh93u+m.
>
> I would of course be very interested in learning what deficiencies
> (other than the below) you've been observing.

Unfortunately I haven't kept track. It's just that I noticed that
a couple things that worked before do not any more. Partly because
they were probably anyway only considered experimental, partly it
were features that I'd suspect are not commonly used.

Off the top of my head one was the support of ksh's alarm timers,
and another thing was in context of discipline functions used on
shell variables. (In the latter case I have no certainty whether
it's a shell issue or whether I should have done this differently.
I detected it with my "kosta" code, a ksh variant of Kaz' "basta"
for bash, only with features from ksh implemented.) Another thing
is how variables are expanded in Vi-mode.

(I will need some time to reproduce and collect the information.)

>
>> The latest observation is that in interactive mode with 'set -o vi'
>> I was able to write a command substitution, say,
>>
>> ls $(date)

(This should of course have been ls $(date +%F) for the output
format shown below.)

>>
>> and with the cursor positioned (e.g.) at the closing parenthesis I
>> could expand the argument by Esc-* to obtain
>>
>> ls 2024\-05\-21
>>
>> This does not seem to work in ksh93u+m. - Is there a chance to get
>> original ksh behavior back?
>
> "Subshell expansion" was never meant to be a thing and is not documented
> anywhere.

Kornshell had always things implemented (for experimentation or
even for production) and not always documented it [in time or
generally].

> The behaviour was a bug; this was not supposed to happen.

(I cannot see where you derive that from.)

All I can say is that because the docs had partly always been
behind of time or inconsistent it's not necessarily a bug; there
are more things implemented than documented. (Not sure about the
"latest" 93u+ but that was an observation since my ksh88 days.)

So if you want to say "all that isn't documented isn't supported"
I can hardly argue with you and will (of course) accept that.

> It
> also behaved inconsistently and crashed the shell in some cases.

The features I used behaved well. But I agree that some usages
showed inconsistencies (a general issue with old ksh; that's
why I'm so happy that you provided a thoroughly fixed ksh93u+m).

>
> For further details and rationale, please see:
> https://github.com/ksh93/ksh/commit/7a2d3564
> https://github.com/ksh93/ksh/issues/268
>
> The documented behaviour of ESC-* in emacs is:

I cannot speak for Emacs-mode since I'm doing everything in
Vi-mode (as indicated above).

>
> M-* Attempt pathname expansion on the current word. An
> asterisk is
> appended if the word doesn't match any file or contain any
> special pattern characters.
>
> Pathname expansion simply has no business executing code; that's not
> what it's for.

The expansion I mentioned was not the only expansion issue.

But first let me explain a short difference between pathname
expansion and command expansion as it was. If I now type

x=$(date +%F)<Esc>*

then <Esc>* is expanding all files. I would have expected (as
it had been before with ksh93u+) that a filename expansion
would appear if I had typed

x=$(date +%F)*<Esc>*

i.e. if I had provided a glob pattern that the Vi command
<Esc>* could then expand, or if the cursor would be on a
word (a file or command) to become expanded (by globbing
and PATH expansion, respectively). Now in ksh93u+m both
depicted commands expand the same way (and lists all files
in the working directory; not very useful, and luckily
there is an undo command to let that filename pollution
vanish).

(I think it's also okay, as it currently works already,
to expand all filenames on a space x=$(date +%F) <Esc>*
but I think [from my perspective] it's a rarer case.)

Moreover, if the cursor is on the opening parenthesis the
<Esc>* will insert an escaped '\*' despite being in command
mode (and not in input mode).

Variable expansion does also misbehave (I think already in
original ksh93u+). With

y='ABC DEF'

z=$y<Esc>*

will expand to

z=ABC DEF

and trying to add quotes

z="$y"<Esc>*

will do nothing. And going back one character to the 'y'
and then typing <Esc>* will produce

z=ABC DEF"

In my daily work with the Kornshell these expansions were
really useful (modulo the bugs/inconsistencies of course).
But of course I cannot expect that features that are (maybe,
haven't checked) not (not well) documented to be supported.
Especially if there were (as you also say) inconsistencies
or bugs already in the original Kornshell.

>
> I can see how the behaviour as you describe it could come in handy,
> though. We could treat it as a feature request for version 93u+m/1.1 and
> up. It would have to be implemented properly. If implemented it should
> probably also be bound to a new key combination.

I found it intuitive as it could be used before in ksh93u+,
so I'm not very keen if another key combination will be
invented. (My muscle memory will certainly fool me then.)

Wouldn't it be consistently describable depending on the
context? (I think it would.) On a command fragment or in
a command position a pathname expansion will be done, on
other word fragments (or globbing characters) a globbing
expansion will be done, on a $ prefixed variable name a
variable expansion will be done (ideally with a correct
quotung), and on a shell construct character (like the
parentheses or dollar in $(...) a command expansion will
be done. - This is not yet thought through, but it
resembles the previous behavior modulo some adjustments
to address some previously existing inconsistency.

Janis

On 22.06.2024 04:10, Janis Papanagnou wrote:
> On 22.06.2024 01:14, Martijn Dekker wrote:
>> Op 21-05-2024 om 17:57 schreef Janis Papanagnou:
>>> I recently replaced all original ksh93u+ by ksh93u+m, and meanwhile
>>> noticed quite some deficiencies, things that worked in original ksh
>>> but not any more in ksh93u+m.
>>
>> I would of course be very interested in learning what deficiencies
>> (other than the below) you've been observing.
>
> Unfortunately I haven't kept track. It's just that I noticed that
> a couple things that worked before do not any more. Partly because
> they were probably anyway only considered experimental, partly it
> were features that I'd suspect are not commonly used.
>
> Off the top of my head one was the support of ksh's alarm timers,
> and another thing was in context of discipline functions used on
> shell variables. (In the latter case I have no certainty whether
> it's a shell issue or whether I should have done this differently.

An example for the above mentioned effect shows up like that...

$ pushftp
pushftp: LINES.set: line 44: .kosta.rows=24: no parent
pushftp: COLUMNS.set: line 45: .kosta.cols=79: no parent
pushftp: line 11: 1: pushftp file ftp-path [ domain ]

where pushftp is an 'expect' wrapper script

file=${1:?"${usage}"}
path=${2:?"${usage%/}"}
domain=${3:-"some-domain"}
path=${domain}/${path}
expect <<EOT
...
EOT

and the "kosta" code contains

function LINES.set { .kosta.rows=${.sh.value} ; .kosta.update ;}
function COLUMNS.set { .kosta.cols=${.sh.value} ;}
function PS1.get { .kosta.compose ;}
namespace kosta
{
...
}

and it's activated by sourcing it from ~/.kshrc

I can't tell whether I have missed something with discipline functs
on built-in variables, how I used the namespaces, or something like
that. (My tries to fix that in my program logic all failed.) Though
the above "no parent" error messages did not show up with ksh93u+.
And it can now be observed in various _script interaction_ contexts.

Janis

On 22.06.2024 14:15, Janis Papanagnou wrote:
> On 22.06.2024 04:10, Janis Papanagnou wrote:
>> On 22.06.2024 01:14, Martijn Dekker wrote:
>>> Op 21-05-2024 om 17:57 schreef Janis Papanagnou:
>>>> I recently replaced all original ksh93u+ by ksh93u+m, and meanwhile
>>>> noticed quite some deficiencies, things that worked in original ksh
>>>> but not any more in ksh93u+m.
>>>
>>> I would of course be very interested in learning what deficiencies
>>> (other than the below) you've been observing.
>>
>> [...] another thing was in context of discipline functions used on
>> shell variables. [...]
>
> [...]
>
> and the "kosta" code contains
>
> function LINES.set { .kosta.rows=${.sh.value} ; .kosta.update ;}
> function COLUMNS.set { .kosta.cols=${.sh.value} ;}
> function PS1.get { .kosta.compose ;}
> namespace kosta
> {
> ...
> }
>
> and it's activated by sourcing it from ~/.kshrc
>
> I can't tell whether I have missed something with discipline functs
> on built-in variables, how I used the namespaces, or something like
> that. (My tries to fix that in my program logic all failed.) Though
> the above "no parent" error messages did not show up with ksh93u+.
> And it can now be observed in various _script interaction_ contexts.

There's another effect in this context.

Previously with ksh93u+ the "kosta" code called '.kosta.compose' if
'PS1.get' gets called; the compose function reconstructs the status
line. If an error occurred or a signal arrived that was indicated
in the status line as, say, "ERR 2!" or "SIG INT(2)!". After I hit
the <Enter> key (an empty line) that part of the status message got
cleared. - Not any more with ksh93u+m. It seem that the 'compose'
function won't be called any more on an empty shell input line.

Janis

Op 22-06-2024 om 13:15 schreef Janis Papanagnou:
> On 22.06.2024 04:10, Janis Papanagnou wrote:
>> On 22.06.2024 01:14, Martijn Dekker wrote:
>>> Op 21-05-2024 om 17:57 schreef Janis Papanagnou:
>>>> I recently replaced all original ksh93u+ by ksh93u+m, and meanwhile
>>>> noticed quite some deficiencies, things that worked in original ksh
>>>> but not any more in ksh93u+m.
>>>
>>> I would of course be very interested in learning what deficiencies
>>> (other than the below) you've been observing.
>>
>> Unfortunately I haven't kept track. It's just that I noticed that
>> a couple things that worked before do not any more. Partly because
>> they were probably anyway only considered experimental, partly it
>> were features that I'd suspect are not commonly used.
>>
>> Off the top of my head one was the support of ksh's alarm timers,

The undocumented 'alarm' built-in was removed in 93u+m/1.0 because it was
broken and could easily crash the shell; Korn kept it in for years but never
fixed it. My standards are higher than that. I prefer not to ship stuff that I
know is broken.

It is still present on the dev branch which will eventually become 93u+m/1.1,
though. If we manage to fix it properly, then it will reappear in that
release, but I can't promise it.

>> and another thing was in context of discipline functions used on
>> shell variables. (In the latter case I have no certainty whether
>> it's a shell issue or whether I should have done this differently.
>
> An example for the above mentioned effect shows up like that...
>
> $ pushftp
> pushftp: LINES.set: line 44: .kosta.rows=24: no parent
> pushftp: COLUMNS.set: line 45: .kosta.cols=79: no parent
> pushftp: line 11: 1: pushftp file ftp-path [ domain ]
[...]
> I can't tell whether I have missed something with discipline functs
> on built-in variables, how I used the namespaces, or something like
> that. (My tries to fix that in my program logic all failed.) Though
> the above "no parent" error messages did not show up with ksh93u+.

The fix for this is a prior 'typeset .kosta' (or even just '.kosta=') to
create the parent variable.

To the best of my knowledge, this is exactly the same in 93u+ 2012-08-01:

$ /bin/ksh -c 'echo ${.sh.version}; .kosta.rows=24'
Version AJM 93u+ 2012-08-01
/bin/ksh: .kosta.rows=24: no parent
$ /bin/ksh -c 'typeset .kosta; .kosta.rows=24; echo ${.kosta.rows}'
24

--
|| modernish -- harness the shell
|| https://github.com/modernish/modernish
||
|| KornShell lives!
|| https://github.com/ksh93/ksh

Thanks for your post, Martijn!

On 02.07.2024 05:35, Martijn Dekker wrote:
> Op 22-06-2024 om 13:15 schreef Janis Papanagnou:
[...]
>>
>> An example for the above mentioned effect shows up like that...
>>
>> $ pushftp
>> pushftp: LINES.set: line 44: .kosta.rows=24: no parent
>> pushftp: COLUMNS.set: line 45: .kosta.cols=79: no parent
>> pushftp: line 11: 1: pushftp file ftp-path [ domain ]
> [...]
>> I can't tell whether I have missed something with discipline functs
>> on built-in variables, how I used the namespaces, or something like
>> that. (My tries to fix that in my program logic all failed.) Though
>> the above "no parent" error messages did not show up with ksh93u+.
>
> The fix for this is a prior 'typeset .kosta' (or even just '.kosta=') to
> create the parent variable.

Adding this typeset doesn't change the observed behavior
with my test case (neither in u+ nor in u+m).

>
> To the best of my knowledge, this is exactly the same in 93u+ 2012-08-01:
>
> $ /bin/ksh -c 'echo ${.sh.version}; .kosta.rows=24'
> Version AJM 93u+ 2012-08-01
> /bin/ksh: .kosta.rows=24: no parent

A plain assignment to a namespace variable doesn't trigger
the "no parent" error in my environment.

> $ /bin/ksh -c 'typeset .kosta; .kosta.rows=24; echo ${.kosta.rows}'
> 24
>

But calling any script does; two examples

(1000)$ pushftp # called in my ksh93u+m shell context
pushftp: LINES.set: line 46: .kosta.rows=24: no parent
pushftp: COLUMNS.set: line 47: .kosta.cols=80: no parent
pushftp: line 11: 1: pushftp file ftp-path [ domain ]
(1001)$ ksh93u+
(1002)$ pushftp
pushftp: line 11: 1: pushftp file ftp-path [ domain ]
(1003)$ # here I'm leaving ksh93u+ with ^D
(1003)$ pushftp # again in ksh93u+m
pushftp: LINES.set: line 46: .kosta.rows=24: no parent
pushftp: COLUMNS.set: line 47: .kosta.cols=80: no parent
pushftp: line 11: 1: pushftp file ftp-path [ domain ]
(1004)$ Version AJM 93u+m/1.0.8 2024-01-01 # Esc ^V

(1004)$ isodate # 'isodate' contains: date "+%F" "$@"
isodate: LINES.set: line 46: .kosta.rows=24: no parent
isodate: COLUMNS.set: line 47: .kosta.cols=80: no parent
2024-07-02
(1005)$ ksh93u+
(1006)$ isodate
2024-07-02
(1007)$

I have no idea why there's a difference between u+ and u+m here.

Janis