(*) Summary

I wrote a sed script that makes a line replacement after it finds the
right spot. So far so good. Then I added quit command after the
change, but the quit does not seem to take effect---violating my
expectation. I'll appreciate any help on understanding what's going on.

(*) A detailed description

I wrote this program:

--8<-------------------------------------------------------->8---
%cat make-release
#!/bin/sh
usage()
{ printf '%s tag file\n' $0
exit 1
} test $# '<' 2 && usage
tag="$1"
shift
sed "/<<Release>>=/ {
n;
c\\
$tag
}" $*
--8<-------------------------------------------------------->8---

Here's how I use it. My objective with it is to replace that
/something/ in the text file with a new argument.

--8<-------------------------------------------------------->8---
%cat sample.txt
Lorem ipsum dolor...

<<Release>>=
something
@

.... sit a met [...]
% --8<-------------------------------------------------------->8---

Here's how I invoke it:

--8<-------------------------------------------------------->8---
%sh make-release release1 sample.txt
Lorem ipsum dolor...

<<Release>>=
release1
@

.... sit a met [...]
--8<-------------------------------------------------------->8---

So far so good. I decided to try it on longer files and I wanted to see
the change more quickly (without long files scrolling past my terminal),
so I decided to add a /q/ command right after the c commmand. I
thought---it will make sed quit right after making the change, so I can
see it works as desired and then I remove the /q/ and release it to
production. But that did not happen.

--8<-------------------------------------------------------->8---
%cat make-release
#!/bin/sh
usage()
{ printf '%s tag file\n' $0
exit 1
} test $# '<' 2 && usage
tag="$1"
shift
sed "/<<Release>>=/ {
n;
c\\
$tag
q}" $*
--8<-------------------------------------------------------->8---

I still see the whole file:

--8<-------------------------------------------------------->8---
%sh make-release release1 sample.txt
Lorem ipsum dolor...

<<Release>>=
release1
@

.... sit a met [...]
% --8<-------------------------------------------------------->8---

I failed the exercise I gave myself. Can you help me to understand why
the q command isn't stopping sed as I thought it would? I'd like to get
a better intuition.

I've been reading Dale Dougherty and Arnold Robin's ``sed & awk'' book.
If you have any recommended sed-related bibliography, I'd appreciate it,
too.

On 2024-12-18 2:46 p.m., Salvador Mirzo wrote:
> (*) Summary
>
> I wrote a sed script that makes a line replacement after it finds the
> right spot. So far so good. Then I added quit command after the
> change, but the quit does not seem to take effect---violating my
> expectation. I'll appreciate any help on understanding what's going on.
[snip]
> So far so good. I decided to try it on longer files and I wanted to see
> the change more quickly (without long files scrolling past my terminal),
> so I decided to add a /q/ command right after the c commmand. I
> thought---it will make sed quit right after making the change, so I can
> see it works as desired and then I remove the /q/ and release it to
> production. But that did not happen.
[snip]
> I failed the exercise I gave myself. Can you help me to understand why
> the q command isn't stopping sed as I thought it would? I'd like to get
> a better intuition.

By default sed prints the "pattern space", i.e. all lines in the file.
You can suppress that with the "-n" option to sed. (In other words, use
"sed -n" instead of plain "sed" in your script.)

The man page for GNU sed 4.9 says about the "q" command:

"Immediately quit the sed script without processing any more input,
except that if auto-print is not disabled the current pattern space will
be printed."

So the behaviour you're seeing without the "-n" option is to be
expected: pattern space still gets auto-printed after the "q" command.

Salvador Mirzo wrote:

> I failed the exercise I gave myself. Can you help me to understand why
> the q command isn't stopping sed as I thought it would? I'd like to get
> a better intuition.

The specification in https://pubs.opengroup.org/onlinepubs/9799919799/
says at the end of the c command "Start the next cycle."
So any commands following it won't be executed.
The rationale does not mention the reason for the behavior specified.

Here, replacing the single-line c command with s/.*/$tag/ would do it.
If you needed a range of lines to be changed, that might become tricky.

Ralf Damaschke <rwspam@gmx.de> writes:

> Salvador Mirzo wrote:
>
>> I failed the exercise I gave myself. Can you help me to understand why
>> the q command isn't stopping sed as I thought it would? I'd like to get
>> a better intuition.
>
> The specification in https://pubs.opengroup.org/onlinepubs/9799919799/

That's the home page. I believe you meant to link the sed page
directly. When copying URLs from the specification, we need to copy the
framed URL, otherwise we always end up at the home page. The framed sed
page is at

https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sed.html

> says at the end of the c command "Start the next cycle."
> So any commands following it won't be executed.
> The rationale does not mention the reason for the behavior specified.

Interesting. Thanks for mentioning. I wonder if I understand the
information. I tried to change a line and then append another. In both
FreeBSD's and GNU's sed, I the append takes place (and I thought they
would not if they were to obey the specification).

$ cat x.txt
a line
x more lines
more lines

$ sed '/^x/{c\
hello
a\
hi
}' x.txt
a line
hello
hi
more lines
more lines
$

> Here, replacing the single-line c command with s/.*/$tag/ would do it.
> If you needed a range of lines to be changed, that might become tricky.

I don't need a range of lines to be changed, but I'd like to replace the
string only in the second line of the narrow region of interest. I
tried your idea and it works, so I guess that way I can be POSIX-compliant.

--
Thanks!

Salvador Mirzo wrote:

> Ralf Damaschke <rwspam@gmx.de> writes:
>> The specification in https://pubs.opengroup.org/onlinepubs/9799919799/
>
> That's the home page. I believe you meant to link the sed page
> directly. When copying URLs from the specification, we need to copy the
> framed URL, otherwise we always end up at the home page. The framed sed
> page is at
>
> https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sed.html

Actually I considered that. But I felt that the top page might be of more
value since it is easy to find "Shell & Utilities" and "Utilities" in the
frames at the left, whereas the HOME link at the subframe leads to a very
detailed table of content where I would have to use the browser's find
command to locate the specs of other commands.

> I tried to change a line and then append another. In both
> FreeBSD's and GNU's sed, I the append takes place (and I thought they
> would not if they were to obey the specification).

> $ sed '/^x/{c\
> hello a\
> hi }' x.txt a line hello hi more lines more lines $
> $ sed '/^x/{c\
> hello
> a\
> hi
> }' x.txt

For GNU sed that might be a version issue.
On my system with sed version "sed (GNU sed) 4.9" the copied&pasted
command only prints "hello" for "x" and ignores the append command.

In article <lsjtj8F2r8aU1@mid.individual.net>,
Salvador Mirzo wrote:
....
>> $ sed '/^x/{c\
>> hello a\
>> hi }' x.txt a line hello hi more lines more lines $
>> $ sed '/^x/{c\
>> hello
>> a\
>> hi
>> }' x.txt

Isn't this about the time where we give the caution about "Don't use sed
for anything beyond the s/foo/bar/g stage" ?

Seriously, the above looks like gobbledegook compared to the equivalent in
AWK (or some other normal scripting langugae). "sed" looks like Intercal
(once you get beyond s/foo/bar/g).

I'm sure that whatever it is that OP is trying to do, it could be easily
translated to (say) AWK and look much nicer.

--
The whole aim of practical politics is to keep the populace alarmed (and hence clamorous
to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.

H. L. Mencken

On 18.12.2024 20:46, Salvador Mirzo wrote:
> (*) Summary
>
> I wrote a sed script that makes a line replacement after it finds the
> right spot. So far so good. Then I added quit command after the
> change, but the quit does not seem to take effect---violating my
> expectation. I'll appreciate any help on understanding what's going on.

First (before I forget it) change your string comparison '<' to the
numerical comparison operator '-lt' as in: test $# -lt 2 && usage
Otherwise, if you get used to using the wrong operator, you may get
subtle errors in future if you continue that habit.

Also note that using $* may not work correctly (e.g. depending on
filenames [containing spaces] used). The safe form is a quoted "$@"

(Then I was tempted to make a similar comment as Kenny. But...)

WRT your question I'd be interested to understand more about the
intention of your original question...

I mean if you don't trust your 'sed' command just pipe it though
'less'; there's no need to change the 'sed' program just for that.

Personally I'd try whether it works (by adding "something" before
and also after the desired place in your sample.txt to be sure the
other occurrences were not changed), and then just call

sed -e '/<<Release>>=/,+1s/something/sth else/' sample.txt

to see it working.

Janis

>
> (*) A detailed description
>
> I wrote this program:
>
> --8<-------------------------------------------------------->8---
> %cat make-release
> #!/bin/sh
> usage()
> {
> printf '%s tag file\n' $0
> exit 1
> }
> test $# '<' 2 && usage
> tag="$1"
> shift
> sed "/<<Release>>=/ {
> n;
> c\\
> $tag
> }" $*
> --8<-------------------------------------------------------->8---
>
> Here's how I use it. My objective with it is to replace that
> /something/ in the text file with a new argument.
>
> --8<-------------------------------------------------------->8---
> %cat sample.txt
> Lorem ipsum dolor...
>
> <<Release>>=
> something
> @
>
> ... sit a met [...]
> %
> --8<-------------------------------------------------------->8---
>
> Here's how I invoke it:
>
> --8<-------------------------------------------------------->8---
> %sh make-release release1 sample.txt
> Lorem ipsum dolor...
>
> <<Release>>=
> release1
> @
>
> ... sit a met [...]
> --8<-------------------------------------------------------->8---
>
> So far so good. I decided to try it on longer files and I wanted to see
> the change more quickly (without long files scrolling past my terminal),
> so I decided to add a /q/ command right after the c commmand. I
> thought---it will make sed quit right after making the change, so I can
> see it works as desired and then I remove the /q/ and release it to
> production. But that did not happen.
>
> --8<-------------------------------------------------------->8---
> %cat make-release
> #!/bin/sh
> usage()
> {
> printf '%s tag file\n' $0
> exit 1
> }
> test $# '<' 2 && usage
> tag="$1"
> shift
> sed "/<<Release>>=/ {
> n;
> c\\
> $tag
> q}" $*
> --8<-------------------------------------------------------->8---
>
> I still see the whole file:
>
> --8<-------------------------------------------------------->8---
> %sh make-release release1 sample.txt
> Lorem ipsum dolor...
>
> <<Release>>=
> release1
> @
>
> ... sit a met [...]
> %
> --8<-------------------------------------------------------->8---
>
> I failed the exercise I gave myself. Can you help me to understand why
> the q command isn't stopping sed as I thought it would? I'd like to get
> a better intuition.
>
> I've been reading Dale Dougherty and Arnold Robin's ``sed & awk'' book.
> If you have any recommended sed-related bibliography, I'd appreciate it,
> too.
>

In article <vk40gi$3g9sm$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>On 18.12.2024 20:46, Salvador Mirzo wrote:
>> (*) Summary
>>
>> I wrote a sed script that makes a line replacement after it finds the
>> right spot. So far so good. Then I added quit command after the
>> change, but the quit does not seem to take effect---violating my
>> expectation. I'll appreciate any help on understanding what's going on.
>
>First (before I forget it) change your string comparison '<' to the
>numerical comparison operator '-lt' as in: test $# -lt 2 && usage
>Otherwise, if you get used to using the wrong operator, you may get
>subtle errors in future if you continue that habit.

Agreed, in general, but in practice, the need rarely arises.

The idiomatic way to do this is just:

[ $# = 2 ] || usage()

Also, when I need to do more complex arg verification, I use bash's [[ ]]
mechanism (Yes, I know OP is using /bin/sh, but there is no reason nowadays
not to use bash). Say I want there to be 2 or 3 args (no other # is
acceptable and the 2nd arg must be numeric. Like this:

[[ $#,$2 =~ ^[23],[0-9]+$ ]] || { echo "Arg error!"; exit; }

--
On the subject of racism being depicted in the media, the far right and the far left have
met up in agreement (sort of like how plus infinity meets up with minus infinity).
The far left doesn't want it, because they are afraid it will make people racist.
The far right doesn't want it, because they are afraid it will make people feel bad about being racist.

On 20.12.2024 16:11, Kenny McCormack wrote:
> In article <vk40gi$3g9sm$1@dont-email.me>,
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> On 18.12.2024 20:46, Salvador Mirzo wrote:
>>> [...]
>>
>> First (before I forget it) change your string comparison '<' to the
>> numerical comparison operator '-lt' as in: test $# -lt 2 && usage
>> Otherwise, if you get used to using the wrong operator, you may get
>> subtle errors in future if you continue that habit.
>
> Agreed, in general, but in practice, the need rarely arises.

I certainly disagree on this; if you have 10..19 (or 100..199 etc.)
arguments the '<' test just doesn't trigger but '-lt' does. I mean,
why use a wrong operator. If it will only in specific cases produce
correct results, or if it produced in most cases correct results;
it's just the wrong thing.

>
> The idiomatic way to do this is just:
>
> [ $# = 2 ] || usage()

Yes, but I don't use that but prefer (like you) [[...]], an in, say,

[[ $# == 2 ]] || usage

(since I use Kornshell's [[...]] I also use its "modern" '==').

>
> Also, when I need to do more complex arg verification, I use bash's [[ ]]
> mechanism (Yes, I know OP is using /bin/sh, but there is no reason nowadays
> not to use bash).

If the OP is on Linux the 'sh' might actually be a Bash. If he's,
say, on an AIX the 'sh' may actually be a 'ksh'. - So in any case
there's a good chance to have [[...]] supported. - But it's true
that if he specifies /bin/sh he should not rely on something else
than a POSIX shell (at least).

> Say I want there to be 2 or 3 args (no other # is
> acceptable and the 2nd arg must be numeric. Like this:
>
> [[ $#,$2 =~ ^[23],[0-9]+$ ]] || { echo "Arg error!"; exit; }

An interesting pattern!

Usually I'm spoiled by Kornshell's 'getopts' feature. As I'm using,
say "s:" in an optstring for an option '-s' with a string argument
I'm just using "i#" for an option '-i' with a numeric argument.
(For simple cases with few options I typically don't use 'getopts',
though; I'm lazy.)

Janis

In article <vk43n0$3gtg6$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
....
>> Agreed, in general, but in practice, the need rarely arises.
>
>I certainly disagree on this; if you have 10..19 (or 100..199 etc.)
>arguments the '<' test just doesn't trigger but '-lt' does. I mean,
>why use a wrong operator. If it will only in specific cases produce
>correct results, or if it produced in most cases correct results;
>it's just the wrong thing.

We're not talking about the same thing.

>>
>> The idiomatic way to do this is just:
>>
>> [ $# = 2 ] || usage()
>
>Yes, but I don't use that but prefer (like you) [[...]], an in, say,

[ ] is easier in the simple cases. But, whatever, either way is fine.

>> Also, when I need to do more complex arg verification, I use bash's [[ ]]
>> mechanism (Yes, I know OP is using /bin/sh, but there is no reason nowadays
>> not to use bash).
>
>If the OP is on Linux the 'sh' might actually be a Bash. If he's,

I assume Linux unless/until I hear otherwise. And I tend to also assume
some Debian-based Linux (again, unless/until ...). In Debian-based
Linuxes, sh is "dash", which is pretty much a minimal
subset/POSIX-compliant version of the shell. So, [[ ]] isn't available
there.

--
The randomly chosen signature file that would have appeared here is more than 4
lines long. As such, it violates one or more Usenet RFCs. In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
http://user.xmission.com/~gazelle/Sigs/GodDelusion

Kenny McCormack schrieb:

> Isn't this about the time where we give the caution about "Don't use sed
> for anything beyond the s/foo/bar/g stage" ?

Simply because you cannot imagine how to use it? No!

On 2024-12-21, Ralf Damaschke <rwspam@gmx.de> wrote:
> Kenny McCormack schrieb:
>
>> Isn't this about the time where we give the caution about "Don't use sed
>> for anything beyond the s/foo/bar/g stage" ?
>
> Simply because you cannot imagine how to use it? No!

He explained the "because", in the immediately following text that you snipped:

KMC> Seriously, the above looks like gobbledegook compared to the equivalent in
KMC> AWK (or some other normal scripting langugae). "sed" looks like Intercal
KMC> (once you get beyond s/foo/bar/g).

KMC> I'm sure that whatever it is that OP is trying to do, it could be easily
KMC> translated to (say) AWK and look much nicer.

The reasoning doesn't quite match up with the characterization "can't imagine
how to use it". Kenny need not imagine; he has seen examples of how to
use Sed in ways beyond s/x/y/g, and has the above remarks about that.

Sed is one of the so-called "esolangs" which some people use for puzzling.
For instance, here is a kind of Lisp interpreter written in Sed:

https://github.com/shinh/sedlisp/blob/master/sedlisp.sed

The goal of writing in sed is not to solve the problem, and to communicate with
future users of the program so that they can adapt it to changing needs; the
goal is to puzzle out what it takes to solve it in Sed, and to show: "Hey,
look, I did this in Sed! Isn't it amazing? (And, by extension, aren't I?)"

Just like the goal of solving a Sudoku isn't to get 9 digits in every row,
colum and box. That's the ostensible goal within the puzzle. The actual goal
of the puzzler is to engage in puzzle solving.

The given text processing problem to be solved in Sed serves a similar purpose
to the 9 digit constraint rules in Sudoku: it just creates the pretext for
entertaining puzzle solving, and is not the actual goal.

People looking for solutions in their production workflow do not want
"hey, look, this can be done in Sed!" type stuff. While people /can/
ramp up on it and easily become proficient, the only ones ever to do so are
going to be engineers looking to waste time on puzzles.

Engineers not looking for puzzing won't ramp up on stuff like this because
it provides no value outside of puzzing. For production work, you need
a language which not only orchestrates the needed computation on the machine,
but also captures the requirements in a way that communicates to people.
Programming languages are a form of documentation!

Nobody wants to write cryptic gobbledygook just for shits and giggles, only to
then have to write an accompanying ream of boring to try to bring it up to the
same communication standard that comes from just writing nothing but code in an
expressive language.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

In article <20241220184059.820@kylheku.com>,
Kaz Kylheku <643-408-1753@kylheku.com> wrote:
....
>
>Sed is one of the so-called "esolangs" which some people use for puzzling.
>For instance, here is a kind of Lisp interpreter written in Sed:
>
>https://github.com/shinh/sedlisp/blob/master/sedlisp.sed
>
>The goal of writing in sed is not to solve the problem, and to communicate with
>future users of the program so that they can adapt it to changing needs; the
>goal is to puzzle out what it takes to solve it in Sed, and to show: "Hey,
>look, I did this in Sed! Isn't it amazing? (And, by extension, aren't I?)"

Exactly. Well said and well put.

(rest clipped, but should be required reading for everyone)

--
He continues to assert that 2 plus 2 equals 4, despite being repeatedly
told otherwise.

On 2024-12-21, Kenny McCormack <gazelle@shell.xmission.com> wrote:
> In article <20241220184059.820@kylheku.com>,
> Kaz Kylheku <643-408-1753@kylheku.com> wrote:
> ...
>>
>>Sed is one of the so-called "esolangs" which some people use for puzzling.
>>For instance, here is a kind of Lisp interpreter written in Sed:
>>
>>https://github.com/shinh/sedlisp/blob/master/sedlisp.sed
>>
>>The goal of writing in sed is not to solve the problem, and to communicate with
>>future users of the program so that they can adapt it to changing needs; the
>>goal is to puzzle out what it takes to solve it in Sed, and to show: "Hey,
>>look, I did this in Sed! Isn't it amazing? (And, by extension, aren't I?)"
>
> Exactly. Well said and well put.
>
> (rest clipped, but should be required reading for everyone)

Sometimes that puzzled out stuff people do in their spare time is really cool!

This just appeared on HackerNews:

https://github.com/izabera/pseudo3d

A raycast first-person maze navigator, written in Bash.

(The code is pretty small and not particularly cryptic.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

> On 18.12.2024 20:46, Salvador Mirzo wrote:
>> (*) Summary
>>
>> I wrote a sed script that makes a line replacement after it finds the
>> right spot. So far so good. Then I added quit command after the
>> change, but the quit does not seem to take effect---violating my
>> expectation. I'll appreciate any help on understanding what's going on.
>
> First (before I forget it) change your string comparison '<' to the
> numerical comparison operator '-lt' as in: test $# -lt 2 && usage
> Otherwise, if you get used to using the wrong operator, you may get
> subtle errors in future if you continue that habit.

Changed. Why is it the wrong operator? It seems it's not the standard
one---checking just now on the POSIX.1 spec. I think I just tried it
out and given it worked as expected, I didn't think of checking it. I'm
new to the whole thing.

> Also note that using $* may not work correctly (e.g. depending on
> filenames [containing spaces] used). The safe form is a quoted "$@"

I learned something here.

--8<-------------------------------------------------------->8---
$ cat script.sh
#!/bin/sh
echo dollar-star $*
../script dollar-star $*

echo quoted-dollar-star "$*"
../script "$*"

echo dollar-at $@
../script $@

echo quoted-dollar-at "$@"
../script "$@"

$ cat script.c
#include <stdio.h>

int main(int argc, char *argv[]) {
printf("\nfull cmdline: ");
for (int i = 0; i < argc; ++i) {
printf("%s ", argv[i]);
}
printf("\nargs: %d\n", argc);
for (int i = 0; i < argc; ++i) {
printf("arg %d: %s\n", i, argv[i]);
}
return 0;
} --8<-------------------------------------------------------->8---

$ ./script.sh 1 2 "th ree"
dollar-star 1 2 th ree

full cmdline: ./script dollar-star 1 2 th ree
args: 6
arg 0: ./script
arg 1: dollar-star
arg 2: 1
arg 3: 2
arg 4: th
arg 5: ree
quoted-dollar-star 1 2 th ree

full cmdline: ./script 1 2 th ree
args: 2
arg 0: ./script
arg 1: 1 2 th ree
dollar-at 1 2 th ree

full cmdline: ./script 1 2 th ree
args: 5
arg 0: ./script
arg 1: 1
arg 2: 2
arg 3: th
arg 4: ree
quoted-dollar-at 1 2 th ree

full cmdline: ./script 1 2 th ree
args: 4
arg 0: ./script
arg 1: 1
arg 2: 2
arg 3: th ree
$

> (Then I was tempted to make a similar comment as Kenny. But...)

I'm studying and I often go back to the past to see what life was I
like. I initially tried to solve the problem with /ed/, but did not
find a way to insert a string coming from the a shell script's cmdline.
Then I thought that /sed/ was there to make /ed/ more scriptable.

> WRT your question I'd be interested to understand more about the
> intention of your original question...

The intention is mostly in the paragraph above, but the way I study is
to put things in real-world practice as much as possible. (When I
realize the solution is indeed too old to make sense, I replace it.
Otherwise I stick with it.) I have a literate programming file that
contains a chunk that's the version of the program I'm writing. So when
you ask the program its version, the information is included in the
executable, an idea which I like. I get the version with a git command
such as

$ git log --oneline | head -1 | awk '{print $1}'
2566d31

So I wanted to include such string in the literate programming file. At
first I wrote a solution in the programming language I'm using, but I
remember seeing many older software using sed for something like that,
so I decided to study sed a bit. I read the sed part of Dale Dougherty
and Arnold Robbins's book "sed & awk", second edition, and I thought sed
was quite neat and sensible. What I'm noticing now is that there are
too many different sed behavior out there to make it sensible to use.
UNIX systems as a whole are like that, so I'm used to reminding myself
of using the common subset of everything. Perhaps the common subset of
sed is too small. If I have to use it just for search and replace, then
perhaps it's not really worth it.

> I mean if you don't trust your 'sed' command just pipe it though
> 'less'; there's no need to change the 'sed' program just for that.
>
> Personally I'd try whether it works (by adding "something" before
> and also after the desired place in your sample.txt to be sure the
> other occurrences were not changed), and then just call
>
> sed -e '/<<Release>>=/,+1s/something/sth else/' sample.txt
>
> to see it working.

Thanks for the excellent instruction!

On 12/18/2024 1:46 PM, Salvador Mirzo wrote:
> (*) Summary
>
> I wrote a sed script that makes a line replacement after it finds the
> right spot. So far so good. Then I added quit command after the
> change, but the quit does not seem to take effect---violating my
> expectation. I'll appreciate any help on understanding what's going on.
>
> (*) A detailed description
>
> I wrote this program:
>
> --8<-------------------------------------------------------->8---
> %cat make-release
> #!/bin/sh
> usage()
> {
> printf '%s tag file\n' $0
> exit 1
> }
> test $# '<' 2 && usage
> tag="$1"
> shift
> sed "/<<Release>>=/ {
> n;
> c\\
> $tag
> }" $*
> --8<-------------------------------------------------------->8---
>
> Here's how I use it. My objective with it is to replace that
> /something/ in the text file with a new argument.
>
> --8<-------------------------------------------------------->8---
> %cat sample.txt
> Lorem ipsum dolor...
>
> <<Release>>=
> something
> @
>
> ... sit a met [...]
> %
> --8<-------------------------------------------------------->8---
<snip>

I'm not going to get into what might be wrong in your sed script
because, while sed is great for doing s/old/new/ on individual lines,
for anything else you should just use awk for some combination of
robustness, clarity, portability, maintainability, and all of the other
desirable attributes of good software.

For example, if there's always just 1 line of text under `<<Release>>=`
then using any POSIX awk you could do:

tag="$tag" awk '
!f { print }
{ f = 0 }
$0 == "<<Release>>=" {
print ENVIRON["tag"]
f = 1
}
' "${@:--}"

or if it can be multiple lines ending with `@`:

tag="$tag" awk '
$0 == "@" { f = 0 }
!f { print }
$0 == "<<Release>>=" {
print ENVIRON["tag"]
f = 1
}
' "${@:--}"

If you want to temporarily exit after printing the replaced value just
add `exit` after `f = 1`.

Note that `tag="$tag" awk` have to be on a single line exactly as shown.
See
https://stackoverflow.com/questions/19075671/how-do-i-use-shell-variables-in-an-awk-script
for more info on using shell variables values in an awk script and
https://stackoverflow.com/questions/29613304/is-it-possible-to-escape-regex-metacharacters-reliably-with-sed
for more info on using shell variables values in a sed script.

Ed.

On 21.12.2024 04:09, Kaz Kylheku wrote:
> On 2024-12-21, Ralf Damaschke <rwspam@gmx.de> wrote:
>> [...]
> [...]
>
> People looking for solutions in their production workflow do not want
> "hey, look, this can be done in Sed!" type stuff. While people /can/
> ramp up on it and easily become proficient, the only ones ever to do so are
> going to be engineers looking to waste time on puzzles.

I like your comparison with puzzles, specifically this essence.

Janis

> [...]

On 21.12.2024 04:57, Kaz Kylheku wrote:
>
> Sometimes that puzzled out stuff people do in their spare time is really cool!
>
> This just appeared on HackerNews:
>
> https://github.com/izabera/pseudo3d
>
> A raycast first-person maze navigator, written in Bash.

Wasn't there a similar code in Awk mentioned some time ago?

> (The code is pretty small and not particularly cryptic.)

Janis

On 21.12.2024 13:17, Salvador Mirzo wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>> On 18.12.2024 20:46, Salvador Mirzo wrote:
>>> (*) Summary
>>>
>>> I wrote a sed script that makes a line replacement after it finds the
>>> right spot. So far so good. Then I added quit command after the
>>> change, but the quit does not seem to take effect---violating my
>>> expectation. I'll appreciate any help on understanding what's going on.
>>
>> First (before I forget it) change your string comparison '<' to the
>> numerical comparison operator '-lt' as in: test $# -lt 2 && usage
>> Otherwise, if you get used to using the wrong operator, you may get
>> subtle errors in future if you continue that habit.
>
> Changed. Why is it the wrong operator? It seems it's not the standard
> one---checking just now on the POSIX.1 spec. I think I just tried it
> out and given it worked as expected, I didn't think of checking it. I'm
> new to the whole thing.

If you're new be very cautious with Shell; it looks easy to handle
but has a lot subtleties!

You have two sets of comparison operators in Unix shell; strangely
the ones we'd expect to compare numerically (=, !=, <, <=, >, >=)
instead do string comparisons (order of characters). And for the
numeric comparison there's other (-eq, -ne, -lt, -le, -gt, -ge)
operators.

The difference can be observed when comparing numbers in strings
like, say, 11 and 2.
[ 11 -lt 2 ] # returns false (numeric 11 is greater than 2)
[ 11 '<' 2 ] # returns true (string 11 comes before string 2)
The first is a comparison [of strings] interpreted as a numeric
entity, the second is a comparison [of strings] as a string with
lexicographic ordering.

Note also that in modern shells there's also $((...)) and ((...))
(arithmetic expansion and the [non-standard] arithmetic command)
available which support a syntax that is less surprising; e.g.

$ echo $(( 11 < 2 ))
$ (( 11 < 2 )) && ...

(without quoted operator and with numerical semantics).

>
>> Also note that using $* may not work correctly (e.g. depending on
>> filenames [containing spaces] used). The safe form is a quoted "$@"
>
> I learned something here.

The point with spaces in filenames is also something elementary
to know; that's why the "$@" needs quoting (and quoted variables
is the rule of thumb for variable expansions [for most cases]).

> [snip code]
>
>> (Then I was tempted to make a similar comment as Kenny. But...)
>
> I'm studying and I often go back to the past to see what life was I
> like. I initially tried to solve the problem with /ed/, but did not
> find a way to insert a string coming from the a shell script's cmdline.
> Then I thought that /sed/ was there to make /ed/ more scriptable.

I very very rarely use 'ed' but I recall to have used it feeding
input from stdin to it, so it is in principle possible to also
feed in expanded shell variables as part of the 'ed' input.

>
>> WRT your question I'd be interested to understand more about the
>> intention of your original question...
>
> The intention is mostly in the paragraph above, but the way I study is
> to put things in real-world practice as much as possible. (When I
> realize the solution is indeed too old to make sense, I replace it.
> Otherwise I stick with it.) I have a literate programming file that
> contains a chunk that's the version of the program I'm writing. So when
> you ask the program its version, the information is included in the
> executable, an idea which I like. I get the version with a git command
> such as

As previously mentioned, 'sed' might not be the best choice for
developing such scripts; you might want to consider to learn 'awk'.

> $ git log --oneline | head -1 | awk '{print $1}'
> 2566d31

With Awk you don't need 'head', it can be done like this

$ git log --oneline | awk 'NR==1 {print $1}'

(For long input files you may want an early exit
...| awk 'NR==1 { print $1 ; exit(0) }'
but that just as an aside.)

>
> So I wanted to include such string in the literate programming file. At
> first I wrote a solution in the programming language I'm using, but I
> remember seeing many older software using sed for something like that,
> so I decided to study sed a bit. I read the sed part of Dale Dougherty
> and Arnold Robbins's book "sed & awk", second edition, and I thought sed
> was quite neat and sensible. What I'm noticing now is that there are
> too many different sed behavior out there to make it sensible to use.
> UNIX systems as a whole are like that, so I'm used to reminding myself
> of using the common subset of everything. Perhaps the common subset of
> sed is too small. If I have to use it just for search and replace, then
> perhaps it's not really worth it.

Awk has a powerful large common subset and its base is standardized.
(GNU Awk as a prevalent variant has a couple interesting extensions.)
Since your book covers Awk you should have a look into it. (Arnold
has also another book solely on Awk published, the GNU Awk Manual,
which is also available online.)

>
>> I mean if you don't trust your 'sed' command just pipe it though
>> 'less'; there's no need to change the 'sed' program just for that.
>>
>> Personally I'd try whether it works (by adding "something" before
>> and also after the desired place in your sample.txt to be sure the
>> other occurrences were not changed), and then just call
>>
>> sed -e '/<<Release>>=/,+1s/something/sth else/' sample.txt
>>
>> to see it working.
>
> Thanks for the excellent instruction!

You're welcome.

Janis

On 21/12/2024 12:17, Salvador Mirzo wrote:
> I'm studying and I often go back to the past to see what life was I
> like. I initially tried to solve the problem with /ed/, but did not
> find a way to insert a string coming from the a shell script's cmdline.
> Then I thought that /sed/ was there to make /ed/ more scriptable.

I think the other contributors are somewhat harsh on Sed. For
those who started on V6 Unix, there was just Ed, and, as you thought,
Sed was added in V7 as a scripting improvement to Ed. Awk also came
in with V7. Some people adopted Awk with enthusiasm, but the early
versions were quite limited/buggy, partly thanks to the limitations of
the PDP-11; Sed was pretty reliable even in those days. So at least
some users tried and failed with Awk, but found Sed usable with very
little to learn, thanks to the relationship with Ed. The arcana of
Sed are much easier to understand if you are/were a regular user of Ed,
whereas Awk requires you to learn a whole new language [I'm not in any
way suggesting that that is unusually difficult].

So students will normally no longer learn or use Ed/Sed, apart
perhaps from "s/foo/bar/" and similar; it makes more sense to learn a
visual editor and Awk. But for older hands, and for those interested
in the history, there is still a use for Sed. Personally, I know my
way around Sed much better than Awk. So, again personally, Sed is my
stream editor of choice for tasks somewhat harder than "s ...", but not
so hard that I need to do some serious programming and checking of the
documentation. [YMMV, and I'm certainly not trying to persuade any Awk
users that they should learn to use Sed instead.]

--
Andy Walker, Nottingham.
Andy's music pages: www.cuboid.me.uk/andy/Music
Composer of the day: www.cuboid.me.uk/andy/Music/Composers/Paderewski

On 21.12.2024 16:34, Andy Walker wrote:
> On 21/12/2024 12:17, Salvador Mirzo wrote:
>> I'm studying and I often go back to the past to see what life was I
>> like. I initially tried to solve the problem with /ed/, but did not
>> find a way to insert a string coming from the a shell script's cmdline.
>> Then I thought that /sed/ was there to make /ed/ more scriptable.
>
> I think the other contributors are somewhat harsh on Sed. For
> those who started on V6 Unix, there was just Ed, and, as you thought,
> Sed was added in V7 as a scripting improvement to Ed.

If all you have is Sed, Sed may look fine.

If you had invested a lot time to learn or master Sed you're of course
less objecting to Sed.

> Awk also came
> in with V7. Some people adopted Awk with enthusiasm, but the early
> versions were quite limited/buggy, partly thanks to the limitations of
> the PDP-11; Sed was pretty reliable even in those days.

You should mention that you are speaking of the 1977 version of AT&T
Awk (released in 1979); that [buggy] version appeared 45 years ago!

The Awk standard (1992) is based on the 1985 version (released 1987);
yet also already 37 years ago, the Awk POSIX standard 32 years ago.

The GNU Awk version was introduced 1986; it is continuously supported
(and is meanwhile available in version 5.x).

> So at least some users tried and failed with Awk,

(There was obviously plenty of time to not need to stick to Sed.)

> but found Sed usable with very
> little to learn, thanks to the relationship with Ed. The arcana of
> Sed are much easier to understand if you are/were a regular user of Ed,
> whereas Awk requires you to learn a whole new language [I'm not in any
> way suggesting that that is unusually difficult].

Spreading FUD (bugs in the first release, whole new language) is not
appropriate here! - Awk is stable now since decades and the language
is extremely terse and with an excellent power/complexity ratio!)

A few more things should be pointed out in that respect...
At Washington University St. Louis there was an examination made how
long it requires to learn Awk; the result was that in one laboratory
50% of the students considered themselves being confident with Awk,
after a week it were 90% of the students.
Awk's syntax is extremely simple. The imperative actions resemble the
"C" language, so they're easy for the many users knowing any language
that was [syntactically] derived from "C" (but far more simple).
The few syntax elements are also readable - no cryptic abbreviations
as in Sed - so they are easily memorable.

I'm sure if one is constantly working with Sed that at some point he
will also have the [cryptic] Sed language incorporated.

>
> So students will normally no longer learn or use Ed/Sed, apart
> perhaps from "s/foo/bar/" and similar; it makes more sense to learn a
> visual editor and Awk. But for older hands, and for those interested
> in the history, there is still a use for Sed. Personally, I know my
> way around Sed much better than Awk. So, again personally, Sed is my
> stream editor of choice for tasks somewhat harder than "s ...", but not
> so hard that I need to do some serious programming and checking of the
> documentation. [YMMV, and I'm certainly not trying to persuade any Awk
> users that they should learn to use Sed instead.]

Fair enough.

For newbies, as the OP seems to be, the personal historic experience
with Sed is not that useful, though. Since he now has the choice he
should take the most appropriate tool.

I also don't think that any of the negative responses concerning Sed
had meant to express any disrespect of Sed users.

Janis

On 21.12.2024 15:38, Janis Papanagnou wrote:
> On 21.12.2024 04:57, Kaz Kylheku wrote:
>>
>> Sometimes that puzzled out stuff people do in their spare time is really cool!
>>
>> This just appeared on HackerNews:
>>
>> https://github.com/izabera/pseudo3d
>>
>> A raycast first-person maze navigator, written in Bash.
>
> Wasn't there a similar code in Awk mentioned some time ago?

I found it (fps.awk) on my file system; ~650 LOC - too long to post?
(I see it uses some GNU Awk features.)
To support searches for it, it says: #Copyright (c) 2016 Fedor Kalugin

Janis

Andy Walker <anw@cuboid.co.uk> writes:

> On 21/12/2024 12:17, Salvador Mirzo wrote:
>> I'm studying and I often go back to the past to see what life was I
>> like. I initially tried to solve the problem with /ed/, but did not
>> find a way to insert a string coming from the a shell script's cmdline.
>> Then I thought that /sed/ was there to make /ed/ more scriptable.
>
> I think the other contributors are somewhat harsh on Sed. For
> those who started on V6 Unix, there was just Ed, and, as you thought,
> Sed was added in V7 as a scripting improvement to Ed. Awk also came
> in with V7. Some people adopted Awk with enthusiasm, but the early
> versions were quite limited/buggy, partly thanks to the limitations of
> the PDP-11; Sed was pretty reliable even in those days. So at least
> some users tried and failed with Awk, but found Sed usable with very
> little to learn, thanks to the relationship with Ed. The arcana of
> Sed are much easier to understand if you are/were a regular user of Ed,
> whereas Awk requires you to learn a whole new language [I'm not in any
> way suggesting that that is unusually difficult].

It doesn't seem difficult to learn, but there could be so many different
implementations now that it could be a challenge to write a script and
distribute it around.

> So students will normally no longer learn or use Ed/Sed, apart
> perhaps from "s/foo/bar/" and similar; it makes more sense to learn a
> visual editor and Awk. But for older hands, and for those interested
> in the history, there is still a use for Sed.

History is my main thing here. I like to know how life was like. I
recently read ``Hackers'' by Steven Levy, 1984. (I was sad it ended.)
I have an open mind about software. I'm here on the USENET because I
discovered it in non-standard media. It turns out I find NNTP a much
better medium of discourse than any other. (I'm writing this from a GNU
EMACS buffer and will be send out using Gnus. Gnus can be dramatically
uninuitive, but there seems to be no real replacement for it when it
comes to USENET and perhaps mail.) It is noticeable that newer
generations are not really better at writing software compared to
previous generations. I wouldn't change make, for example, for newer
ones. Have you seen Gradle? What was that for? There's a website
whose slogan is ``newer is not always better''.

> Personally, I know my way around Sed much better than Awk. So, again
> personally, Sed is my stream editor of choice for tasks somewhat
> harder than "s ...", but not so hard that I need to do some serious
> programming and checking of the documentation. [YMMV, and I'm
> certainly not trying to persuade any Awk users that they should learn
> to use Sed instead.]

I actually know awk a little bit. I've read the ``AWK Programming
Language'' by Aho, Kernighan and Weinberger, 1988. I loved it. It
replaced pretty everything I used when shell scripting something. But
it doesn't feel right to ignore ed and sed. I need to know at least how
to use them for the essential cases. These tools incorporate a certain
way of thinking that's very valuable and knowing where they come from is
very valuable.

Salvador Mirzo <smirzo@example.com>:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>
>> On 18.12.2024 20:46, Salvador Mirzo wrote:
>>
>> First (before I forget it) change your string comparison '<' to the
>> numerical comparison operator '-lt' as in: test $# -lt 2 && usage
>> Otherwise, if you get used to using the wrong operator, you may get
>> subtle errors in future if you continue that habit.
>
> Changed. Why is it the wrong operator?

'<' compares two strings lexically, whereas '-lt' compares two
numbers numerically.

Some examples:

1 < 2 ==> true

2 < 2 ==> false

3 < 2 ==> false

10 < 2 ==> true, probably not your intention.

1 -lt 2 ==> true

2 -lt 2 ==> false

3 -lt 2 ==> false

10 -lt 2 ==> false

> It seems it's not the standard one---checking just now on the
> POSIX.1 spec.

The recent POSIX standard knows of the '<' operator, see
<https://pubs.opengroup.org/onlinepubs/9799919799/utilities/test.html#tag_20_121_05>.

>> Also note that using $* may not work correctly (e.g. depending on
>> filenames [containing spaces] used). The safe form is a quoted "$@"
>
> I learned something here.
>
> --8<-------------------------------------------------------->8---
> $ cat script.sh
> #!/bin/sh
> echo dollar-star $*
> ./script dollar-star $*
>
> echo quoted-dollar-star "$*"
> ./script "$*"
>
> echo dollar-at $@
> ./script $@
>
> echo quoted-dollar-at "$@"
> ./script "$@"
>
> $ cat script.c
> #include <stdio.h>
>
> int main(int argc, char *argv[]) {
> printf("\nfull cmdline: ");
> for (int i = 0; i < argc; ++i) {
> printf("%s ", argv[i]);
> }
> printf("\nargs: %d\n", argc);
> for (int i = 0; i < argc; ++i) {
> printf("arg %d: %s\n", i, argv[i]);
> }
> return 0;
> }
> --8<-------------------------------------------------------->8---
>
> $ ./script.sh 1 2 "th ree"
> dollar-star 1 2 th ree
>
> full cmdline: ./script dollar-star 1 2 th ree
> args: 6
> arg 0: ./script
> arg 1: dollar-star
> arg 2: 1
> arg 3: 2
> arg 4: th
> arg 5: ree
> quoted-dollar-star 1 2 th ree
>
> full cmdline: ./script 1 2 th ree
> args: 2
> arg 0: ./script
> arg 1: 1 2 th ree
> dollar-at 1 2 th ree
>
> full cmdline: ./script 1 2 th ree
> args: 5
> arg 0: ./script
> arg 1: 1
> arg 2: 2
> arg 3: th
> arg 4: ree
> quoted-dollar-at 1 2 th ree
>
> full cmdline: ./script 1 2 th ree
> args: 4
> arg 0: ./script
> arg 1: 1
> arg 2: 2
> arg 3: th ree
> $

So using "$@" is the only way to have the shell pass its
positional parameters unmodified (i. e. neither glued together
nor split at internal white space (more precise: split according
to the contents of the IFS shell parameter)) to commands it
invokes.

On 21.12.2024 19:21, Salvador Mirzo wrote:
>
> [...] I'm here on the USENET because I
> discovered it in non-standard media. It turns out I find NNTP a much
> better medium of discourse than any other. (I'm writing this from a GNU
> EMACS buffer and will be send out using Gnus. Gnus can be dramatically
> uninuitive, but there seems to be no real replacement for it when it
> comes to USENET and perhaps mail.) [...]

There's tools like Thunderbird where you can work with Usenet similarly
to using a typical GUI-based email client (it is actually also an email
client). If you like historic tools I've heard that there's still 'nn'
around, a text-oriented newsreader that I used in the 1990's (loved it).

>
> I actually know awk a little bit. I've read the ``AWK Programming
> Language'' by Aho, Kernighan and Weinberger, 1988. I loved it. [...]

This is an excellent source! So you might want to look into the GNU Awk
Manual just for a contemporary Awk variant, to see a couple more useful
features supported in an efficient Awk implementation.

Janis