On 20.11.2024 12:30, Muttley@DastartdlyHQ.org wrote:
> On Wed, 20 Nov 2024 11:51:11 +0100
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> boring babbled:
>> On 20.11.2024 09:21, Muttley@DastartdlyHQ.org wrote:
>>> On Tue, 19 Nov 2024 18:43:48 -0800
>>> merlyn@stonehenge.com (Randal L. Schwartz) boring babbled:
>>>>
>>>> I'm often reminded of this as I've been coding very little in Perl these
>>>> days, and a lot more in languages like Dart, where the regex feels like
>>>> a clumsy bolt-on rather than a proper first-class citizen.
>>>
>>> Regex itself is clumsy beyond simple search and replace patterns. A lot of
>>> stuff I've seen done in regex would have better done procedurally at the
>>> expense of slightly more code but a LOT more readability. Also given its
>>> effectively a compact language with its own grammar and syntax IMO it should
>>> not be the core part of any language as it can lead to a syntatic mess,
>> which
>>> is what often happens with Perl.
>>
>> I wouldn't look at it that way. I've seen Regexps as part of languages
>> usually in well defined syntactical contexts. For example, like strings
>> are enclosed in "...", Regexps could be seen within /.../ delimiters.
>> GNU Awk (in recent versions) went towards first class "strongly typed"
>> Regexps which are then denoted by the @/.../ syntax.
>>
>> I'm curious what you mean by Regexps presented in a "procedural" form.
>> Can you give some examples?
>
> Anything that can be done in regex can obviously also be done procedurally.
> At the point regex expression become unwieldy - usually when substitution
> variables raise their heads - I prefer procedural code as its also often
> easier to debug.

You haven't even tried to honestly answer my (serious) question.
With your statement above and your hostility below, it rather seems
you have no clue of what I am talking about.

>
>> In practice, given that a Regexp conforms to a FSA, any Regexp can be
>> precompiled and used multiple times. The thing I had used in Java - it
>
> Precompiled regex is no more efficient than precompiled anything , its all
> just assembler at the bottom.

The Regexps are a way to specify the words of a regular language;
for pattern matching the expression gets interpreted or compiled; you
specify it, e.g., using strings of characters and meta-characters.
If you have a programming language where that string gets repeatedly
interpreted then it's slower than a precompiled Regexp expression.

I give you examples...

(1) DES encryption function

(1a) ciphertext = des_encode (key, plaintext)

(1b) cipher = des (key)
ciphertext = cipher.encode (plaintext)

In case (1) you can either call the des encription (decription) for
any (key, plaintext)-pair in a procedural function as in (1a), or
you can create the key-specific encryption once and encode various
texts with the same cipher object as in (1b).

(2) regexp matching

(2a) location = regexp (pattern, string)

(2b) fsm = rexexp (pattern)
location = fsm.match (string)

In case (2) you can either do the match in a string with a pattern
in a procedural form as in (2a) or you can create the FSM for the
given Regexp just once and apply it on various strings as in (2b).

That's what I was talking about.

Only if key (in (1)) or pattern (in (2)) are static or "constant"
that compilation could (but only theoretically) be done in advance
and optimizing system may (or may not) precompile it (both) to
[similar] assembler code. How should that work with regexps or DES?
The optimizing system would need knowledge how to use the library
code (DES, Regexps, ...) to create binary structures based on the
algorithms (key-initialization in DES, FSM-generation in Regexps).
This is [statically] not done.

Otherwise - i.e. the normal, expected case - there's an efficiency
difference to observe between the respective cases of (a) and (b).

>
>> then operate on that same object. (Since there's still typical Regexp
>> syntax involved I suppose that is not what you meant by "procedural"?)
>
> If you don't know the different between declarative syntax like regex and
> procedural syntax then there's not much point continuing this discussion.

Why do you think so, and why are you saying that? - That wasn't and
still isn't the point. - You said upthread

"A lot of stuff I've seen done in regex would have better done
procedurally at the expense of slightly more code but a LOT more
readability."

and I asked

"I'm curious what you mean by Regexps presented in a "procedural"
form.
Can you give some examples?"

What you wanted to say wasn't clear to me, since you were complaining
about the _Regexp syntax_. So it couldn't be meant to just write
regexp (pattern, string) instead of pattern ~ string
but to somehow(!) transform "pattern", say, like /[0-9]+(ABC)?x*foo/,
to something syntactically "better".
I was interested in that "somehow" (that I emphasized), and in an
example how that would look like in your opinion.
If you're unable to answer that simple question then just take that
simple regexp /[0-9]+(ABC)?x*foo/ example and show us your preferred
procedural variant.

But my expectation is that you cannot provide any reasonable example
anyway.

Personally I think that writing bulky procedural stuff for something
like [0-9]+ can only be much worse, and that further abbreviations
like \d+ are the better direction to go if targeting a good interface.
YMMV.

Janis

On 20.11.2024 12:46, Ed Morton wrote:
>
> Definitely. The most relevant statement about regexps is this:
>
>> Some people, when confronted with a problem, think "I know, I'll use
>> regular expressions." Now they have two problems.

(Worth a scribbling on a WC wall.)

>
> Obviously regexps are very useful and commonplace but if you find you
> have to use some online site or other tools to help you write/understand
> one or just generally need more than a couple of minutes to
> write/understand it then it's time to back off and figure out a better
> way to write your code for the sake of whoever has to read it 6 months
> later (and usually for robustness too as it's hard to be sure all rainy
> day cases are handled correctly in a lengthy and/or complicated regexp).

Regexps are nothing for newbies.

The inherent fine thing with Regexps is that you can incrementally
compose them[*].[**]

It seems you haven't found a sensible way to work with them?
(And I'm really astonished about that since I know you worked with
Regexps for years if not decades.)

In those cases where Regexps *are* the tool for a specific task -
I don't expect you to use them where they are inappropriate?! -
what would be the better solution[***] then?

Janis

[*] Like the corresponding FSMs.

[**] And you can also decompose them if they are merged in a huge
expression, too large for you to grasp it. (BTW, I'm doing such
decompositions also with other expressions in program code that
are too bulky.)

[***] Can you answer the question that another poster failed to do?

On Wed, 20 Nov 2024 16:38:24 +0100
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> boring babbled:
>On 20.11.2024 12:30, Muttley@DastartdlyHQ.org wrote:
>> Anything that can be done in regex can obviously also be done procedurally.
>> At the point regex expression become unwieldy - usually when substitution
>> variables raise their heads - I prefer procedural code as its also often
>> easier to debug.
>
>You haven't even tried to honestly answer my (serious) question.

You mean you can't figure out how to do something like string search and replace
procedurally? I'm not going to show you, ask a kid who knows Python or Basic.

>With your statement above and your hostility below, it rather seems

If you think my reply was hostile then I suggest you go find a safe space
and cuddle your teddy bear snowflake.

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

[...]

> Personally I think that writing bulky procedural stuff for something
> like [0-9]+ can only be much worse, and that further abbreviations
> like \d+ are the better direction to go if targeting a good interface.
> YMMV.

Assuming that p is a pointer to the current position in a string, e is a
pointer to the end of it (ie, point just past the last byte) and -
that's important - both are pointers to unsigned quantities, the 'bulky'
C equivalent of [0-9]+ is

while (p < e && *p - '0' < 10) ++p;

That's not too bad. And it's really a hell lot faster than a
general-purpose automaton programmed to recognize the same pattern
(which might not matter most of the time, but sometimes, it does).

Muttley@DastartdlyHQ.org writes:

[...]

>>With your statement above and your hostility below, it rather seems
>
> If you think my reply was hostile then I suggest you go find a safe space
> and cuddle your teddy bear snowflake.

There's surely no reason why anyone could ever think you were inclined
to substitute verbal aggression for arguments.

On Wed, 20 Nov 2024 17:54:22 +0000
Rainer Weikusat <rweikusat@talktalk.net> wrote:

> There's surely no reason why anyone could ever think you were inclined
> to substitute verbal aggression for arguments.

I mean, it's his whole thing - why would he stop now?

On Wed, 20 Nov 2024 12:27:54 -0000 (UTC), Muttley wrote:

> Edge cases are regex achilles heal, eg an expression that only accounted
> for 1 -> N chars, not 0 -> N, or matches in the middle but not at the
> ends.

That’s what “^” and “$” are for.

On Wed, 20 Nov 2024 17:54:22 +0000
Rainer Weikusat <rweikusat@talktalk.net> boring babbled:
>Muttley@DastartdlyHQ.org writes:
>
>[...]
>
>>>With your statement above and your hostility below, it rather seems
>>
>> If you think my reply was hostile then I suggest you go find a safe space
>> and cuddle your teddy bear snowflake.
>
>There's surely no reason why anyone could ever think you were inclined
>to substitute verbal aggression for arguments.

I have zero time for anyone who claims hurt feelings or being slighted as
soon as they're losing an argument.

On Wed, 20 Nov 2024 21:43:41 -0000 (UTC)
Lawrence D'Oliveiro <ldo@nz.invalid> boring babbled:
>On Wed, 20 Nov 2024 12:27:54 -0000 (UTC), Muttley wrote:
>
>> Edge cases are regex achilles heal, eg an expression that only accounted
>> for 1 -> N chars, not 0 -> N, or matches in the middle but not at the
>> ends.
>
>That’s what “^” and “$” are for.

Yes, but people forget about those (literal) edge cases.

On Wed, 20 Nov 2024 10:03:47 -0800
John Ames <commodorejohn@gmail.com> boring babbled:
>On Wed, 20 Nov 2024 17:54:22 +0000
>Rainer Weikusat <rweikusat@talktalk.net> wrote:
>
>> There's surely no reason why anyone could ever think you were inclined
>> to substitute verbal aggression for arguments.
>
>I mean, it's his whole thing - why would he stop now?

Whats it like being so wet? Do you get cold easily?

>>>>> "Rainer" == Rainer Weikusat <rweikusat@talktalk.net> writes:

Rainer> ¹ I used to use a JSON parser written in OO-Perl which made
Rainer> extensive use of regexes for that. I've recently replaced that
Rainer> with a C/XS version which - while slightly larger (617 vs 410
Rainer> lines of text) - is over a hundred times faster and conceptually
Rainer> simpler at the same time.

I wonder if that was my famous "JSON parser in a single regex" from
https://www.perlmonks.org/?node_id=995856, or from one of the two CPAN
modules that incorporated it.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Dart/Flutter consulting, Technical writing, Comedy, etc. etc.
Still trying to think of something clever for the fourth line of this .sig

In article <20241120100347.00005f10@gmail.com>,
John Ames <commodorejohn@gmail.com> wrote:
>On Wed, 20 Nov 2024 17:54:22 +0000
>Rainer Weikusat <rweikusat@talktalk.net> wrote:
>
>> There's surely no reason why anyone could ever think you were inclined
>> to substitute verbal aggression for arguments.
>
>I mean, it's his whole thing - why would he stop now?

This is the guy who didn't know what a compiler is, right?

- Dan C.

In article <875xohbxre.fsf@doppelsaurus.mobileactivedefense.com>,
Rainer Weikusat <rweikusat@talktalk.net> wrote:
>Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>
>[...]
>
>> Personally I think that writing bulky procedural stuff for something
>> like [0-9]+ can only be much worse, and that further abbreviations
>> like \d+ are the better direction to go if targeting a good interface.
>> YMMV.
>
>Assuming that p is a pointer to the current position in a string, e is a
>pointer to the end of it (ie, point just past the last byte) and -
>that's important - both are pointers to unsigned quantities, the 'bulky'
>C equivalent of [0-9]+ is
>
>while (p < e && *p - '0' < 10) ++p;
>
>That's not too bad. And it's really a hell lot faster than a
>general-purpose automaton programmed to recognize the same pattern
>(which might not matter most of the time, but sometimes, it does).

It's also not exactly right. `[0-9]+` would match one or more
characters; this possibly matches 0 (ie, if `p` pointed to
something that wasn't a digit).

- Dan C.

cross@spitfire.i.gajendra.net (Dan Cross) writes:
> Rainer Weikusat <rweikusat@talktalk.net> wrote:
>>Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>>
>>[...]
>>
>>> Personally I think that writing bulky procedural stuff for something
>>> like [0-9]+ can only be much worse, and that further abbreviations
>>> like \d+ are the better direction to go if targeting a good interface.
>>> YMMV.
>>
>>Assuming that p is a pointer to the current position in a string, e is a
>>pointer to the end of it (ie, point just past the last byte) and -
>>that's important - both are pointers to unsigned quantities, the 'bulky'
>>C equivalent of [0-9]+ is
>>
>>while (p < e && *p - '0' < 10) ++p;
>>
>>That's not too bad. And it's really a hell lot faster than a
>>general-purpose automaton programmed to recognize the same pattern
>>(which might not matter most of the time, but sometimes, it does).
>
> It's also not exactly right. `[0-9]+` would match one or more
> characters; this possibly matches 0 (ie, if `p` pointed to
> something that wasn't a digit).

The regex won't match any digits if there aren't any. In this case, the
match will fail. I didn't include the code for handling that because it
seemed pretty pointless for the example.

On Thu, 21 Nov 2024 08:18:06 -0000 (UTC)
Muttley@DastartdlyHQ.org wrote:

> Whats it like being so wet? Do you get cold easily?

No, I have a soft gray hoodie with a nice fleecy lining that I quite
like. It's very warm for not being too heavy.

Also: *huh?*

On Thu, 21 Nov 2024 08:13:39 -0000 (UTC)
Muttley@DastartdlyHQ.org wrote:

> I have zero time

I approve, it's a wonderful album!

On Thu, 21 Nov 2024 14:13:37 -0000 (UTC)
cross@spitfire.i.gajendra.net (Dan Cross) boring babbled:
>In article <20241120100347.00005f10@gmail.com>,
>John Ames <commodorejohn@gmail.com> wrote:
>>On Wed, 20 Nov 2024 17:54:22 +0000
>>Rainer Weikusat <rweikusat@talktalk.net> wrote:
>>
>>> There's surely no reason why anyone could ever think you were inclined
>>> to substitute verbal aggression for arguments.
>>
>>I mean, it's his whole thing - why would he stop now?
>
>This is the guy who didn't know what a compiler is, right?

Wrong. Want another go?

merlyn@stonehenge.com (Randal L. Schwartz) writes:
>>>>>> "Rainer" == Rainer Weikusat <rweikusat@talktalk.net> writes:
> Rainer> ¹ I used to use a JSON parser written in OO-Perl which made
> Rainer> extensive use of regexes for that. I've recently replaced that
> Rainer> with a C/XS version which - while slightly larger (617 vs 410
> Rainer> lines of text) - is over a hundred times faster and conceptually
> Rainer> simpler at the same time.
>
> I wonder if that was my famous "JSON parser in a single regex" from
> https://www.perlmonks.org/?node_id=995856, or from one of the two CPAN
> modules that incorporated it.

No. One of my use-cases is an interactive shell running in a web browser
using ActionCable messages to relay data between the browser and the
shell process on the computer supposed to be accessed in this way. For
this, I absolutely do need \u escapes. I also need this to be fast. Eg,
one of the nice properties of JSON is that the type of a value can be
determined by looking at the first character of it. This cries for an
implementation based on an array of pointers to 'value parsing routines'
of size 256 and determining the parser routine to use by using the first
character as index into this table (which will either yield a pointer to
the correct parser routine or NULL for a syntax error).

On 2024-11-20, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> On 20.11.2024 09:21, Muttley@DastartdlyHQ.org wrote:
>> Regex itself is clumsy beyond simple search and replace patterns. A lot of
>> stuff I've seen done in regex would have better done procedurally at the
>> expense of slightly more code but a LOT more readability. Also given its
>> effectively a compact language with its own grammar and syntax IMO it should
>> not be the core part of any language as it can lead to a syntatic mess, which
>> is what often happens with Perl.
>
> I wouldn't look at it that way. I've seen Regexps as part of languages
> usually in well defined syntactical contexts. For example, like strings
> are enclosed in "...", Regexps could be seen within /.../ delimiters.
> GNU Awk (in recent versions) went towards first class "strongly typed"
> Regexps which are then denoted by the @/.../ syntax.

These features solve the problem of regexes being stored as character
strings not being recognized by the language compiler and then having
to be compiled at run-time.

They don't solve all the ergonomics of regexes that Muttley is talking
about.

> I'm curious what you mean by Regexps presented in a "procedural" form.
> Can you give some examples?

Here is an example: using a regex match to capture a C comment /* ... */
in Lex compared to just recognizing the start sequence /* and handling
the discarding of the comment in the action.

Without non-greedy repetition matching, the regex for a C comment is
quite obtuse. The procedural handling is straightforward: read
characters until you see a * immediately followed by a /.

In the wild, you see regexes being used for all sorts of stupid stuff,
like checking whether numeric input is in a certain range, rather than
converting it to a number and doing an arithmetic check.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On Thu, 21 Nov 2024 08:15:41 -0000 (UTC), Muttley wrote:

> On Wed, 20 Nov 2024 21:43:41 -0000 (UTC)
> Lawrence D'Oliveiro <ldo@nz.invalid> boring babbled:
>
>>On Wed, 20 Nov 2024 12:27:54 -0000 (UTC), Muttley wrote:
>>
>>> Edge cases are regex achilles heal, eg an expression that only
>>> accounted for 1 -> N chars, not 0 -> N, or matches in the middle but
>>> not at the ends.
>>
>>That’s what “^” and “$” are for.
>
> Yes, but people forget about those (literal) edge cases.

Those of us who are accustomed to using regexes do not.

Another handy one is “\b” for word boundaries.

On Thu, 21 Nov 2024 19:12:03 -0000 (UTC)
Kaz Kylheku <643-408-1753@kylheku.com> boring babbled:
>On 2024-11-20, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> I'm curious what you mean by Regexps presented in a "procedural" form.
>> Can you give some examples?
>
>Here is an example: using a regex match to capture a C comment /* ... */
>in Lex compared to just recognizing the start sequence /* and handling
>the discarding of the comment in the action.
>
>Without non-greedy repetition matching, the regex for a C comment is
>quite obtuse. The procedural handling is straightforward: read
>characters until you see a * immediately followed by a /.

Its not that simple I'm afraid since comments can be commented out.

eg:

// int i; /*
int j;
/*
int k;
*/
++j;

A C99 and C++ compiler would see "int j" and compile it, a regex would
simply remove everything from the first /* to */.

Also the same probably applies to #ifdef's.

On 20.11.2024 18:50, Rainer Weikusat wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>
>> Personally I think that writing bulky procedural stuff for something
>> like [0-9]+ can only be much worse, and that further abbreviations
>> like \d+ are the better direction to go if targeting a good interface.
>> YMMV.
>
> Assuming that p is a pointer to the current position in a string, e is a
> pointer to the end of it (ie, point just past the last byte) and -
> that's important - both are pointers to unsigned quantities, the 'bulky'
> C equivalent of [0-9]+ is
>
> while (p < e && *p - '0' < 10) ++p;
>
> That's not too bad. And it's really a hell lot faster than a
> general-purpose automaton programmed to recognize the same pattern
> (which might not matter most of the time, but sometimes, it does).

Okay, I see where you're coming from (and especially in that simple
case).

Personally (and YMMV), even here in this simple case I think that
using pointers is not better but worse - and anyway isn't [in this
form] available in most languages; in other cases (and languages)
such constructs get yet more clumsy, and for my not very complex
example - /[0-9]+(ABC)?x*foo/ - even a "catastrophe" concerning
readability, error-proneness, and maintainability.

If that is what the other poster meant I'm fine with your answer;
there's no need to even consider abandoning regular expressions
in favor of explicitly codified parsing.

Janis

PS: And thanks for answering on behalf of the other poster whom I
see in his followups just continuing his very personal style.

On 21.11.2024 20:12, Kaz Kylheku wrote:
> [...]
>
> In the wild, you see regexes being used for all sorts of stupid stuff,

No one can prevent folks using features for stupid things. Yes.

> like checking whether numeric input is in a certain range, rather than
> converting it to a number and doing an arithmetic check.

Janis

On 21.11.2024 23:05, Lawrence D'Oliveiro wrote:
> On Thu, 21 Nov 2024 08:15:41 -0000 (UTC), Muttley wrote:
>> On Wed, 20 Nov 2024 21:43:41 -0000 (UTC)
>> Lawrence D'Oliveiro <ldo@nz.invalid> boring babbled:
>>[...]
>>>
>>> That’s what “^” and “$” are for.
>>
>> Yes, but people forget about those (literal) edge cases.

But *only* _literally_ "edge cases". Rather they're simple
and basics of regexp parsers since their beginning.

> Those of us who are accustomed to using regexes do not.

It's one of the first things that regexp newbies learn,
I'd say.

>
> Another handy one is “\b” for word boundaries.

I prefer \< and \> (that are quite commonly used) for such
structural things, also \( and \) for allowing references
to matched parts. And I prefer the \alpha regexp pattern
extension forms for things like \d \D \w \W \s \S . (But
that's not only a matter of taste but also a question of
what any regexp parser actually supports.)

Janis

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> On 20.11.2024 18:50, Rainer Weikusat wrote:
>> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>>
>>> Personally I think that writing bulky procedural stuff for something
>>> like [0-9]+ can only be much worse, and that further abbreviations
>>> like \d+ are the better direction to go if targeting a good interface.
>>> YMMV.
>>
>> Assuming that p is a pointer to the current position in a string, e is a
>> pointer to the end of it (ie, point just past the last byte) and -
>> that's important - both are pointers to unsigned quantities, the 'bulky'
>> C equivalent of [0-9]+ is
>>
>> while (p < e && *p - '0' < 10) ++p;
>>
>> That's not too bad. And it's really a hell lot faster than a
>> general-purpose automaton programmed to recognize the same pattern
>> (which might not matter most of the time, but sometimes, it does).
>
> Okay, I see where you're coming from (and especially in that simple
> case).
>
> Personally (and YMMV), even here in this simple case I think that
> using pointers is not better but worse - and anyway isn't [in this
> form] available in most languages;

That's a question of using the proper tool for the job. In C, that's
pointer and pointer arithmetic because it's the simplest way to express
something like this.

> in other cases (and languages)
> such constructs get yet more clumsy, and for my not very complex
> example - /[0-9]+(ABC)?x*foo/ - even a "catastrophe" concerning
> readability, error-proneness, and maintainability.

Procedural code for matching strings constructed in this way is
certainly much simpler¹ than the equally procedural code for a
programmable automaton capable of interpreting regexes. Your statement
is basically "If we assume that the code interpreting regexes doesn't
exist, regexes need much less code than something equivalent which does
exist." Without this assumption, the picture becomes a different one
altogether.

¹ This doesn't even need a real state machine, just four subroutines
executed in succession (and two of these can share an implementation as
"matching ABC" and "matching foo" are both cases of matching a constant
string.

> If that is what the other poster meant I'm fine with your answer;
> there's no need to even consider abandoning regular expressions
> in favor of explicitly codified parsing.

This depends on the specific problem and the constraints applicable to a
solution. For the common case, regexes, if easily available, are an
obvious good solution. But not all cases are common.