On Tue, 26 Nov 2024 21:30:12 +0000, Richard Kettlewell wrote:

> Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>
>> On Tue, 26 Nov 2024 08:32:32 +0000, Richard Kettlewell wrote:
>>
>>>> The examples show that freeing the same pointer twice can be detected
>>>> reliably.
>>>
>>> They do no such thing.
>>
>> free(p);
>> printf("%s(): returned from first free() call\n", __func__);
>>
>> free(p);
>> printf("%s(): returned from second free() call\n", __func__);
>>
>> Is that or is that not freeing the same pointer twice?
>
> Obviously you are not arguing in good faith.

Is this some special meaning of “they do no such thing” that is private to
you?

On 11/26/2024 2:22 PM, Lawrence D'Oliveiro wrote:
> On Tue, 26 Nov 2024 08:32:32 +0000, Richard Kettlewell wrote:
>
>>> The examples show that freeing the same pointer twice can be
>>> detected reliably.
>> They do no such thing.
> free(p);
> printf("%s(): returned from first free() call\n", __func__);
>
> free(p);
> printf("%s(): returned from second free() call\n", __func__);
>
> Is that or is that not freeing the same pointer twice?

In the example in the man page, if the two free() calls are back to back
with no intervening malloc() calls, the double free() is detected. I
tried adding a malloc(1000) call in the middle:

    free(p);
    printf("%s(): returned from first free() call\n", __func__);

    malloc(1000);
    printf("%s(): returned from second malloc(1000) call\n", __func__);

    free(p);
    printf("%s(): returned from second free() call\n", __func__);

and the program ran to completion with no apparent problem.

The compiler does what it can and glibc does what it can, but C is C and
the heap is what it is, and no, I don't know how this would work in Rust.

Louis

D <nospam@example.net> wrote:
>
>
> On Tue, 26 Nov 2024, rbowman wrote:
>
>> On Tue, 26 Nov 2024 13:17:08 -0000 (UTC), Rich wrote:
>>
>>> Yep, exactly. If they can be given instructions that match their "lego
>>> brick set" they can snap something together. Ask them to do anything
>>> that requires creativity or research and understanding, and you get back
>>> a turd that has had hours of polishing applied.
>>
>> The holy grail for management is a design methodology that gets adequate
>> results from a workforce of varying aptitudes. Particularly for larger
>> corporations you'll get a normal distribution, a few very good, a few
>> completely useless, and a lot of mediocrity. That's what you have to work
>> with.
>>
>> What I've seen over the years is a company will luck out, get a better
>> than average distribution, and achieve success. Whatever they're doing is
>> taken as an example of the right way and copied mechanically. Top down
>> structured programming, agile, devops, and so forth have their day.
>>
>> TI lucked out in the '70s and used something they called 'matrix
>> management' that became the new Wunderkind. The '80s brought 'In Search of
>> Excellence'.
>>
>> https://en.wikipedia.org/wiki/In_Search_of_Excellence
>>
>> Good money was made from book sales and training sessions from independent
>> snake oil salesmen preaching the gospel. The company I worked for had one
>> of the sessions. Not too many of the 'excellent' corporations are around
>> today.
>>
>> About 10 years ago the company I now work for had a 'pair programming'
>> session. That was hilarious. The 'experts' were only familiar with Apple
>> machines and other than the one they brought there wasn't an Apple in the
>> building. Having been through required attendance things before my team
>> carefully stayed to the back of the room where we could slink away and get
>> back to business.
>>
>> I'm sure the next methodology will wrap itself around AI, spin off
>> training companies, and mostly fail to deliver on the promises.
>>
>>
>
> Ahh... and today you have agile snakeoil salesmen! I heard a story from
> the girlfriend of one of my consultants. Her company, a computer game
> company, hired an "agile coach" who was workshopping away like a madman.
>
> One programmer said... "but this agile thing, it seems to me like it will
> become less efficient and more work, that's bad", the snakeoil salesman
> responded "I hear you and appreciate your concern, but if agile makes
> things worse, you're not doing it right, and that's why we are here"! ;)

“It is difficult to get a man to understand something, when his salary
depends on his not understanding it.” -- Upton Sinclair

https://www.goodreads.com/quotes/21810-it-is-difficult-to-get-a-man-to-understand-something

Agile, at least to the extent I've seen it being actually practiced so
far, is of little benefit to the actual programmers, and adds burden
(more daily meetings for the 'scrum' and sprint periodic meetings for
the finale, retro, planning, etc. meetings).

The one thing it does provide is give management more frequent feedback
that "something's happening" so they can feel cozy that their budget
funds have been used sensibly (i.e., CYA for managers).

But given that most 'enterprise' execution of 'agile' tends to, in the
limit, approach an "agile waterfall" type of arrangement, the
'wonderful promise' of agile of quick turnaround for bug fixes and more
speedy accumulation of incrememtal improvements that deploy every
sprint tends to get lost in the surrounding bureaucracy tarpit that
holds everything else back.

On Tue, 26 Nov 2024 21:58:45 +0100, D wrote:

> Ahh... and today you have agile snakeoil salesmen! I heard a story from
> the girlfriend of one of my consultants. Her company, a computer game
> company, hired an "agile coach" who was workshopping away like a madman.

Right... I wonder if they even believe in the snake oil.We had a police
chief who was a pretty nice guy but had a weakness for traveling medicine
shows that would teach the officers about gang violence and so forth. Even
the low lifes in town knew the 'gangs' were bored 12 year olds with spray
cans that would last 4 seconds in Compton but he lapped it up.

Then the Hells Angels came for vacation and he went off the deep end. The
Angels were better behaved than when the BMW riders came to town but he
imported cops from another state to deal with the potential bloody
violence. The violence turned out to be the out of state cops arresting
college kids who were protesting the over the top treatment of the bikers.

City council went ballistic when they got the bill and it turned out it's
against state law to import cops, something that goes back to the copper
mines and Pinkertons, so the chief was fired.

I'm sure the snake oil salesmen took the show to some other town. The
agile people probably use the same model.

Louis Krupp <lkrupp@invalid.pssw.com.invalid> wrote:
> On 11/26/2024 2:22 PM, Lawrence D'Oliveiro wrote:
>> On Tue, 26 Nov 2024 08:32:32 +0000, Richard Kettlewell wrote:
>>
>>>> The examples show that freeing the same pointer twice can be
>>>> detected reliably.
>>> They do no such thing.
>> free(p);
>> printf("%s(): returned from first free() call\n", __func__);
>>
>> free(p);
>> printf("%s(): returned from second free() call\n", __func__);
>>
>> Is that or is that not freeing the same pointer twice?
>
> In the example in the man page, if the two free() calls are back to
> back with no intervening malloc() calls, the double free() is
> detected. I tried adding a malloc(1000) call in the middle:
>
>     free(p);
>     printf("%s(): returned from first free() call\n", __func__);
>
>     malloc(1000);
>     printf("%s(): returned from second malloc(1000) call\n", __func__);
>
>     free(p);
>     printf("%s(): returned from second free() call\n", __func__);
>
> and the program ran to completion with no apparent problem.
>
> The compiler does what it can and glibc does what it can, but C is C
> and the heap is what it is, and no, I don't know how this would work
> in Rust.

Which was exactly Richard K's point to Lawrence. It shows that one
simple pattern is detected, but does not support that *all* possible
usage patterns are detected.

In fact, for at least the free() manpage on my Slackware system, it
states this:

The free() function frees the memory space pointed to by ptr, which
must have been returned by a previous call to malloc(), calloc(), or
realloc(). Otherwise, or if free(ptr) has already been called
before, undefined behavior occurs. If ptr is NULL, no operation is
performed.

The paragraph above explicitly states that a second free(ptr) to a ptr
that was already freed, is undefined. And with "undefined" we enter
that wonderful world of "almost anything at all can occur now".

The only two guarantees given in that paragraph are:

1) if ptr was returned by one of the malloc's, the space referenced by
ptr is deallocated (albeit only for the first call of free(ptr), per
the second sentence);
and
2) if ptr is NULL, nothing occurs.

On Tue, 26 Nov 2024 17:55:10 -0700, Louis Krupp wrote:

> In the example in the man page, if the two free() calls are back to back
> with no intervening malloc() calls, the double free() is detected. I
> tried adding a malloc(1000) call in the middle:
>
>     free(p);
>     printf("%s(): returned from first free() call\n", __func__);
>
>     malloc(1000);
>     printf("%s(): returned from second malloc(1000) call\n", __func__);
>
>     free(p);
>     printf("%s(): returned from second free() call\n", __func__);
>
> and the program ran to completion with no apparent problem.

I wonder if alternative memory allocators might not do better.
jemalloc, for example
<https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Find-a-memory-corruption-bug>.

On 11/26/2024 9:39 PM, Lawrence D'Oliveiro wrote:
> On Tue, 26 Nov 2024 17:55:10 -0700, Louis Krupp wrote:
>
>> In the example in the man page, if the two free() calls are back to back
>> with no intervening malloc() calls, the double free() is detected. I
>> tried adding a malloc(1000) call in the middle:
>>
>>     free(p);
>>     printf("%s(): returned from first free() call\n", __func__);
>>
>>     malloc(1000);
>>     printf("%s(): returned from second malloc(1000) call\n", __func__);
>>
>>     free(p);
>>     printf("%s(): returned from second free() call\n", __func__);
>>
>> and the program ran to completion with no apparent problem.
> I wonder if alternative memory allocators might not do better.
> jemalloc, for example
> <https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Find-a-memory-corruption-bug>.
As far as I can tell, the distributed version of jemalloc isn't
configured with the --enable-debug or --enable-fill options, either or
both of which might -- or might not -- help detect problems like this.
In my personal opinion, relying on any library to catch raw pointer
errors at runtime is probably not the way to go. It might be better to
(1) never define an uninitialized pointer, and (2) wrap free() in
function that looked like this (and never use free() by itself):

===
void my_free(void **pp)
{     free(*pp);
    *pp = NULL;
}

....
char *p = NULL;
....
    p = malloc(...);
....
    my_free(&p);
....
===

At this point, it might be easier to code in C++ (using smart pointers)
or Rust (Rust is supposed to be good at this sort of thing).

Louis

On 11/25/24 4:49 PM, Lawrence D'Oliveiro wrote:
> On Mon, 25 Nov 2024 12:04:11 +0000, The Natural Philosopher wrote:
>
>> The problem with Python is it seems to be the new BASIC.
>
> No, it is not. It is a far superior basis on which to build on than BASIC
> ever was.
>
>> An entry point for PWCP People Who Cant Program.
>
> There is a “new BASIC” that fits that description: it’s PHP.

Ummmmmm ... CAN use it for general programming ... but
it's not super-pleasant :-)

Good old BASIC worked well, was fairly easy to read and
understand and once you could structure stuff nicely ...
really nothing *wrong* with it other than being old
and unfashionable.

Try : https://www.freebasic.net/

Python is "better" - but not THAT much better. It has
become totally ubiquitous so there's really no avoiding
the lang.

On 11/26/24 4:12 AM, D wrote:
>
>
> On Mon, 25 Nov 2024, 186282@ud0s4.net wrote:
>
>> On 11/25/24 4:56 AM, D wrote:
>>>
>>>
>>> On Mon, 25 Nov 2024, 186282@ud0s4.net wrote:
>>>
>>>> On 11/24/24 7:36 PM, Lawrence D'Oliveiro wrote:
>>>>> On Sun, 24 Nov 2024 19:35:57 -0500, 186282@ud0s4.net wrote:
>>>>>
>>>>>> Do kinda pref "{ }" or "begin end" over the dangling depth thing ...
>>>>>> get six or eight levels into something and it's a total bitch to spot
>>>>>> what's inside what without using comments.
>>>>>
>>>>> So use the comments. That’s what I do.
>>>>
>>>>  Only good way ...
>>>>
>>>>  Not a killer, but kinda annoying. The eye doesn't
>>>>  spot indents nearly was well as hard delimiters.
>>>>
>>>
>>> You can have indents _and_ delimiters for the ultimate in eye
>>> spotting capability! ;)
>>
>>  As I said to Larry, I almost always use just 2-space
>>  indents so deeply-nested stuff doesn't tend to run
>>  off the page margin. Object langs make this even worse
>>  with all the something.something.something.something
>>  sorts of lines.
>>
>>  SOME of the IDEs for Python KINDA help, can spot
>>  nestings fairly well, but I mostly just use nano
>>  in one terminal and do test runs from another.
>>  Something like PyCharm or Visual are kinda overkill
>>  most of the time.
>>
>
> I use four, but since worked as a systems administrator (or what today
> be called "devops") I never wrote any programs large enough, or
> complicated enough, to run out of line space.

My last big Python app was about 450 lines of code - and
it had LOTS of option switches (TOO many!). Things got
nested really deep sometimes.

Shrank that to about 250 lines of Pascal (the re-think
plus leaving out the options even I never used).

> This is what I do not like about power shell. Some of the commands are
> way too long to type. I like ls, df, du & co! It would be horrible to
> have to type list_files every time.

I kinda have to admit, or brag, that I never used PowerShell.

But yea, shortish generally IS a lot better. Longish is
one reason I hate JS, and then there was COBOL :-)

Haven't done a COBOL app for a long time ... I'll have
to do something ... found a COBOL IDE of sorts somewhere ...
ah, OpenCobolIDE (a PyPy pgm).

On Wed, 27 Nov 2024, rbowman wrote:

> On Tue, 26 Nov 2024 21:58:45 +0100, D wrote:
>
>> Ahh... and today you have agile snakeoil salesmen! I heard a story from
>> the girlfriend of one of my consultants. Her company, a computer game
>> company, hired an "agile coach" who was workshopping away like a madman.
>
> Right... I wonder if they even believe in the snake oil.We had a police
> chief who was a pretty nice guy but had a weakness for traveling medicine
> shows that would teach the officers about gang violence and so forth. Even
> the low lifes in town knew the 'gangs' were bored 12 year olds with spray
> cans that would last 4 seconds in Compton but he lapped it up.
>
> Then the Hells Angels came for vacation and he went off the deep end. The
> Angels were better behaved than when the BMW riders came to town but he
> imported cops from another state to deal with the potential bloody
> violence. The violence turned out to be the out of state cops arresting
> college kids who were protesting the over the top treatment of the bikers.
>
> City council went ballistic when they got the bill and it turned out it's
> against state law to import cops, something that goes back to the copper
> mines and Pinkertons, so the chief was fired.
>
> I'm sure the snake oil salesmen took the show to some other town. The
> agile people probably use the same model.
>

Next up... AI snakeoil salesmen! I saw in the swedish mainstream news,
that ex BP chairman Carl-Henrik Svanbergs has been the government
AI-commissioner.

He is now saying that only the government can make sweden a leading
AI-nation. Therefore the government must create a program educating the
population at large in Chat GPT, and it must "kickstart innovation".

I find this hilarious since the Northvolt battery factory is filing for
chapter 11 and the governments retirement funds invested some hundreds of
millions of dollars in this green bubble. Now they will probably invite
Microsoft to become swedens "AI" partner, teaching the population about
Chat GPT, and probably a government requirement in all tenders for end
user software must be that it has Chat GPT.

Nothing like the government when it comes to inviting the latest snakeoil
salesmen! =)

On Wed, 27 Nov 2024, 186282@ud0s4.net wrote:

> On 11/26/24 4:12 AM, D wrote:
>>
>>
>> On Mon, 25 Nov 2024, 186282@ud0s4.net wrote:
>>
>>> On 11/25/24 4:56 AM, D wrote:
>>>>
>>>>
>>>> On Mon, 25 Nov 2024, 186282@ud0s4.net wrote:
>>>>
>>>>> On 11/24/24 7:36 PM, Lawrence D'Oliveiro wrote:
>>>>>> On Sun, 24 Nov 2024 19:35:57 -0500, 186282@ud0s4.net wrote:
>>>>>>
>>>>>>> Do kinda pref "{ }" or "begin end" over the dangling depth thing ...
>>>>>>> get six or eight levels into something and it's a total bitch to spot
>>>>>>> what's inside what without using comments.
>>>>>>
>>>>>> So use the comments. That’s what I do.
>>>>>
>>>>>  Only good way ...
>>>>>
>>>>>  Not a killer, but kinda annoying. The eye doesn't
>>>>>  spot indents nearly was well as hard delimiters.
>>>>>
>>>>
>>>> You can have indents _and_ delimiters for the ultimate in eye spotting
>>>> capability! ;)
>>>
>>>  As I said to Larry, I almost always use just 2-space
>>>  indents so deeply-nested stuff doesn't tend to run
>>>  off the page margin. Object langs make this even worse
>>>  with all the something.something.something.something
>>>  sorts of lines.
>>>
>>>  SOME of the IDEs for Python KINDA help, can spot
>>>  nestings fairly well, but I mostly just use nano
>>>  in one terminal and do test runs from another.
>>>  Something like PyCharm or Visual are kinda overkill
>>>  most of the time.
>>>
>>
>> I use four, but since worked as a systems administrator (or what today be
>> called "devops") I never wrote any programs large enough, or complicated
>> enough, to run out of line space.
>
> My last big Python app was about 450 lines of code - and
> it had LOTS of option switches (TOO many!). Things got
> nested really deep sometimes.
>
> Shrank that to about 250 lines of Pascal (the re-think
> plus leaving out the options even I never used).
>
>> This is what I do not like about power shell. Some of the commands are way
>> too long to type. I like ls, df, du & co! It would be horrible to have to
>> type list_files every time.
>
> I kinda have to admit, or brag, that I never used PowerShell.
>
> But yea, shortish generally IS a lot better. Longish is
> one reason I hate JS, and then there was COBOL :-)
>
> Haven't done a COBOL app for a long time ... I'll have
> to do something ... found a COBOL IDE of sorts somewhere ...
> ah, OpenCobolIDE (a PyPy pgm).
>

Ahh yes... forgot about javascript. It makes me cry when I see it. =(

Note that the 250 lines of pascal could be reduced further to 1 line if
you find out the true name of god and use that instead!

On 11/27/24 00:55, Louis Krupp wrote:
> On 11/26/2024 2:22 PM, Lawrence D'Oliveiro wrote:
>> On Tue, 26 Nov 2024 08:32:32 +0000, Richard Kettlewell wrote:
>>
>>>> The examples show that freeing the same pointer twice can be
>>>> detected reliably.
>>> They do no such thing.
>> free(p);
>> printf("%s(): returned from first free() call\n", __func__);
>>
>> free(p);
>> printf("%s(): returned from second free() call\n", __func__);
>>
>> Is that or is that not freeing the same pointer twice?
>
> In the example in the man page, if the two free() calls are back to back
> with no intervening malloc() calls, the double free() is detected. I
> tried adding a malloc(1000) call in the middle:
>
>     free(p);
>     printf("%s(): returned from first free() call\n", __func__);
>
>     malloc(1000);
>     printf("%s(): returned from second malloc(1000) call\n", __func__);
>
>     free(p);
>     printf("%s(): returned from second free() call\n", __func__);
>
> and the program ran to completion with no apparent problem.
>

Yes, possibly this malloc reassigned the same address as the unseen
original malloc. In general, without fat pointers, it is hard to see how
free could detect that memory had been reassigned.

It is interesting to know that unoptimised compilations try to spot
double free, but only a little interesting. I remember we ran a tool
called Purify to spot this type of error, amongst other errors.
Presumably that injected additional code to check reliably.

On 11/26/24 14:15, Chris Ahlstrom wrote:
> Rich wrote this post while blinking in Morse code:
>
>> Pancho <Pancho.Jones@proton.me> wrote:
>>>
>>> <snip>
>>>
>>> The fundamental characteristic of a good programmer is to be able to
>>> deliver an application that is useful. Everything else is secondary.
>
> Nah, the application must also be maintainable.
>

The need for maintainability is a consequence of people wanting to use a
program. Even useful programs, may never need to be maintained.

Additionally, too much concentration on maintainability in development
can delay a project and cause cancellation before delivery.

On 26/11/2024 18:57, rbowman wrote:
> On Tue, 26 Nov 2024 13:17:08 -0000 (UTC), Rich wrote:
>
>> Yep, exactly. If they can be given instructions that match their "lego
>> brick set" they can snap something together. Ask them to do anything
>> that requires creativity or research and understanding, and you get back
>> a turd that has had hours of polishing applied.
>
> The holy grail for management is a design methodology that gets adequate
> results from a workforce of varying aptitudes. Particularly for larger
> corporations you'll get a normal distribution, a few very good, a few
> completely useless, and a lot of mediocrity. That's what you have to work
> with.
>

I've encountered that in my time in mil spec aerospace.
A very few people analysed the project and broke it down in to circuit
board specs.

(Even down to a board the sized of a paperback with gold plated edge
connector containing 6 resistors two capacitors and an 8 pin IC)

Then a selection of random monkeys who could solder fucked around with
random components *until it met the specification*.

And the whole thing was a closed feedback loop. You went round and round
until the whole bloody thing either met the spec, or ran out of
government budget,

> What I've seen over the years is a company will luck out, get a better
> than average distribution, and achieve success. Whatever they're doing is
> taken as an example of the right way and copied mechanically. Top down
> structured programming, agile, devops, and so forth have their day.
>

Indeed. At top management level, one does not care how they did it, only
that they did.

> TI lucked out in the '70s and used something they called 'matrix
> management' that became the new Wunderkind. The '80s brought 'In Search of
> Excellence'.
>
> https://en.wikipedia.org/wiki/In_Search_of_Excellence
>
> Good money was made from book sales and training sessions from independent
> snake oil salesmen preaching the gospel. The company I worked for had one
> of the sessions. Not too many of the 'excellent' corporations are around
> today.
>
I was surprised to discover that what I did naturally - coming in in the
morning and looking around to see who looked miserable and asking them
what was wrong, and then trying to sort it out, was actually known as
'management by walking about'

> About 10 years ago the company I now work for had a 'pair programming'
> session. That was hilarious. The 'experts' were only familiar with Apple
> machines and other than the one they brought there wasn't an Apple in the
> building. Having been through required attendance things before my team
> carefully stayed to the back of the room where we could slink away and get
> back to business.
>
> I'm sure the next methodology will wrap itself around AI, spin off
> training companies, and mostly fail to deliver on the promises.
>
Nothing is worse than a CompSci graduate trying to introduce a whole new
language based on its elegance, expect a MBA (Mostly bloody arseholes)
trying to implement the latest fashionable management theory with no
idea as to why it is fashionable, where it might fit, or what good it
might do if any.

--
"What do you think about Gay Marriage?"
"I don't."
"Don't what?"
"Think about Gay Marriage."

On 27/11/2024 01:48, Rich wrote:
> The only two guarantees given in that paragraph are:
>
> 1) if ptr was returned by one of the malloc's, the space referenced by
> ptr is deallocated (albeit only for the first call of free(ptr), per
> the second sentence);
> and
> 2) if ptr is NULL, nothing occurs.

Exactly. The result of anything else is implementation dependent on the
library.

One might write a free that says 'look at our list of allocated blocks,
compare with the pointer and if no match do nothing'

Or one that says 'look at our list of allocated block compare with the
pointer and if no match, segfault'

I am not au fait with the internal of 'free()' so I cannot comment as to
why the first is not always the case.

--
“People believe certain stories because everyone important tells them,
and people tell those stories because everyone important believes them.
Indeed, when a conventional wisdom is at its fullest strength, one’s
agreement with that conventional wisdom becomes almost a litmus test of
one’s suitability to be taken seriously.”

Paul Krugman

The Natural Philosopher <tnp@invalid.invalid> wrote:
> On 27/11/2024 01:48, Rich wrote:
>> The only two guarantees given in that paragraph are:
>>
>> 1) if ptr was returned by one of the malloc's, the space referenced by
>> ptr is deallocated (albeit only for the first call of free(ptr), per
>> the second sentence);
>> and
>> 2) if ptr is NULL, nothing occurs.
>
> Exactly. The result of anything else is implementation dependent on the
> library.
>
> One might write a free that says 'look at our list of allocated blocks,
> compare with the pointer and if no match do nothing'
>
> Or one that says 'look at our list of allocated block compare with the
> pointer and if no match, segfault'
>
> I am not au fait with the internal of 'free()' so I cannot comment as to
> why the first is not always the case.

I can hazzard a guess. The time taken to perform the search, or the
effort needed to maintain an "index structure" to perform an optimized
search, plus the time for the optimized search, was felt to be
excessive and wasteful when the "spec" says "don't ever do this".

On 27/11/2024 16:33, Rich wrote:
> The Natural Philosopher <tnp@invalid.invalid> wrote:
>> On 27/11/2024 01:48, Rich wrote:
>>> The only two guarantees given in that paragraph are:
>>>
>>> 1) if ptr was returned by one of the malloc's, the space referenced by
>>> ptr is deallocated (albeit only for the first call of free(ptr), per
>>> the second sentence);
>>> and
>>> 2) if ptr is NULL, nothing occurs.
>>
>> Exactly. The result of anything else is implementation dependent on the
>> library.
>>
>> One might write a free that says 'look at our list of allocated blocks,
>> compare with the pointer and if no match do nothing'
>>
>> Or one that says 'look at our list of allocated block compare with the
>> pointer and if no match, segfault'
>>
>> I am not au fait with the internal of 'free()' so I cannot comment as to
>> why the first is not always the case.
>
> I can hazzard a guess. The time taken to perform the search, or the
> effort needed to maintain an "index structure" to perform an optimized
> search, plus the time for the optimized search, was felt to be
> excessive and wasteful when the "spec" says "don't ever do this".
>

Look I don't know how memory allocation and de allocation is done
but my instinct would be to hold a long list of pointers to allocated
blocks. Possibly in order of the allocated addresses, which would make
allocating a new block a case of finding the first gap in the list that
is [required blocksize] and inserting a new list element, and de
allocation a question of searching the list for a matching allocation,
and deleting it from the list.

It would be trivial to get to the end of the list and discover that that
was the end, and, with no match found simply ignore the free call

--
Climate Change: Socialism wearing a lab coat.

The Natural Philosopher <tnp@invalid.invalid> writes:
> On 27/11/2024 01:48, Rich wrote:
>> The only two guarantees given in that paragraph are:
>> 1) if ptr was returned by one of the malloc's, the space referenced
>> by
>> ptr is deallocated (albeit only for the first call of free(ptr), per
>> the second sentence);
>> and
>> 2) if ptr is NULL, nothing occurs.
>
> Exactly. The result of anything else is implementation dependent on
> the library.
>
> One might write a free that says 'look at our list of allocated
> blocks, compare with the pointer and if no match do nothing'
>
> Or one that says 'look at our list of allocated block compare with
> the pointer and if no match, segfault'
>
> I am not au fait with the internal of 'free()' so I cannot comment as
> to why the first is not always the case.

You can absolutely write an allocator that does that if you want. You’ll
pay for it in performance and memory usage, but it might well be
worthwhile in a given application.

However, it doesn’t fully solve the problem. Suppose the sequence of
operations (with lots going on between) happens to look like this:

int *p = malloc(xxx);
// ...
free(p);
// ...
int *q = malloc(yyy);
// ...
free(p); // double free of p
// ...
*q = 1;

If p!=q then a tracking allocator like you envisage would spot the
error. But if it happens that p=q (which is entirely possible) then the
tracking allocator won’t notice the problem. The program will start to
misbehave seriously when the it access the (now freed) memory pointed to
be q.

Nulling out pointers after freeing them, as suggested elsewhere helps,
but that depends on perfect play by human programmers, not something you
want to rely on.

To go further you need to track not just the status of each allocation
but the provenance of the pointers to. valgrind does that (and a lot
more) but the performance penalty means it’s only practical to use for
testing and debugging.

--
https://www.greenend.org.uk/rjk/

On Wed, 27 Nov 2024 12:05:02 +0000, The Natural Philosopher wrote:

> Or one that says 'look at our list of allocated block compare with the
> pointer and if no match, segfault'

https://elinux.org/Electric_Fence

I often use efence as a quick check. valgrind is more comprehensive but
when you run the programs with gdb any questionable access drops you into
the debugger then and there.

The Natural Philosopher <tnp@invalid.invalid> wrote:
> On 27/11/2024 16:33, Rich wrote:
>> The Natural Philosopher <tnp@invalid.invalid> wrote:
>>> On 27/11/2024 01:48, Rich wrote:
>>>> The only two guarantees given in that paragraph are:
>>>>
>>>> 1) if ptr was returned by one of the malloc's, the space referenced by
>>>> ptr is deallocated (albeit only for the first call of free(ptr), per
>>>> the second sentence);
>>>> and
>>>> 2) if ptr is NULL, nothing occurs.
>>>
>>> Exactly. The result of anything else is implementation dependent on the
>>> library.
>>>
>>> One might write a free that says 'look at our list of allocated blocks,
>>> compare with the pointer and if no match do nothing'
>>>
>>> Or one that says 'look at our list of allocated block compare with the
>>> pointer and if no match, segfault'
>>>
>>> I am not au fait with the internal of 'free()' so I cannot comment as to
>>> why the first is not always the case.
>>
>> I can hazzard a guess. The time taken to perform the search, or the
>> effort needed to maintain an "index structure" to perform an optimized
>> search, plus the time for the optimized search, was felt to be
>> excessive and wasteful when the "spec" says "don't ever do this".
>>
>
> Look I don't know how memory allocation and de allocation is done
> but my instinct would be to hold a long list of pointers to allocated
> blocks. Possibly in order of the allocated addresses, which would make
> allocating a new block a case of finding the first gap in the list that
> is [required blocksize] and inserting a new list element, and de
> allocation a question of searching the list for a matching allocation,
> and deleting it from the list.
>
> It would be trivial to get to the end of the list and discover that that
> was the end, and, with no match found simply ignore the free call

Yes, but now you burden /every/ free(ptr) call with an O(N) linear
search of all allocated blocks to determine if "ptr" has been
previously freed. With today's CPU's one could be excused in thinking
"not such a big deal". In the days of a PDP-11 or VAX-11/780 CPU
performance levels, doing an O(N) linear search, on every call to free,
to catch something the programmer was never supposed to do in the first
place, was likely viewed as too much overhead.

On Wed, 27 Nov 2024 10:12:06 +0100, D wrote:

> Ahh yes... forgot about javascript. It makes me cry when I see it. =(

Apropos:

https://www.infoworld.com/article/3612364/uspto-petitioned-to-cancel-
oracles-javascript-trademark.html

Oracle is doing their usual dog in the manger tactic with ECMAScript.
They acquired the trademark from Sun and never had a part in it. I wonder
if Eich wakes up at 3 AM thinking about what he created? He wasn't happy
with the name in the first place.

On Wed, 27 Nov 2024 11:58:35 +0000, The Natural Philosopher wrote:

> I've encountered that in my time in mil spec aerospace.
> A very few people analysed the project and broke it down in to circuit
> board specs.

I worked on one DoD project in my career. I got bored and wandered off
after 6 months of wrangling about the spec. About a year later I talked to
one of the programmers who had stayed and asked if they'd written any code
yet. "Nope."

So much blood, sweat, tears, and ego involvement is involved in specs like
that it will be implemented even if it becomes apparent it isn't going to
work. Fiascos like the F-35 don't surprise me at all.

On Wed, 27 Nov 2024, rbowman wrote:

> On Wed, 27 Nov 2024 10:12:06 +0100, D wrote:
>
>> Ahh yes... forgot about javascript. It makes me cry when I see it. =(
>
> Apropos:
>
> https://www.infoworld.com/article/3612364/uspto-petitioned-to-cancel-
> oracles-javascript-trademark.html
>
> Oracle is doing their usual dog in the manger tactic with ECMAScript.
> They acquired the trademark from Sun and never had a part in it. I wonder
> if Eich wakes up at 3 AM thinking about what he created? He wasn't happy
> with the name in the first place.

Ahh Oracle... a perfect example of what the maffia would look like if they
were a corporation. ;)

On Wed, 27 Nov 2024 09:27:35 +0000, Pancho wrote:

> I remember we ran a tool
> called Purify to spot this type of error, amongst other errors.
> Presumably that injected additional code to check reliably.

As I understand it, Valgrind does it on your executable without source
code changes.

On Wed, 27 Nov 2024 17:20:44 +0000, The Natural Philosopher wrote:

> ... my instinct would be to hold a long list of pointers to allocated
> blocks.

Actually you want a list of free blocks. Structures that include allocated
blocks would be for consistency-checking purposes.

Then you have to decide on what to do if an allocation request doesn’t
exactly match the size of an available free block: do you do “first-fit”
(allocate part of the first block that’s big enough, and return the
remainder to the free list) or “best-fit” (allocate part of the smallest
available block that’s big enough, and return any leftover to the free
list)? The latter might minimize fragmentation, but take more work.

Then you might have “lookaside lists” for common block sizes, to speed up
allocations of those sizes.

Then you realize that having a single list becomes a bottleneck in a
multithreaded program, so you really need multiple free lists. Then you
have to watch out for certain lists shrinking while others grow, and
figure out how to rebalance them once in a while.

There’s a whole lot more to the issue, but you start to get the idea.