RISKS-LIST: Risks-Forum Digest Thursday 25 Jul 2024 Volume 34 : Issue 37

ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, founder and still moderator

***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
<http://catless.ncl.ac.uk/Risks/34.37>
The current issue can also be found at
<http://www.csl.sri.com/users/risko/risks.txt>

Contents:
When it comes to math, AI is dumb (Steve Lohr)
Microsoft's Global Sprawl Under Fire After Historic Outage
(WashPost)
Why no public outrage over CrowdStrike/Microsoft and ATT failures?
(John Rushby, Andy Poggio)
Worldwide BSOD outage (via Rebecca Mercuri)
Crowdstrike references (Cliff Kilby)
Secure Boot is Completely Compromised (ArsTechnica via Wendy Grossman)
Hackers could create traffic jams thanks to flaw in
traffic-light controller, researcher says (TechCrunch)
Encultured: an AI doomer’s video game startup pivots to
medicine. It’ll be fine. (Pivot to AI)
New findings shed light on risks and benefits of integrating AI
into medical decision-making (medicalxpress.com)
Steven Wilson Struggles To Hear That It's Not Him Singingxo
AI-Created Songs (Blabbermouth)
Limitless AI (Gabe Goldberg)
AI captions (Jim Geissman)
Switzerland now requires all government software to be
open source (ZDNET)
Bipartisan legislation that would require all users to use government IDs to
access major websites advances in Senate (NBC News)
LLM AI Bios (Rob Slade)
Re: U.S. Gender Care Is Ignoring Science (Martin Ward)
Re: In Ukraine War, AI Begins Ushering In an Age of Killer Robots
(Amos Shapir)
Re: Fwd: Ozone Hole Mk. II (Cliff Kilby)
Abridged info on RISKS (comp.risks)

----------------------------------------------------------------------

Date: Tue, 23 Jul 2024 10:23:22 PDT
From: Peter G Neumann <neumann@csl.sri.com>
Subject: When it comes to math, AI is dumb (Steve Lohr)

Steve Lohr, *The New York Times* Business Section front
page, 23 Jul 2024

Early computers followed rules. AI follows probabilities. But in
mathematics, there is no probable answer, only the right one.

``This technology does brilliant things, but it doesn't do everything.''
Kristian Hammond

------------------------------

Date: Wed, 24 Jul 2024 10:44:35 -0400 (EDT)
From: ACM TechNews <technews-editor@acm.org>
Subject: Microsoft's Global Sprawl Under Fire After Historic Outage"\
(WashPost)

Cristiano Lima-Strong, Cat Zakrzewski, and Jeff Stein,
*The Washington Post*, 20 Jul 2024

The July 19 computer outage resulting from a defective CrowdStrike update to
Windows systems worldwide shines a spotlight on the global economy's
dependence on Microsoft. Although Microsoft said only an estimated 8.5
million devices were impacted, accounting for less than 1% of computers
running the Windows operating system, U.S. Federal Trade Commission Chair
Lina Khan said it underscores "how concentration can create fragile
systems."

------------------------------

Date: Tue, 23 Jul 2024 12:30:06 -0700
From: John Rushby <rushby@csl.sri.com>
Subject: Why no public outrage over CrowdStrike/Microsoft and ATT
failures?

There's been plenty of anger at the consequences of these failures,
but I'm surprised to see no public outrage over their causes.

The CrowdStrike bug was apparently a null dereference in C++ code
operating in kernel mode.

First of all, this is an inexcusable failure of quality assurance
and raises all manner of questions about CrowdStrike's competence.
Why C++, why no static analysis and other checks?

Then there's the whole rationale for the existence of this stuff. Why do
organizations need a third-party virus detector? And why does it need to
operate in kernel mode?

Because Microsoft's OS is a pile of vulnerabilities.
And why does it need to operate in kernel mode?
And yet they tout Windows 365 as a security "solution". And we use it.

Not to mention that their laggard design combined with ruthless
business practices retarded human progress by a decade
(compared with might have been from Engelbart/PARC/Apple).

It's obvious that any adversary will have cyberattacks ready to go that will
inflict greater and more lasting outages during any period of threat or
conflict.

Then there's the ATT debacle. The public suffers ever more intrusive
and ponderous "security" procedures: cryptographically strong and
frequently changed passwords, dual-factor authentication, you name it.
These impose economic as well as personal costs. Yet in the entire
history of computing I doubt there's been any widespread loss due to
penetration of individual accounts: why bother when it's easier to get
the whole lot from the corporate source?

It seems to me that these failures of corporate competence and
diligence are of the same order (and have the same source) as Boeing's
safety failures, yet the public and government and legislature are not
showing comparable outrage and investigative zeal.

Why not? Is there something we should be doing?

------------------------------

Date: Tue, 23 Jul 2024 20:37:30 -0700
From: Andy Poggio <poggio@csl.sri.com>
Subject: Why no public outrage over CrowdStrike/Microsoft and ATT
failures?

I am very sympathetic to John Rushby’s concerns, and concur with his
analysis.

With respect to Internet security, the guilty parties are the Internet
creators, of which SRI was one of three -— the other two were BBN in
Cambridge, MA and ISI in Santa Monica, CA. Vint Cerf at DARPA was a leader
and driving force. We just didn’t consider security -— it was challenging
enough to get things to work at all.

To give you an example, in the 70s and 80s there were a few Packet Radio
networks around (the precursor to cellular networks). One was in the the
San Francisco Bay Area with its central router located at SRI. There were a
number of radio nodes scattered around local mountain tops as well as in a
van (a model of which is in SRI’s Menlo Park lobby) and other locations.
Anybody could send a certain type of packet to a radio node. When the radio
node received the packet and determined that it was this certain type, it
executed a JUMP to the first word of the packet payload and executed
whatever code the packet happened to contain. These packets were called
“flying subroutines”. Why would anyone implement such a dangerous
capability?

1. The mountaintop nodes had no connection to anything besides the radio
connection. And, the nodes were time-consuming to reach physically. We
needed to have the nodes make changes that we couldn’t predict in
advance, and the nodes were very resource limited, especially RAM. The
flying subroutines solved the problem in an effective, if insecure, way.

2. There were no bad guys. Essentially no one outside of the research
community had ever heard of the Internet, let alone had any access to it.
So security wasn’t yet an issue. To my knowledge, no malicious use of
flying subroutines ever occurred.

So, the take away is: let’s not do this again. And my question is: are we
doing it again with AI and its currently, promising technologies, LLMs and
friends? What security issues are not being addressed?

[Some of you will remember the four-hour ARPAnet collapse on 27 Oct 1980,
when multiple bit-corrupted once-a-minute status messages accidentally
propagated, overflowing the buffers when the six-bit indices could not be
deleted: A > B > C > A with unchecked wrap-around and the first-ever
failure of the deletion algorithm for the previous status messages).
There's an article in my ACM SIGSOFT Software Engineering Notes, vol 6 no
1. by Eric Rosen, Vulnerabilities of network control protocols, January
1981, pp. 6-8. It's online, along with all other SEN issues (thanks to
Will Tracz). BBN learned that the status messages circulating on the net
needed error-detection or even error-correction. PGN]

------------------------------

Date: Tue, 23 Jul 2024 16:31:21 -0400
From: DrM <notable@mindspring.com>
Subject: Worldwide BSOD outage

Pete Buttigieg announced today that the Department of Transportation has
opened an investigation into Delta over flight disruptions. (Search -- pete
Buttigieg Delta -- for a whole bunch of other links to recent news coverage
regarding that carrier.)

Here's his recent postings on X:

..@USDOT has opened an investigation into Delta Air Lines to ensure the
airline is following the law and taking care of its passengers during
continued widespread disruptions. All airline passengers have the right

[PGN noted Delta's Delays Signal Slow Recovery from Tech Outage
Christine Chung and Yan Zhuang, *The New York Times* Business Section front
page, 23 Jul 2024
Scalded by Buttigieg for *unacceptable customer service*.]

------------------------------

Date: Wed, 24 Jul 2024 08:43:27 -0400
From: Cliff Kilby <cliffjkilby@gmail.com>
Subject: CrowdStrike references

If your org had to do a manual recovery from CrowdStrike, you should
probably rotate your BitLocker recovery keys, as this appears to have
happened.