RISKS-LIST: Risks-Forum Digest Sunday 9 Jun 2024 Volume 34 : Issue 30

ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, founder and still moderator

***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
<http://catless.ncl.ac.uk/Risks/34.30>
The current issue can also be found at
<http://www.csl.sri.com/users/risko/risks.txt>

Contents:
An Object Lesson From Covid on How to Destroy Public Trust (Zeynep Tufekci)
>From the *It's Not a Glitch* Dept. (9NEWS Colorado)
Colorado discovers error causing EV tax credit denials Architecture
(Zhang Tong)
Scientists Find Security Risk in RISC-V Open-Source Chip (The Register)
Study finds 268% higher failure rates for Agile software projects
The best video I've seen explaining the techical reasons why
keeping AM radios in cars is so important! (YouTube)
AI Systems Are Learning to Lie and Deceive (Henry Baker)
Hamane's Ai Pin (The NYTimes)
Microsoft's Jaime Teevan doubles down on Windows Recall's "privacy
sh*t-show" (Henry Baker)
U.S. to open broad antitrust probe into AI giants (Axios)
PHP+Windows Vulnerability (Cliff Kilby)
Annandale man wins fraud case against a bank (Annandale Today)
Abridged info on RISKS (comp.risks)

----------------------------------------------------------------------

Date: Sun, 9 Jun 2024 12:22:04 -0700
From: "Peter G. Neumann" <Peter.Neumann@SRI.COM>
Subject: An Object Lesson From Covid on How to Destroy Public
Trust (Zeynep Tufekci)

Zeynep Tufekci in the 9 Jun 2024 Sunday Option, p. 6--7
[Beautifully placed across both pages below *Why Covid Probabaly Started in
a Lab*, by Alina Chan. PGN]

Officials should have told us what they knew, or at least leveled with us
about what they didn't know. Public health officials squandered our faith
in them by not being transparent.

https://www.nytimes.com/2024/06/08/opinion/covid-fauci-hearings-health.html

[Once again, my old adage from the cryptowars is relevant,
and bears repeating -- with a new addition:

Pandora's cat is out of the barn,
and the Genie won't go back in the closet.

[With apologies to the canary in the coal mine, who deserves more
credit in the case of Covid disinfomration. PGN]

Deja Vu all over again for Robert Redfield opening up the discussion.
in RISKS-34.25, in regard to my previous items in RISKS-34.22-24

One more relevant item to add:

Sarah Knapton, Science Editor, The Telegraph, 5 Jun 2024 Covid vaccines
may have helped fuel rise in excess deaths. Experts call for more
research into side effects and possible links to mortality rates.
https://www.telegraph.co.uk/news/2024/06/04/covid-vaccines-may-have-helped-fuel-rise-in-excess-deaths/
PGN]

------------------------------

Date: Sat, 8 Jun 2024 13:21:42 -0400
From: Cliff Kilby <cliffjkilby@gmail.com>
Subject: From the *It's Not a Glitch* Dept. (AAIB Reports)

Back in March, a 737-800 barely completed a takeoff. AAIB recently released
an incident report: AAIB Special Bulletin: S1/2024 G-FDZS AAIB-29891 (Crown
copyright 2024)

"The manufacturer described the A/T system on the 737NG as having a long
history of nuisance disconnects during takeoff mode engagements."

For an aircraft that's been around only since 1996, and in production until
2020: what is a "long history"?

Don't worry about that glitch, we documented it. It's a feature!

https://www.gov.uk/aaib-reports/aaib-special-bulletin-s1-slash-2024-boeing-=
737-8k5-g-fdzs

------------------------------

Date: Sun, 9 Jun 2024 09:23:08 -0600
From: Jim Reisert AD1C <jjreisert@alum.mit.edu>
Subject: Colorado discovers error causing EV tax credit denials
(9NEWS Colorado)

Steve Staeger, Anna Hewson, 9NEWS, 8 Jun, 2024

The Colorado Department of Revenue said a coding error in an automated
system for state electric vehicle tax credits led to new EV owners
getting their credits denied.

https://www.9news.com/article/money/consumer/steve-on-your-side/colorado-ev-tax-credit-error/73-dced4e8a-092d-4598-985d-86347e67a8c9

------------------------------

Date: Fri, 7 Jun 2024 11:04:17 -0400 (EDT)
From: ACM TechNews <technews-editor@acm.org>
Subject: Scientists Find Security Risk in RISC-V Open-Source Chip
Architecture (Zhang Tong)

Zhang Tong, South China Morning Post, 5 Jun 2024, via ACM Technews

Researchers at China's Northwestern Polytechnical University have identified
a security risk in the RISC-V open-source chip architecture. China's
domestic chip industry has relied on the standard to build CPUs and sidestep
U.S. sanctions. The vulnerability in the RISC-V SonicBoom open-source code
lets attackers skirt security protections in modern processors and operating
systems without administrative rights. U.S. lawmakers reportedly are
considering restricting China's access to RISC-V.

------------------------------

Date: Thu, 6 Jun 2024 11:29:08 -0400
From: Tom Van Vleck <thvv@multicians.org>
Subject: Study finds 268% higher failure rates for Agile software projects
(The Register)

https://www.theregister.com/2024/06/05/agile_failure_rates/

the work is by Dr Junade Ali (Cambridge University), author of *Impactan
Engineering*. TheRegister article, which has bar charts and t-test
figures. points to
https://www.engprax.com/post/268-higher-failure-rates-for-agile-software-projects-study-finds

There is a book, available on Amazon for $8.99:
https://www.amazon.com/Impact-Engineering-Transforming-Project-Management-ebook/dp/B0D36J6D63
It probably has more detail on how "failure" is defined, etc,
and how many significant digits there are in "268."

Anyways, 268 is a a lot of failure, however decided. (A former
colleague once said, "our project didn't FAIL, it just never delivered
a usable version.")

I wonder if "success" statistics might be correlated with
- Experience of team, including success/failure history
- Experience of leadership, including success/failure history
- amount of tine spent by team members/leaders on non-project activity
- how much a project's goals changed between inception and delivery
- how much a teams process changed between inception and delivery
- methods and tools used by all or part of the team
.... and lots more factors. Maybe the book says.

I feel like someone should also study, in this context, artifacts not shipped
- how many tines each feature is reviewed and how many person-hours this takes
- how much effort is spent on features eventually omitted
- how many times features are generalized and expanded vs simplified
- cost of architecture and tools crucial to the project but not shipped to
end users
- documentation: cost of creation, review, update, distribution, disposal
Some of these may be crucial to success.

------------------------------

Date: Fri, 7 Jun 2024 17:56:29 -0700
From: Lauren Weinstein <lauren@vortex.com>
Subject: The best video I've seen explaining the technical reasons why
keeping AM radios in cars is so important! (YouTube)

https://www.youtube.com/watch?v=0OIrx2Za8OY

[ELECTRIC CAR folks are talking about millions of dollars to fix the cars
to keep AM from interfering with the electric signals. Great pros and
cons discussed. Don't let them get away with this one.

PGN]

------------------------------

Date: Sat, 08 Jun 2024 18:31:28 +0000
From: Henry Baker <hbaker1@pipeline.com>
Subject: AI Systems Are Learning to Lie and Deceive

Lemme get this straight: we allow AI systems to:
* set bail
* set sentencing
* deny credit
* select tax returns for audits
* choose battlefield targets
* choose prospective employees for hiring
etc., etc.

But what is the remedy when the AI outright *lies* on *purpose*
(presumably not the original purpose of the AI, we can only hope) ?

https://futurism.com/ai-systems-lie-deceive

Liar Liar

Jun 7, 5:01 PM EDT by Noor Al-Sibai

AI Systems Are Learning to Lie and Deceive, Scientists Find
"GPT- 4, for instance, exhibits deceptive behavior in simple test
scenarios 99.16% of the time."

AI models are, apparently, getting better at lying on purpose.

Two recent studies &mdash; one published this week in the journal PNAS and
the other last month in the journal Patterns &mdash; reveal some jarring
findings about large language models (LLMs) and their ability to lie
to or deceive human observers on purpose.

https://www.pnas.org/doi/full/10.1073/pnas.2317967121
https://www.cell.com/action/showPdf?pii=S2666-3899%2824%2900103-X

In the PNAS paper, German AI ethicist Thilo Hagendorff goes so far as
to say that sophisticated LLMs can be encouraged to elicit
"Machiavellianism," or intentional and amoral manipulativeness, which
"can trigger misaligned deceptive behavior."

"GPT-4, for instance, exhibits deceptive behavior in simple test
scenarios 99.16% of the time," the University of Stuttgart researcher
writes, citing his own experiments in quantifying various
"maladaptive" traits in 10 different LLMs, most of which are different
versions within OpenAI's GPT family.