RISKS-LIST: Risks-Forum Digest Saturday 18 November 2023 Volume 33 : Issue 94

ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, founder and still moderator

***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
<http://catless.ncl.ac.uk/Risks/33.94>
The current issue can also be found at
<http://www.csl.sri.com/users/risko/risks.txt>

Contents:
How the Railroad Industry Intimidates Employees Into Putting Speed Before
Safety (ProPublica)
Hikers Rescued After Following Nonexistent Trail on Google Maps (NTimes)
Admission of the state of software (David Lamkin)
500 chatbots read the news and discussed it on social media. Guess
how that went. (Business Insider)
The Problem with Regulating AI (Tim Wu)
ChatGPT Created a Fake Dataset With Skewed Results (MedPage Today)
Researchers Discover New Vulnerability in Large Language Models
(Carnegie Mellon University)
Ten ways AI will change democracy (Bruce Schneier)
Fake Reviews Are Rampant Online. Can a Crackdown End Them? (NYTimes)
OpenAI co-founder & president Greg Brockmane quits after firing of
CEO Altman (TechCrunch)
The AI Pin (Rob Slade)
Ukraine's 'Secret Weapon' Against Russia Is a U.S. Tech Company
(Vera Bergengruen)
Cryptographic Keys Protecting SSH Connections Exposed (Dan Goodin)
Developers can't seem to stop exposing credentials in publicly
accessible code (Ars Technica)
Hacking Some More Secure USB Flash Drives -- Part II (SySS Tech Blog)
Social media gets teens hooked while feeding aggression and impulsivity, and
researchers think they know why (CBC)
X marks the non-spot? (PGN adapted from Lauren Weinstein)
It's Still Easy for Anyone to Become You at Experian
(Krebs on Security)
Paying ransom for data stolen in cyberattack bankrolls further crime,
experts caution (CBC)
Toronto Public Library cyber-attack (Mark Brader)
People selling cars via Internet get phished (CBC)
Data breach of Michigan healthcare giant exposes millions of records
(Engadget)
More on iLeakage (Victor Miller)
Using your iPhone to start your car is about to get a lot easier (The Verge)
Massive cryptomining rig discovered under Polish court's floor, stealing
power (Ars Technica)
A Coder Considers the Waning Days of the Craft (The New Yorker via
Steve Bacher)
Re: Industrial Robot Crushes Worker to Death (PGN)
Re: Toyota has built an EV with a fake transmission (Peter Houppermans)
Re: Data on 267,000 Sarnia patients going back 3 decades
among cyberattack thefts at 5 Ontario hospitals Digest (Mark Brader)
Abridged info on RISKS (comp.risks)

----------------------------------------------------------------------

Date: Wed, 15 Nov 2023 23:43:11 -0500
From: Gabe Goldberg <gabe@gabegold.com>
Subject: How the Railroad Industry Intimidates Employees Into
Putting Speed Before Safety (ProPublica)

Railroad companies have penalized workers for taking the time to make needed
repairs and created a culture in which supervisors threaten and fire the
very people hired to keep trains running safely. Regulators say they can’t
stop this intimidation.

Bradley Haynes and his colleagues were the last chance Union Pacific had to
stop an unsafe train from leaving one of its railyards. Skilled in spotting
hidden dangers, the inspectors in Kansas City, Missouri, wrote up so-called
“bad orders” to pull defective cars out of assembled trains and send them
for repairs.

But on Sept. 18, 2019, the area’s director of maintenance, Andrew Letcher,
scolded them for hampering the yard’s ability to move trains on time.

“We're a transportation company, right? We get paid to move freight. We
don't get paid to work on cars,” he said.

https://www.propublica.org/article/railroad-safety-union-pacific-csx-bnsf-trains-freight

------------------------------

Date: Sun, 12 Nov 2023 17:04:52 -0500
From: Jan Wolitzky <jan.wolitzky@gmail.com>
Subject: Hikers Rescued After Following Nonexistent Trail on Google Maps
(

A Canadian search-and-rescue group said it had conducted two missions
recently after hikers may have sought to follow a nonexistent trail on
Google Maps

A search-and-rescue group in British Columbia advised hikers to use a paper
map and compass instead of street map programs after it said two hikers had
been rescued by helicopter after likely following a trail that did not
exist but that appeared on Google Maps.

The group, North Shore Rescue, said on Facebook that on 6 Nov 2023 Google
Maps had removed the nonexistent trail, which was in a very steep area with
cliffs north of Mount Fromme, which overlooks Vancouver.

https://www.nytimes.com/2023/11/12/world/canada/google-maps-trail-british-columbia.html

[Fromme here to eternity? PGN]

------------------------------

Date: Thu, 16 Nov 2023 09:51:56 +0000
From: David Lamkin <drl@shelford.org>
Subject: Admission of the state of software

Having put of buying a 'smart car' for as long as possible I am now the
proud (?) owner of a SEAT Arona. The instruction manual is long and detailed
but one statement does not inspire confidence:

> As with most state-of-the-art computer and electronic equipment, in
> certain cases the system may need to be rebooted to make sure that it
> operates correctly.

This statement should shame all software engineers!

[Does the SEAT Arona have the classical new-seat aroma as an inSCENTive?
PGN]

------------------------------

Date: Thu, 16 Nov 2023 00:29:03 +0900
From: Dave Farber <farber@gmail.com>
Subject: 500 chatbots read the news and discussed it on social media. Guess
how that went. (Business Insider)

https://www.businessinsider.com/ai-chatbots-less-toxic-social-networks-twitter-simulation-2023-11

------------------------------

Date: Sun, 12 Nov 2023 16:09:15 PST
From: Peter Neumann <neumann@csl.sri.com>
Subject: The Problem with Regulating AI (Tim Wu)

Tim Wu, *The New York Times*, 12 Nov 2023

If the government acts prematurely on this evolving
technology, it could fail to prevent concrete harm.

[... and we certainly don't want AI mixing concrete for bridges
and other life-critical structures. PGN]

Final para: The existence of actual social harm has long been a
touchstone of legitimate state action. But that point cuts both
ways: The state should proceed cautiously in the absence of harm,
but it also has duty, given evidence of harm, to take action. By
that measure, with AI we are at risk of doing too much and too
little at the same time.

[The lesser of weasels? That is indeed a well-crafted weasel. Be
careful of what you ask for. You might get [stuck with] it. PGN]

------------------------------

Date: Mon, 13 Nov 2023 20:59:13 +0000
From: Victor Miller <victorsmiller@gmail.com>
Subject: ChatGPT Created a Fake Dataset With Skewed Results (MedPage Today)

https://www.medpagetoday.com/special-reports/features/107247

[What could possibly go wrong? PGN]

------------------------------

Date: Tue, 14 Nov 2023 16:21:02 +0000
From: Victor Miller <victorsmiller@gmail.com>
Subject: Researchers Discover New Vulnerability in Large Language
Models (Carnegie Mellon University)

https://www.cmu.edu/news/stories/archives/2023/july/researchers-discover-new-vulnerability-in-large-language-models

------------------------------

Date: Wed, 15 Nov 2023 08:48:25 +0000
From: Bruce Schneier <schneier@schneier.com>
Subject: Ten ways AI will change democracy

[PGN-extracted from Bruce's CRYPTO-GRAM, 15 Nov 2023]

A free monthly newsletter providing summaries, analyses,
and commentaries on security: computer and otherwise.

** TEN WAYS AI WILL CHANGE DEMOCRACY

[2023.11.13]
[https://www.schneier.com/blog/archives/2023/11/ten-ways-ai-will-change-democrac
y.html]
Artificial intelligence will change so many aspects of society, largely in
ways that we cannot conceive of yet. Democracy, and the systems of
governance that surround it, will be no exception. In this short essay, I
want to move beyond the *AI-generated disinformation* trope and speculate on
some of the ways AI will change how democracy functions -- in both large and
small ways.

When I survey how artificial intelligence might upend different aspects of
modern society, democracy included, I look at four different dimensions of
change: speed, scale, scope, and sophistication. Look for places where
changes in degree result in changes of kind. Those are where the societal
upheavals will happen.

Some items on my list are still speculative, but none require
science-fictional levels of technological advance. And we can see the first
stages of many of them today. When reading about the successes and failures
of AI systems, it's important to differentiate between the fundamental
limitations of AI as a technology, and the practical limitations of AI
systems in the fall of 2023. Advances are happening quickly, and the
impossible is becoming the routine. We don't know how long this will
continue, but my bet is on continued major technological advances in the
coming years. Which means it's going to be a wild ride.