I have a Raspberry Pi in my boat that uses I2C to read a number of
voltages and currents (using ADS1115 A2D) so I can monitor the battery
condition etc.

At present various different scripts (i.e. processes) just read the
values using the I2C bus whenever they need to but I'm pretty sure
this (quite rarely) results in false readings because two processes
try to read at the same time.

Thus I'm looking for ways to prevent simultaneous access.

One fairly obvious way is to have single process/script which reads
the A2D values continuously and writes them to a file. All other
scripts then read from the file as needed, a simple file lock can then
be used to prevent simultaneous access (well, simultaneous access when
the writing process is writing).

Is this the simplest approach? Are there better ways using
multiprocess? (They look more complicated though).

The I2C bus itself has a mutex but I don't think this guarantees that
(for example) an A2D reading is atomic because one reading takes more
than one I2C bus access.

Would a mutex of some sort around each I2C transaction (i.e. complete
A2D reading) be a better way to go?

--
Chris Green
·

Chris Green <cl@isbd.net> wrote or quoted:
>Would a mutex of some sort around each I2C transaction (i.e. complete
>A2D reading) be a better way to go?

I not an expert here, but from what I heard, you could use
a named POSIX semaphore as a kind of system-wide mutex.

Or, the single process reading the A2D values could write
them to a shared memory segment the access to which is
controlled by a mutex.

Or, you could use a message queue (like POSIX mq) to
distribute readings.

But why overengineer? If you feel comfortable with the file
solution, go for it! The only drawback might be that it's a
bit slower than other approaches.

On 06/07/2024 12:32, Stefan Ram wrote:
> But why overengineer? If you feel comfortable with the file
> solution, go for it! The only drawback might be that it's a
> bit slower than other approaches.

I absolutely agree. Overengineering is generally a bad idea because
you're using a complex solution to solve a simple problem, which leads
to unexpected breakage.

The file solution is perfectly fine, and file locks are used by really
important software (i.e package managers and such) because its
simplicity makes it almost never fail.

So yeah, go for it!

On 06/07/2024 09.28, Chris Green wrote:
> I have a Raspberry Pi in my boat that uses I2C to read a number of
> voltages and currents (using ADS1115 A2D) so I can monitor the battery
> condition etc.
>
> At present various different scripts (i.e. processes) just read the
> values using the I2C bus whenever they need to but I'm pretty sure
> this (quite rarely) results in false readings because two processes
> try to read at the same time.
>
> Thus I'm looking for ways to prevent simultaneous access.

Why using "different scripts"?
Is it there any particular reason?

Maybe it would be better, if possible, to have
a single script, which, sequentially, reads
whatever needs to be read (or written).
In a loop.

This is even simpler than using a file.

bye,

pg

> One fairly obvious way is to have single process/script which reads
> the A2D values continuously and writes them to a file. All other
> scripts then read from the file as needed, a simple file lock can then
> be used to prevent simultaneous access (well, simultaneous access when
> the writing process is writing).
>
> Is this the simplest approach? Are there better ways using
> multiprocess? (They look more complicated though).
>
> The I2C bus itself has a mutex but I don't think this guarantees that
> (for example) an A2D reading is atomic because one reading takes more
> than one I2C bus access.
>
> Would a mutex of some sort around each I2C transaction (i.e. complete
> A2D reading) be a better way to go?
>

--

piergiorgio

On Sat, 6 Jul 2024 08:28:41 +0100, Chris Green wrote:

> One fairly obvious way is to have single process/script which reads the
> A2D values continuously and writes them to a file. All other scripts
> then read from the file as needed, a simple file lock can then be used
> to prevent simultaneous access (well, simultaneous access when the
> writing process is writing).

The thing with a file is, it persists even when the collector process is
not running. Do you want data that persists when the collector process is
not running?

Is this a history of values, or just a snapshot of current values? A
history of values could be written to a database. Databases provide their
own transactions and interlocking to prevent readers from reading partial
updates.

If it’s a snapshot of current values, that does not persist when the
collector process is not running, then why not just keep the data in the
memory of the collector process, and have it concurrently listen on a
socket for connections from readers requesting a copy of the current data?

> On 7 Jul 2024, at 22:13, Chris Green via Python-list <python-list@python.org> wrote:
>
> a simple file lock can then
> be used to prevent simultaneous access (well, simultaneous access when
> the writing process is writing).

There is a simple pattern to make this robust.

Write new values to a tmp file.
Close the tmp file.
Then use os.rename(tmpfile, productionfile).

This is guaranteed that any process that reads the file will only see all the old file contents or all the new file contents, never a mix of both.

Barry

On 2024-07-07 23:27, Barry via Python-list wrote:
>
>
>> On 7 Jul 2024, at 22:13, Chris Green via Python-list <python-list@python.org> wrote:
>>
>> a simple file lock can then
>> be used to prevent simultaneous access (well, simultaneous access when
>> the writing process is writing).
>
> There is a simple pattern to make this robust.
>
> Write new values to a tmp file.
> Close the tmp file.
> Then use os.rename(tmpfile, productionfile).
>
> This is guaranteed that any process that reads the file will only see all the old file contents or all the new file contents, never a mix of both.
>
For clarity I'd recommend os.replace instead. This is because on Windows
os.rename it would complain if the target file already exists, but
os.replace has the same behaviour on both Linux and Windows.

> On 7 Jul 2024, at 23:47, MRAB via Python-list <python-list@python.org> wrote:
>
> For clarity I'd recommend os.replace instead. This is because on Windows os.rename it would complain if the target file already exists, but os.replace has the same behaviour on both Linux and Windows.

Agreed.

In this case the OP is on an RPi.

Barry

If resource usage isn't an issue, then the _easy_ thing to do, that
would also be easily correct is to have a server doing all the
h/w-related reading and clients talking to that server. Use for the
server the technology you feel most confident with. Eg. you may use
Python's http package. I believe that the server from this package
runs in a single thread, and thus processes all requests
synchronously. So, you'll get synchronization for free.

Then, the rest of the scripts that need to talk to h/w will instead be
talking to this server.

Again, this isn't an _efficient_ solution... but, sometimes you don't
need one. And this one is easy to make, easy to debug, easy to expand.
But, if instead you were looking for a more efficient solution, then,
the general idea that allows the http server to work in this case
would still apply: have a single synchronization program that takes
requests asynchronously, and orders them. So, a basic TCP server would
also work as well as a UNIX socket. Your idea with holding a lock on a
file would also work (in fact, plenty of Linux utilities work that
way, eg. apt-get or yum).

----

If you don't want to change the existing script, then instead of
running them directly, you could run them through batch:
https://man7.org/linux/man-pages/man1/batch.1p.html this is a very
simply queuing program that's available for Linux. It will take care
of synchronization by putting the scripts you want to run in a queue
and executing them one at a time.

On Sun, Jul 7, 2024 at 11:12 PM Chris Green via Python-list
<python-list@python.org> wrote:
>
> I have a Raspberry Pi in my boat that uses I2C to read a number of
> voltages and currents (using ADS1115 A2D) so I can monitor the battery
> condition etc.
>
> At present various different scripts (i.e. processes) just read the
> values using the I2C bus whenever they need to but I'm pretty sure
> this (quite rarely) results in false readings because two processes
> try to read at the same time.
>
> Thus I'm looking for ways to prevent simultaneous access.
>
> One fairly obvious way is to have single process/script which reads
> the A2D values continuously and writes them to a file. All other
> scripts then read from the file as needed, a simple file lock can then
> be used to prevent simultaneous access (well, simultaneous access when
> the writing process is writing).
>
> Is this the simplest approach? Are there better ways using
> multiprocess? (They look more complicated though).
>
> The I2C bus itself has a mutex but I don't think this guarantees that
> (for example) an A2D reading is atomic because one reading takes more
> than one I2C bus access.
>
> Would a mutex of some sort around each I2C transaction (i.e. complete
> A2D reading) be a better way to go?
>
> --
> Chris Green
> ·
> --
> https://mail.python.org/mailman/listinfo/python-list

Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
> On Sat, 6 Jul 2024 08:28:41 +0100, Chris Green wrote:
>
> > One fairly obvious way is to have single process/script which reads the
> > A2D values continuously and writes them to a file. All other scripts
> > then read from the file as needed, a simple file lock can then be used
> > to prevent simultaneous access (well, simultaneous access when the
> > writing process is writing).
>
> The thing with a file is, it persists even when the collector process is
> not running. Do you want data that persists when the collector process is
> not running?
>
> Is this a history of values, or just a snapshot of current values? A
> history of values could be written to a database. Databases provide their
> own transactions and interlocking to prevent readers from reading partial
> updates.
>
There's a separate (crontab driven) process that writes the history to
a sqlite3 database,

> If it’s a snapshot of current values, that does not persist when the
> collector process is not running, then why not just keep the data in the
> memory of the collector process, and have it concurrently listen on a
> socket for connections from readers requesting a copy of the current data?

That's exactly the sort of solution I was wondering about. Is there a
ready made module/library for handling this sort of thing? Basically
it will just be a string of a few tens of characters that would be
kept up to date by one process and asked for by all the others.

--
Chris Green
·

Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.removethis.de> wrote:
> On 06/07/2024 09.28, Chris Green wrote:
> > I have a Raspberry Pi in my boat that uses I2C to read a number of
> > voltages and currents (using ADS1115 A2D) so I can monitor the battery
> > condition etc.
> >
> > At present various different scripts (i.e. processes) just read the
> > values using the I2C bus whenever they need to but I'm pretty sure
> > this (quite rarely) results in false readings because two processes
> > try to read at the same time.
> >
> > Thus I'm looking for ways to prevent simultaneous access.
>
> Why using "different scripts"?
> Is it there any particular reason?
>
> Maybe it would be better, if possible, to have
> a single script, which, sequentially, reads
> whatever needs to be read (or written).
> In a loop.
>
> This is even simpler than using a file.
>
Yes, but it's conceptually (and programming wise) much simpler to have
separate scripts. Some of them are simple 'on demand' scripts that I
run from the command line when I want to know something. Others are
scripts that drive displays on control panels.

--
Chris Green
·

Chris Green <cl@isbd.net> wrote or quoted:
>That's exactly the sort of solution I was wondering about. Is there a
>ready made module/library for handling this sort of thing? Basically
>it will just be a string of a few tens of characters that would be
>kept up to date by one process and asked for by all the others.

I'm not an expert here, and just quickly tried to make it
run, so the code will still contain errors and not contain
something necessary, but might give you a starting point.

A process doing something (here: printing an incrementing value
named "info") and also serving requests from other processes
for this "info" value:

import asyncio
import socket

class Server:
def __init__( self ):
self.info = 0

async def handle_client( self, reader, writer ):
data = await reader.read( 100 )
message = data.decode()
addr = writer.get_extra_info( 'peername' )
print( f"Received {message!r} from {addr}" )

if message.strip() == "getinfo":
writer.write( str( self.info ).encode() )
await writer.drain()

writer.close()
await writer.wait_closed()

async def print_number( self ):
while True:
print( self.info )
self.info += 1
await asyncio.sleep( 1 )

async def serve( self ):
server = await asyncio.start_server( self.handle_client, '127.0.0.1', 8888 )
addr = server.sockets[0].getsockname()
print( f'Serving on {addr}' )

async with server:
await asyncio.gather( server.serve_forever(), self.print_number() )

asyncio.run( Server().serve() )

, and an example client:

import socket
import time

while True:
s = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
s.connect( ( '127.0.0.1', 8888 ))
s.send( "getinfo".encode() )
data = s.recv( 100 )
message = data.decode().strip()
print( f"Received: {message}" )
time.sleep( 3 )

.

On Mon, 8 Jul 2024 13:56:34 +0100, Chris Green wrote:

> Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>
>> If it’s a snapshot of current values, that does not persist when the
>> collector process is not running, then why not just keep the data in
>> the memory of the collector process, and have it concurrently listen on
>> a socket for connections from readers requesting a copy of the current
>> data?
>
> That's exactly the sort of solution I was wondering about. Is there a
> ready made module/library for handling this sort of thing? Basically it
> will just be a string of a few tens of characters that would be kept up
> to date by one process and asked for by all the others.

You probably don’t need to define any nontrivial kind of protocol at all.
Just accept a connection from a client, send it the current data in some
convenient format (e.g. JSON), and disconnect.

Stefan Ram <ram@zedat.fu-berlin.de> wrote:
> Chris Green <cl@isbd.net> wrote or quoted:
> >That's exactly the sort of solution I was wondering about. Is there a
> >ready made module/library for handling this sort of thing? Basically
> >it will just be a string of a few tens of characters that would be
> >kept up to date by one process and asked for by all the others.
>
> I'm not an expert here, and just quickly tried to make it
> run, so the code will still contain errors and not contain
> something necessary, but might give you a starting point.
>
> A process doing something (here: printing an incrementing value
> named "info") and also serving requests from other processes
> for this "info" value:
>
[snip]

Thanks, that should get me started! :-)

--
Chris Green
·