Message-ID:

BOFH excuse #320: You've been infected by the Telescoping Hubble virus.

comp / comp.lang.python / Re: Best use of "open" context manager

Subject: Re: Best use of "open" context manager
From: dn
Newsgroups: comp.lang.python
Organization: DWM
Date: Sat, 6 Jul 2024 21:05 UTC
References: 1 2

Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: PythonList@DancesWithMice.info (dn)
Newsgroups: comp.lang.python
Subject: Re: Best use of "open" context manager
Date: Sun, 7 Jul 2024 09:05:36 +1200
Organization: DWM
Lines: 106
Message-ID: <mailman.9.1720299957.2981.python-list@python.org>
References: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com>
<cd084a1a-fe58-4190-92bb-6f4002517fa5@DancesWithMice.info>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de V+qyn25m6duP5i6dOOTLewiqgfKJNoelEVM3f1Kyg0bA==
Cancel-Lock: sha1:o0Ut03q51G1p4pAWcgIk1QUF9Wg= sha256:3JLs4FyPeTBi8a++U6z50Dv1O7elUSwWpBeyPaJKEOo=
Return-Path: <PythonList@DancesWithMice.info>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=danceswithmice.info header.i=@danceswithmice.info
header.b=gbrQsQu+; dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.064
X-Spam-Evidence: '*H*': 0.87; '*S*': 0.00; 'this:': 0.03; '(which':
0.04; 'e.g.': 0.07; '=dn': 0.09; 'action,': 0.09; 'else:': 0.09;
'from:addr:danceswithmice.info': 0.09; 'from:addr:pythonlist':
0.09; 'memory.': 0.09; 'valueerror:': 0.09; 'coding': 0.13;
'decreases': 0.16; 'ideal.': 0.16; 'message-
id:@DancesWithMice.info': 0.16; 'oop': 0.16; 'pythonic': 0.16;
'received:cloud': 0.16; 'received:rangi.cloud': 0.16; 'targeted':
0.16; 'ways.': 0.16; 'whilst': 0.16; 'wrote:': 0.16; 'to:addr
:python-list': 0.20; 'closed': 0.22; 'exception': 0.22; 'ran':
0.22; 'code': 0.23; 'lines': 0.23; 'idea': 0.24; 'python,': 0.25;
'seems': 0.26; 'manager,': 0.26; 'opening': 0.26; 'bit': 0.27;
'function': 0.27; 'error': 0.29; 'header:User-Agent:1': 0.30;
'attempt': 0.31; 'modify': 0.31; 'header:Organization:1': 0.31;
'context': 0.32; 'python-list': 0.32; 'received:192.168.1': 0.32;
'but': 0.32; 'there': 0.33; 'header:In-Reply-To:1': 0.34;
'handling': 0.35; 'close': 0.35; 'work,': 0.36; 'really': 0.37;
'using': 0.37; 'class': 0.37; 'received:192.168': 0.37; 'file':
0.38; 'could': 0.38; 'put': 0.38; 'read': 0.38; 'two': 0.39;
'quite': 0.39; 'added': 0.39; 'text': 0.39; 'program.': 0.40;
'something': 0.40; 'want': 0.40; 'try': 0.40; 'should': 0.40;
'kept': 0.61; 'remember': 0.61; 'skip:o 10': 0.61; "there's":
0.61; 'here': 0.62; 'between': 0.63; 'similar': 0.65; 'less':
0.65; 'per': 0.68; 'closing': 0.69; 'generator': 0.69; 'small,':
0.69; 'within': 0.69; 'care': 0.71; 'suite': 0.71; 'relevant':
0.73; 'easy': 0.74; 'handles': 0.76; 'exceptions': 0.84; 'forgot':
0.84; 'habits': 0.84; 'readability.': 0.84; 'rob': 0.84; 'styles':
0.84; 'subject:manager': 0.84; 'subject:open': 0.84; 'thus,':
0.84; 'violates': 0.84; 'with...': 0.84; 'loses': 0.91; 'race':
0.93
DKIM-Filter: OpenDKIM Filter v2.11.0 vps.rangi.cloud 2DECB41D0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=danceswithmice.info;
s=staff; t=1720299942;
bh=fJSmBkhRoKWrtMFQyQbslewpLSYe0jmL9yoVrAk3IOk=;
h=Date:From:Subject:To:References:In-Reply-To:From;
b=gbrQsQu+NI/3+KNf+k37t/d09lIHn5HyYmMWvtTFvQiP7zECH1Jjg8BoUxK8/qn/V
FuK6BQv/FQYPjhtvHsE4cdDyqME1340i3+LmGUtVQC9aeQxJ4pAEK4R+Rph18vj78w
aD7nh4ozrNVsfPOUjBESCb1fOlogWW/vUBbTUaYDdIejAUurxbH0Cp13dJU/bR9D5p
aPz3h3/1bJZgsUnWJNRMUNo8ddBK5ieFm1Cgx4lxIA58qodazduXCB/B42s7RwEgm5
Jg0RmszBDYLxnK8gFp81wmu3XepzsGm0oqmh9Asma1TZDAMxt1BBkR+vwf1cMvMlss
ZMi44UqxihoZw==
User-Agent: Mozilla Thunderbird
Content-Language: en-US
In-Reply-To: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <cd084a1a-fe58-4190-92bb-6f4002517fa5@DancesWithMice.info>
X-Mailman-Original-References: <954c4ca8-cf37-4482-a1be-46d39cb503f9@btinternet.com>

View all headers

On 6/07/24 22:49, Rob Cliffe via Python-list wrote:
> Consider this scenario (which I ran into in real life):
>     I want to open a text file and do a lot of processing on the lines
> of that file.
>     If the file does not exist I want to take appropriate action, e.g.
> print an error message and abort the program.
> I might write it like this:
>
> try:
>     with open(FileName) as f:
>         for ln in f:
>             print("I do a lot of processing here")
>             # Many lines of code here .....
> except FileNotFoundError:
>     print(f"File {FileName} not found")
>     sys.exit()
>
> but this violates the principle that a "try" suite should be kept small,
> so that only targeted exceptions are trapped,

Yes!

> not to mention that having "try" and "except" far apart decreases
> readability.

Uh-oh!

- and there's a bit of a hang-over for old-timers who had to take care
of file-locking within the application - we try to minimise 'time'
between opening a file and closing it (etc)!

As it seems the file is opened to read. Less relevant in this case, but
habits and styles of coding matter...

> Or I might write it like this:
>
> try:
>     f = open(FileName) as f:
>     FileLines = f.readlines()
> except FileNotFoundError:
>     print(f"File {FileName} not found")
>     sys.exit()
> # I forgot to put "f.close()" here -:)
> for ln in File Lines:
>         print("I do a lot of processing here")
>         # Many lines of code here .....
>
> but this loses the benefits of using "open" as a context manager,
> and would also be unacceptable if the file was too large to read into
> memory.

So, now there are two concerns:
1 FileNotFoundError, and
2 gradual processing to avoid memory-full

- added to remembering to close the file.

> Really I would like to write something like
>
> try:
>     with open(FileName) as f:
> except FileNotFoundError:
>     print(f"File {FileName} not found")
>     sys.exit()
> else: # or "finally:"
>         for ln in f:
>             print("I do a lot of processing here")
>             # Many lines of code here .....
>
> but this of course does not work because by the time we get to "for ln
> in f:" the file has been closed so we get
> ValueError: I/O operation on closed file
>
> I could modify the last attempt to open the file twice, which would
> work, but seems like a kludge (subject to race condition, inefficient).
>
> Is there a better / more Pythonic solution?

Idea 1: invert the exception handling and the context-manager by writing
a custom context-manager class which handles FileNotFoundError
internally. Thus, calling-code becomes:

with...
for...
processing

Idea 2: incorporate idea of encapsulating "processing" into a
(well-named) function to shorten the number of lines-of-code inside the
with-suite.

Idea 3: consider using a generator to 'produce' lines of data
one-at-a-time. Remember that whilst context-managers and generators are
distinct concepts within Python, they are quite similar in many ways.
So, a custom generator could work like a context-manager or 'wrap' a
context-manager per Idea 1.

Building a custom-class (Idea 1 or Idea 3) enables the components to be
kept together, per the ideal. It keeps the try-except components close
and easy to relate. It is Pythonic (in the OOP style).

--
Regards,
=dn

Subject	Author
Re: Best use of "open" context manager	dn