Rocksolid Light

News from da outaworlds

mail  files  register  groups  login

Message-ID:  

BOFH excuse #198: Post-it Note Sludge leaked into the monitor.

comp / comp.lang.python / Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment

SubjectAuthor
o Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environmentLeft Right

1
Subject: Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment
From: Left Right
Newsgroups: comp.lang.python
Date: Thu, 24 Oct 2024 15:17 UTC
References: 1 2
Path: eternal-september.org!news.eternal-september.org!feeder2.eternal-september.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: olegsivokon@gmail.com (Left Right)
Newsgroups: comp.lang.python
Subject: Re: shutil.rmtree() fails when used in Fedora (rpm) "mock" environment
Date: Thu, 24 Oct 2024 17:17:26 +0200
Lines: 112
Message-ID: <mailman.34.1729783646.4695.python-list@python.org>
References: <4a13731716200669342338ae409e73ca@posteo.de>
<CAJQBtg=kCr0VMGXvkBcj4U6CwatC2_G2dstO5pS56_koAXJurA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de oydS605XEEIoSraTvAxtSQirPz4NrkKxZwIQPJSyAgTg==
Cancel-Lock: sha1:ZCOBLn6C/S6rdXa1ptK6UPUvEXk= sha256:IwwzOZcKSl4Y27CpmheTaNPD2LA8bfV/OUPJtjaELR4=
Return-Path: <olegsivokon@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=W51lESh4;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'def': 0.04;
'subject:when': 0.07; 'url:mailman': 0.09; 'cc:addr:python-list':
0.09; 'environments': 0.09; 'fails': 0.09; 'fedora': 0.09;
'happens.': 0.09; 'identical': 0.09; 'like,': 0.09; 'python)':
0.09; 'skip:_ 20': 0.09; 'cc:no real name:2**0': 0.14; 'url-
ip:140/8': 0.14; 'memory': 0.15; 'url:listinfo': 0.15;
'(especially': 0.16; '(more': 0.16; 'addresses.': 0.16; 'assert':
0.16; 'cases,': 0.16; 'cleaned': 0.16; 'elements:': 0.16;
'functions.': 0.16; 'intention': 0.16; "isn't,": 0.16; 'path:':
0.16; 'received:mail-qt1-x82e.google.com': 0.16; 'reproduce':
0.16; 'subject:() ': 0.16; 'subject:Fedora': 0.16;
'subject:fails': 0.16; 'subject:used': 0.16; 'tuples': 0.16;
'wrote:': 0.16; 'problem': 0.16; 'python': 0.16; 'url-
ip:140.82.121/24': 0.17; 'url-ip:140.82/16': 0.17; 'values': 0.17;
'solve': 0.19; 'thu,': 0.19; 'cc:addr:python.org': 0.20; 'url-
ip:188.166.95.178/32': 0.20; 'url-ip:188.166.95/24': 0.20;
'issue': 0.21; 'exception': 0.22; 'i.e.': 0.22; 'maybe': 0.22;
'url:github': 0.23; 'code': 0.23; 'list,': 0.24; 'url-
ip:188.166/16': 0.24; "i'd": 0.24; 'idea': 0.24; 'actual': 0.25;
'cc:2**0': 0.25; 'seems': 0.26; "isn't": 0.27; 'else': 0.27;
'coming': 0.27; 'function': 0.27; 'output': 0.28; 'error': 0.29;
'packages': 0.31; 'module': 0.31; 'message-id:@mail.gmail.com':
0.31; 'checks': 0.32; 'christian': 0.32; 'elements': 0.32;
'from,': 0.32; 'guess': 0.32; 'performed': 0.32; 'python-list':
0.32; 'but': 0.32; "i'm": 0.33; 'there': 0.33; 'path': 0.33;
'header:In-Reply-To:1': 0.34; 'received:google.com': 0.34; 'able':
0.34; "didn't": 0.34; 'from:addr:gmail.com': 0.34; 'item': 0.35;
"we're": 0.35; 'those': 0.36; 'using': 0.37; "it's": 0.37;
'could': 0.37; 'directory': 0.37; 'file': 0.38; 'read': 0.38;
'both': 0.38; 'list': 0.39; 'thanks': 0.39; 'use': 0.39; 'hello,':
0.39; "that's": 0.39; 'this,': 0.39; 'edit': 0.39; 'hand': 0.40;
'four': 0.60; 'skip:o 10': 0.61; "there's": 0.61; 'between': 0.63;
'me.': 0.64; 'skip:t 20': 0.65; 'supply': 0.65; 'tool': 0.65;
'look': 0.65; '[1]': 0.67; 'skip:t 30': 0.67; 'operations': 0.68;
'skip:i 40': 0.68; 'skip:/ 30': 0.69; '[2]': 0.70; 'raised': 0.70;
'note:': 0.71; 'suite': 0.71; 'operate': 0.75; 'subsequent': 0.76;
'known': 0.84; 'clue': 0.84; 'stack,': 0.84; 'behind': 0.88
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1729783057; x=1730387857; darn=python.org;
h=content-transfer-encoding:cc:to:subject:message-id:date:from
:in-reply-to:references:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=+9dh5Wi8fX/daZJUM+gaVOF1i+GcEMpHBgUhHLeyVSo=;
b=W51lESh4O9c1VA2xtZK21kJtj7XRBTaBab1jPzXSqkI58ctk2/2yUyZQBKFpI5kH8I
h0Hut+T1YHoN1/15tttah0op8pxQYA4GP5yWqbUOC3tmLxkXzAmrJPqWu2Ft2914HZA/
HfRyMLGnatnFcW+PDP1SMBC73dYYn9fLx0H+PGiLq5GeTIepfE4twM+nu3+jiPblS5Vq
QZEGpS5L80WmG84HIa5eWmFksnwWlB/z+puGdetmd1j6q/kXfJvt/Py/uodPaAx7oEjw
gTW0+xIrtIbmWooiQScspWqUFz/y6S/r9/vsSNxg1gP4GIUtEJCQ40PLPGfcBVYXFfl6
Kc8g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1729783057; x=1730387857;
h=content-transfer-encoding:cc:to:subject:message-id:date:from
:in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=+9dh5Wi8fX/daZJUM+gaVOF1i+GcEMpHBgUhHLeyVSo=;
b=dmOXsuSQS6ABkg6sxFXbuCE16x0FD2qFrX+wGQ85x33MTjppKXv9JcOmL7WqbryxDR
XJ7R+txvbxsp9siQO3f2AEeR5QXoZo+3j0d4s0NchthLWz1HBk4ZVATQyzoSOg4hUiW1
YCDa33b7y1RY/+9k+03j9k5a8b+CrxzRV/nWjSiY7WgohoXIWNnReECf5V8a5uOmy0p2
GnwKOt1Ul7hjXMj0u2pb+vzl5GAz25XXSVTa1X+ifOFJ4MagwfakH6vQo9bZH6r0cLA9
/BNCodzj6CpmZvH7meZy0kOw4VGsKM5/O6u+0uJL+mH3t4Z+nmky0iDX13pcl/9ZIAfi
AXbg==
X-Gm-Message-State: AOJu0YzST6dBv55VyGLoB3qBEDa7bClQz9Yaclw4dKtFk6y+4PjhbbEp
O7V1hmW6KZjP3Q79IJZ1nMSXdmAaF6hEOVGRG/YFFa+rSZggsNvpvRePD29IM2Q72EZheHNJoJi
JQzIUvHQ1c66fMtJdHTiF/cBQj5QR+w==
X-Google-Smtp-Source: AGHT+IFARjWGYTNSUFLQfpcqtpRblCtmyQg4Gp+VubpU8C8x1g1zObUdsQ6K5osMzbgyc4DEDPD9+66SljwUt/y9M3g=
X-Received: by 2002:a05:622a:1a09:b0:460:ff33:9141 with SMTP id
d75a77b69052e-461146be568mr87720011cf.15.1729783057563; Thu, 24 Oct 2024
08:17:37 -0700 (PDT)
In-Reply-To: <4a13731716200669342338ae409e73ca@posteo.de>
X-Mailman-Approved-At: Thu, 24 Oct 2024 11:27:25 -0400
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAJQBtg=kCr0VMGXvkBcj4U6CwatC2_G2dstO5pS56_koAXJurA@mail.gmail.com>
X-Mailman-Original-References: <4a13731716200669342338ae409e73ca@posteo.de>
View all headers

>From reading the code where the exception is coming from, this is how
I interpret the intention of the author: they build a list (not sure
why they used list, when there's a stack datastructure in Python)
which they use as a stack, where the elements of the stack are
4-tuples, the important part about these tuples is that the first
element is the operation to be performed by rmtree() has to be one of
the known filesystem-related functions. The code raising the exception
checks that it's one of those kinds and if it isn't, crashes.

There is, however, a problem with testing equality (more strictly,
identity in this case) between functions. I.e. it's possible that a
function isn't identical to itself is, eg. "os" module was somehow
loaded twice. I'm not sure if that's a real possibility with how
Python works... but maybe in some cases, like, multithreaded
environments it could happen...

To investigate this, I'd edit the file with the assertion and make it
print the actual value found in os.lstat and func. My guess is that
they are both somehow "lstat", but with different memory addresses.

On Thu, Oct 24, 2024 at 4:06 PM Christian Buhtz via Python-list
<python-list@python.org> wrote:
>
> Hello,
> I am upstream maintainer of "Back In Time" [1] investigating an issue a
> distro maintainer from Fedora reported [2] to me.
>
> On one hand Fedora seems to use a tool called "mock" to build packages
> in a chroot environment.
> On the other hand the test suite of "Back In Time" does read and write
> to the real file system.
> One test fails because a temporary directory is cleaned up using
> shutil.rmtree(). Please see the output below.
>
> I am not familiar with Fedora and "mock". So I am not able to reproduce
> this on my own.
> It seems the Fedora maintainer also has no clue how to solve it or why
> it happens.
>
> Can you please have a look (especially at the line "assert func is
> os.lstat").
> Maybe you have an idea what is the intention behind this error raised by
> an "assert" statement inside "shutil.rmtree()".
>
> Thanks in advance,
> Christian Buhtz
>
> [1] -- <https://github.com/bit-team/backintime>
> [2] -- <https://github.com/bit-team/backintime/issues/1911>
>
> __________________________ General.test_ctor_defaults
> __________________________
> self = <test.test_uniquenessset.General testMethod=test_ctor_defaults>
> def test_ctor_defaults(self):
> """Default values in constructor."""
> > with TemporaryDirectory(prefix='bit.') as temp_name:
> test/test_uniquenessset.py:47:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _
> /usr/lib64/python3.13/tempfile.py:946: in __exit__
> self.cleanup()
> /usr/lib64/python3.13/tempfile.py:950: in cleanup
> self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
> /usr/lib64/python3.13/tempfile.py:930: in _rmtree
> _shutil.rmtree(name, onexc=onexc)
> /usr/lib64/python3.13/shutil.py:763: in rmtree
> _rmtree_safe_fd(stack, onexc)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _
> stack = []
> onexc = <function TemporaryDirectory._rmtree.<locals>.onexc at
> 0xffffb39bc860>
> def _rmtree_safe_fd(stack, onexc):
> # Each stack item has four elements:
> # * func: The first operation to perform: os.lstat, os.close or
> os.rmdir.
> # Walking a directory starts with an os.lstat() to detect
> symlinks; in
> # this case, func is updated before subsequent operations and
> passed to
> # onexc() if an error occurs.
> # * dirfd: Open file descriptor, or None if we're processing the
> top-level
> # directory given to rmtree() and the user didn't supply
> dir_fd.
> # * path: Path of file to operate upon. This is passed to
> onexc() if an
> # error occurs.
> # * orig_entry: os.DirEntry, or None if we're processing the
> top-level
> # directory given to rmtree(). We used the cached stat() of
> the entry to
> # save a call to os.lstat() when walking subdirectories.
> func, dirfd, path, orig_entry = stack.pop()
> name = path if orig_entry is None else orig_entry.name
> try:
> if func is os.close:
> os.close(dirfd)
> return
> if func is os.rmdir:
> os.rmdir(name, dir_fd=dirfd)
> return
>
> # Note: To guard against symlink races, we use the standard
> # lstat()/open()/fstat() trick.
> > assert func is os.lstat
> E AssertionError
> /usr/lib64/python3.13/shutil.py:663: AssertionError
>
> --
> https://mail.python.org/mailman/listinfo/python-list

1

rocksolid light 0.9.8
clearnet tor