rotten news relay - comp.lang.python - Python crash together with threads

Subject: Python crash together with threads
From: Guenther Sohler
Newsgroups: comp.lang.python
Date: Wed, 2 Oct 2024 13:26 UTC
References: 1

Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: guenther.sohler@gmail.com (Guenther Sohler)
Newsgroups: comp.lang.python
Subject: Python crash together with threads
Date: Wed, 2 Oct 2024 15:26:47 +0200
Lines: 53
Message-ID: <mailman.26.1727875622.3018.python-list@python.org>
References: <CAASeUHoPO8kd0j7Uwn2T6R52W+7en8Q7DQDbRa1_6pQurRc-Dg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Cancel-Lock: sha1:1wAgTV7s6FJbaE92TwKgoRgBeO4= sha256:gUvVL0rV6a+Q/Yb6FCmICyng4ogZ3wD72euZKc30fAo=
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=k5FCEKnl;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.065
X-Spam-Evidence: '*H*': 0.88; '*S*': 0.01; 'thread': 0.05;
'architecture': 0.07; 'forced': 0.09; 'skip:_ 20': 0.09;
'skip:\xc2 20': 0.09; 'subject:Python': 0.12; '#endif': 0.16;
'clues': 0.16; 'mitigate': 0.16; 'permanently.': 0.16;
'scenario,': 0.16; 'skip:. 40': 0.16; 'static': 0.16; 'thread.':
0.16; 'void': 0.16; 'python': 0.16; 'to:addr:python-list': 0.20;
'(and': 0.25; 'suspect': 0.26; 'function': 0.27; 'skip:_ 40':
0.31; 'url-ip:188/8': 0.31; 'program': 0.31; 'here,': 0.32;
'message-id:@mail.gmail.com': 0.32; 'received:google.com': 0.34;
'fine': 0.35; 'from:addr:gmail.com': 0.35; 'cases': 0.36;
'possibly': 0.36; 'main': 0.37; 'using': 0.37; 'skip:_ 30': 0.37;
'quite': 0.39; 'happen': 0.40; 'inline': 0.61; 'above': 0.62;
'skip:\xc2 10': 0.62; 'here': 0.62; 'skip:o 20': 0.63;
'dedicated': 0.65; 'parts': 0.65; 'right': 0.68; 'skip:/ 10':
0.69; '1st': 0.78; 'happens': 0.84; 'flows': 0.84; 'pointer':
0.84; 'whats': 0.84; 'skip:d 30': 0.86; 'newly': 0.93; 'skip:q
20': 0.95
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1727875619; x=1728480419; darn=python.org;
h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
:date:message-id:reply-to;
bh=Jmeb56UgKIGHlRLgzMySRWJAKgOe7GtoMGSN1DqPm/s=;
b=k5FCEKnlnNBTa3MxhNkNWIQK92JzHrYHZ6wfOVcw8sWCpO/Zxb/gPeTYhhOdt5g3u8
oJf3YBclEEweSlFzQXntVZOtum0PagCxySV/TiHtQAbJ8maPYb0EOhFvNl2it/g6mzFm
sFXNgXA2t5A45eIvAUdKQk8arTKzugdWfPi+z5929IdyOSYCxkJ+wb5WGKJFi7nPu+dh
rprHGRu/bMPzkYV2MvMuh+359hbyASGJuKVI4H2vb8tPl7EYNhIVArEHFvHKQKQ47SGG
+vtD0KRDEzUtvyUJTkJX5a2XdaQU1Upb/sKl7zUQBZ+RIW8RakPwBI0GMv104hTAm7JI
wUyA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1727875619; x=1728480419;
h=to:subject:message-id:date:from:mime-version:x-gm-message-state
:from:to:cc:subject:date:message-id:reply-to;
bh=Jmeb56UgKIGHlRLgzMySRWJAKgOe7GtoMGSN1DqPm/s=;
b=Bnx5BEB73FXrpdn9DcnGRJmhdhQPmqhfUmwvhdG4o+BQFEEuaP5DnRZpGsIUtPQ+yp
vr7AVXO1Vcrq7LJZ2yrZNIas1dOS+mrrSgIJqQ70dqDKmHXccDOcHVWN+m7ThrMqnYB9
q6pjwsho4AaR0dszo2JzMTVOdJjqaw5IE6un5tKNyLqaryu3A86TSgSY+tdomFKoU19q
vIK4AtrpDxEUKLj8McwRH8OztCYz8tJS0Dsgmmr96+v3HYlwp2bhk7h6oYJYNwIw9rbI
MLGfMl7nqF/eUuF02gwwbxxTiMIMwMDXNRMM+ISg30YeYFRfex77wOCSIGgLWtWPMn5W
Lqaw==
X-Gm-Message-State: AOJu0YzMNT30Foc1t6xnvmf4306SVu69dRL5NcLEpuPSmAwrppJcKHgH
Rt72rJjMZrTAd6Bd7ysI9i3hm1VRnWpi1oGCuAoRAI1XcueXfPFJHVUp3taKnBftZ9OWpviVKKn
e0QExqRR4RCdWVcheXrwdk+ILfrDzsIbbZpU=
X-Google-Smtp-Source: AGHT+IF8XL7oAE6AHxAOKiK/fl2/J0+JuJseSzK+YE/K8XeEQWf0p0ZIzeGY03aEq5xcKiefzOM57ELoClM7PzF6W/E=
X-Received: by 2002:a05:6122:1791:b0:50a:d28e:5ca4 with SMTP id
71dfb90a1353d-50ad3b7f7f2mr4243202e0c.2.1727875618535; Wed, 02 Oct 2024
06:26:58 -0700 (PDT)
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAASeUHoPO8kd0j7Uwn2T6R52W+7en8Q7DQDbRa1_6pQurRc-Dg@mail.gmail.com>

View all headers

My Software project is working fine in most of the cases
(www.pythonscad.org)
however I am right now isolating a scenario, which makes it crash
permanently.

It does not happen with Python 3.11.6 (and possibly below), it happens with
3.12 and above
It does not happen when not using Threads.

However due to the architecture of the program I am forced to evaluate some
parts in main thread and some parts in a dedicated Thread. The Thread is
started with QThread(QT 5.0)
whereas I am quite sure that program flows do not overlap.

When I just execute my 1st very simple Python function inside the newly
created thread, like:

PyObject *a = PyFloat_FromDouble(3.3);

my program crashes with this Stack trace

0 0x00007f6837fe000f in _PyInterpreterState_GET () at
../Include/internal/pycore_pystate.h:179
#1 get_float_state () at Objects/floatobject.c:38
#2 PyFloat_FromDouble (fval=3.2999999999999998) at
Objects/floatobject.c:136
#3 0x00000000015a021f in python_testfunc() ()
#4 0x0000000001433301 in CGALWorker::work() ()
#5 0x0000000000457135 in CGALWorker::qt_static_metacall(QObject*,
QMetaObject::Call, int, void**) ()
#6 0x00007f68364d0f9f in void doActivate<false>(QObject*, int, void**) ()
at /lib64/libQt5Core.so.5
#7 0x00007f68362e66ee in QThread::started(QThread::QPrivateSignal) () at
/lib64/libQt5Core.so.5
#8 0x00007f68362e89c4 in QThreadPrivate::start(void*) () at
/lib64/libQt5Core.so.5
#9 0x00007f6835cae19d in start_thread () at /lib64/libc.so.6
#10 0x00007f6835d2fc60 in clone3 () at /lib64/libc.so.6

I suspect, that this is a Null pointer here
See also _PyInterpreterState_Get()
and _PyGILState_GetInterpreterStateUnsafe(). */
static inline PyInterpreterState* _PyInterpreterState_GET(void) {
PyThreadState *tstate = _PyThreadState_GET();
#ifdef Py_DEBUG
_Py_EnsureTstateNotNULL(tstate);
#endif
# <<----------- suspect state is nullpointer
return tstate->interp;
}

any clues , whats going on here, and how I can mitigate that ?

Subject: Re: Python crash together with threads
From: Louis Krupp
Newsgroups: comp.lang.python
Organization: Newshosting.com - Highest quality at a great price! www.newshosting.com
Date: Wed, 2 Oct 2024 21:06 UTC
References: 1 2

Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!panix!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer01.iad!feed-me.highwinds-media.com!news.highwinds-media.com!fx46.iad.POSTED!not-for-mail
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Python crash together with threads
Newsgroups: comp.lang.python
References: <CAASeUHoPO8kd0j7Uwn2T6R52W+7en8Q7DQDbRa1_6pQurRc-Dg@mail.gmail.com>
<mailman.26.1727875622.3018.python-list@python.org>
Content-Language: en-US
From: lkrupp@invalid.pssw.com.invalid (Louis Krupp)
In-Reply-To: <mailman.26.1727875622.3018.python-list@python.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Lines: 57
Message-ID: <%AiLO.42528$s7Ce.9174@fx46.iad>
X-Complaints-To: abuse(at)newshosting.com
NNTP-Posting-Date: Wed, 02 Oct 2024 21:06:03 UTC
Organization: Newshosting.com - Highest quality at a great price! www.newshosting.com
Date: Wed, 2 Oct 2024 15:06:03 -0600
X-Received-Bytes: 3098

View all headers

On 10/2/2024 7:26 AM, Guenther Sohler wrote:
> My Software project is working fine in most of the cases
> (www.pythonscad.org)
> however I am right now isolating a scenario, which makes it crash
> permanently.
>
> It does not happen with Python 3.11.6 (and possibly below), it happens with
> 3.12 and above
> It does not happen when not using Threads.
>
> However due to the architecture of the program I am forced to evaluate some
> parts in main thread and some parts in a dedicated Thread. The Thread is
> started with QThread(QT 5.0)
> whereas I am quite sure that program flows do not overlap.
>
> When I just execute my 1st very simple Python function inside the newly
> created thread, like:
>
> PyObject *a = PyFloat_FromDouble(3.3);
>
> my program crashes with this Stack trace
>
> 0 0x00007f6837fe000f in _PyInterpreterState_GET () at
> ./Include/internal/pycore_pystate.h:179
> #1 get_float_state () at Objects/floatobject.c:38
> #2 PyFloat_FromDouble (fval=3.2999999999999998) at
> Objects/floatobject.c:136
> #3 0x00000000015a021f in python_testfunc() ()
> #4 0x0000000001433301 in CGALWorker::work() ()
> #5 0x0000000000457135 in CGALWorker::qt_static_metacall(QObject*,
> QMetaObject::Call, int, void**) ()
> #6 0x00007f68364d0f9f in void doActivate<false>(QObject*, int, void**) ()
> at /lib64/libQt5Core.so.5
> #7 0x00007f68362e66ee in QThread::started(QThread::QPrivateSignal) () at
> /lib64/libQt5Core.so.5
> #8 0x00007f68362e89c4 in QThreadPrivate::start(void*) () at
> /lib64/libQt5Core.so.5
> #9 0x00007f6835cae19d in start_thread () at /lib64/libc.so.6
> #10 0x00007f6835d2fc60 in clone3 () at /lib64/libc.so.6
>
>
> I suspect, that this is a Null pointer here
> See also _PyInterpreterState_Get()
> and _PyGILState_GetInterpreterStateUnsafe(). */
> static inline PyInterpreterState* _PyInterpreterState_GET(void) {
> PyThreadState *tstate = _PyThreadState_GET();
> #ifdef Py_DEBUG
> _Py_EnsureTstateNotNULL(tstate);
> #endif
> # <<----------- suspect state is nullpointer
> return tstate->interp;
> }
>
> any clues , whats going on here, and how I can mitigate that ?
Can you post a small, self-contained program that demonstrates the problem?

Louis

BOFH excuse #42: spaghetti cable cause packet failure

comp / comp.lang.python / Python crash together with threads

Subject	Author
Python crash together with threads	Guenther Sohler
Re: Python crash together with threads	Louis Krupp