Rocksolid Light

News from da outaworlds

mail  files  register  groups  login

Message-ID:  

Patch griefs with proverbs. -- William Shakespeare, "Much Ado About Nothing"

comp / comp.lang.python / Re: PyGILState_Ensure() deadlocks, why?

SubjectAuthor
o Re: PyGILState_Ensure() deadlocks, why?MRAB

1
Subject: Re: PyGILState_Ensure() deadlocks, why?
From: MRAB
Newsgroups: comp.lang.python
Date: Sun, 7 Jul 2024 22:44 UTC
References: 1 2 3
Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: python@mrabarnett.plus.com (MRAB)
Newsgroups: comp.lang.python
Subject: Re: PyGILState_Ensure() deadlocks, why?
Date: Sun, 7 Jul 2024 23:44:01 +0100
Lines: 144
Message-ID: <mailman.16.1720392430.2981.python-list@python.org>
References: <y-6AMIAbWkvLII_tfcbRs5NtbKgQRos3XlZ83qbGCwuq7RSQJBGniGJEn4Nzwr_iVQfJOdvLVzOycTR0V-F-pDKt2rNPj3cHmFVY06vNQnI=@protonmail.ch>
<DGC1c0YSrMcV0n6rNNeplHySXzBWUfTUwJbBVPgOocO1evufk5MKe8QWWEzO6Zj5-_0VrJJbK3OpB5-vrvMdDRKITiZrvq0RMY69IND05u4=@protonmail.ch>
<fd0e2e55-34ef-4149-84c8-a2a26596b6b0@mrabarnett.plus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de wPvXoV93ikOiJnI6Ri3D2gUEvITwH/W0Tf+KjhLlVyQw==
Cancel-Lock: sha1:0MvFx5OIYKXJAlC+e46ou5S3bU8= sha256:Rxz8uJP5GmK3ntWx/RHRHZMtLkydquiNiEVpUeGpv3Y=
Return-Path: <python@mrabarnett.plus.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=plus.com header.i=@plus.com header.b=sH8cLNAX;
dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'thread': 0.05;
'from:addr:python': 0.09; 'int': 0.09; 'linux': 0.09;
'received:192.168.1.64': 0.09; 'skip:` 10': 0.09; 'subject:skip:P
10': 0.09; 'threads': 0.09; 'ubuntu': 0.09; '#include': 0.16;
'2024': 0.16; 'c++': 0.16; 'forever.': 0.16;
'from:addr:mrabarnett.plus.com': 0.16; 'from:name:mrab': 0.16;
'gil': 0.16; 'gnu/linux': 0.16; 'initialize': 0.16; 'interpreter':
0.16; 'long-running': 0.16; 'main()': 0.16; 'message-
id:@mrabarnett.plus.com': 0.16; 'owns': 0.16; 'received:84.93':
0.16; 'received:84.93.230': 0.16; 'received:plus.net': 0.16;
'subject:() ': 0.16; 'thread.': 0.16; 'vector': 0.16; 'void':
0.16; 'wrote:': 0.16; 'python': 0.16; 'api': 0.17; 'larger': 0.17;
'code.': 0.17; 'bug': 0.19; 'to:addr:python-list': 0.20; 'code':
0.23; 'skip:p 30': 0.23; 'task': 0.26; 'header:User-Agent:1':
0.30; 'point,': 0.32; 'python-list': 0.32; 'received:192.168.1':
0.32; 'but': 0.32; 'there': 0.33; 'release': 0.34; 'header:In-
Reply-To:1': 0.34; 'following': 0.35; 'main': 0.37; "it's": 0.37;
'received:192.168': 0.37; 'still': 0.40; 'try': 0.40; "there's":
0.61; 'skip:t 40': 0.64; 'owned': 0.69; 'waiting': 0.73;
'acquire': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plus.com; s=042019;
t=1720392242; bh=Yvk/AoHSruGKULhdJxkoC53o1IPpI460JoXcUIQPi44=;
h=Date:Subject:To:References:From:In-Reply-To;
b=sH8cLNAX+nIniHEKeMGfXk6N5EkXoOIx514YIashyewzL8+mEM2rCdMRlcS3uuhEt
z2oO0NFPYGCzLcqLhr87AFpZoKiixJVuNvXg7RrLZ6A4YJyfKVeca7Hn9bBVKsiYeb
35QJ4IOPpsTPM8qzdrHe45D7Vn2MsWLP+szP3SUtAhllErzJtcD2aEJh7pA/HR0jAI
fD3hxBlYJaIFKHuSNBl6gNqYPJ+BtBiOD+eEk4hQFizKOkPHMH1Sm/N/hKY+qS2MVn
OYCwApALvMZVFGwmN7g66pGwkv1ack/JmRLrDxu1vRC0moI/V0N7nlnQ79tfE5TyN9
bu9B/hT9F6r1Q==
X-Clacks-Overhead: "GNU Terry Pratchett"
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.4 cv=AfoSHGXG c=1 sm=1 tr=0 ts=668b1a32
a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17
a=IkcTkHD0fZMA:10 a=ebcnKlX9jUEQYcSTvwIA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10
X-AUTH: mrabarnett@:2500
User-Agent: Mozilla Thunderbird
Content-Language: en-GB
In-Reply-To: <DGC1c0YSrMcV0n6rNNeplHySXzBWUfTUwJbBVPgOocO1evufk5MKe8QWWEzO6Zj5-_0VrJJbK3OpB5-vrvMdDRKITiZrvq0RMY69IND05u4=@protonmail.ch>
X-CMAE-Envelope: MS4xfCeUqwjmHHo182luOK4rR4KLPeluWtdX8XSawwR8wVurd3LhG5cGWjNL13cSJiZh/VZekl8pDt5LpWQGDl0QKn7DZFnqK394pl9RioqVkjFU90TXiJIu
0lSAnnavrIZBRwkp4H5+SZZcFHIVH6/4XyQH+01iTKbJdb4uVMoVwr/nN/8Ei36iOwiLXN7okL+hkkcPlP1gcIJD0J1HYrciSOo=
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <fd0e2e55-34ef-4149-84c8-a2a26596b6b0@mrabarnett.plus.com>
X-Mailman-Original-References: <y-6AMIAbWkvLII_tfcbRs5NtbKgQRos3XlZ83qbGCwuq7RSQJBGniGJEn4Nzwr_iVQfJOdvLVzOycTR0V-F-pDKt2rNPj3cHmFVY06vNQnI=@protonmail.ch>
<DGC1c0YSrMcV0n6rNNeplHySXzBWUfTUwJbBVPgOocO1evufk5MKe8QWWEzO6Zj5-_0VrJJbK3OpB5-vrvMdDRKITiZrvq0RMY69IND05u4=@protonmail.ch>
View all headers

On 2024-07-07 20:40, Tomas Ukkonen via Python-list wrote:
> Hi
>
> There was a bug in the example code. I fixed it and it STILL deadlocks (my larger software project deadlocks when I call python from C++).
>
> Updated code:
>
> /* * This code deadlocks on python3-dev 3.12.3 (ubuntu 24.04 lts)
>  *
>  * g++ python_thread_test.cpp `python3-config --cflags --libs --embed`
>  * ./a.out
>  *
>  * uname:
>  * Linux softice 6.8.0-36-generic SMP PREEMPT_DYNAMIC x86_64 GNU/Linux
>  */
>
> #include <Python.h>
> #include <thread>
> #include <vector>
> #include <iostream>
>
>
> void thread_function() {
>     // Ensure this thread has the GIL
>     PyGILState_STATE gstate = PyGILState_Ensure();
>
>     // Execute some Python code
>     PyRun_SimpleString("print('Hello from std::thread!')");
>
>     // Release the GIL
>     PyGILState_Release(gstate);
> }
>
> int main() {
>     // Initialize the Python Interpreter
>     Py_Initialize();
>
>     // Create a vector of threads
>     std::vector<std::thread> threads;
>
>     // Launch threads
>     for (int i = 0; i < 5; ++i) {
>         threads.push_back(std::thread(thread_function));
>     }
>
>     // Join threads
>     for (auto& t : threads) {
>         t.join();
>     }
>
>     // Finalize the Python Interpreter
>     Py_Finalize();
>
>     return 0;
> }
> sunnuntaina 7. heinäkuuta 2024 klo 10:24 ip, Tomas Ukkonen <tomas.ukkonen@protonmail.ch> kirjoitti:
>
>> Hello
>> Is this python c api bug? The following C++ code (g++) deadlocks on Ubuntu Linux.
>>
>
>> /* * This code deadlocks on python3-dev 3.12.3 (ubuntu 24.04 lts)
>>  *
>>  * g++ python_thread_test.cpp `python3-config --cflags --libs --embed`
>>  * ./a.out
>>  *
>>  * uname:
>>  * Linux softice 6.8.0-36-generic SMP PREEMPT_DYNAMIC x86_64 GNU/Linux
>>  */
>>
>
>> #include <Python.h>
>> #include <thread>
>> #include <vector>
>> #include <iostream>
>>
>
>> void perform_long_operation() {
>>     // Simulate a long-running task
>>     std::this_thread::sleep_for(std::chrono::seconds(5));
>> }
>>
>
>> void thread_function() {
>>     // Ensure this thread has the GIL
>>     PyGILState_STATE gstate = PyGILState_Ensure();
>>
>
>>     // Execute some Python code
>>     PyRun_SimpleString("print('Hello from std::thread!')");
>>
>
>>     // Release the GIL for long operation
>>     Py_BEGIN_ALLOW_THREADS
>>     perform_long_operation();
>>     Py_END_ALLOW_THREADS
>>
>
>>     // Re-acquire the GIL and execute more Python code
>>     gstate = PyGILState_Ensure();
>>     PyRun_SimpleString("print('Thread operation completed!')");
>>
>
>>     // Release the GIL
>>     PyGILState_Release(gstate);
>> }
>>
>
>> int main() {
>>     // Initialize the Python Interpreter
>>     Py_Initialize();
>>
At this point, there's only one thread (the main thread) and it owns the
GIL.
>
>>     // Create a vector of threads
>>     std::vector<std::thread> threads;
>>
>
>>     // Launch threads
>>     for (int i = 0; i < 5; ++i) {
>>         threads.push_back(std::thread(thread_function));
>>     }
>>
The threads will each try to acquire and release the GIL, but it's still
owned by the main thread.
>
>>     // Join threads
>>     for (auto& t : threads) {
>>         t.join();
>>     }
The main thread is waiting for the sub-threads to finish, and the
threads waiting for the GIL, but the main thread still owns the GIL, so
they'll be waiting forever. Deadlock.
>>
>
>>     // Finalize the Python Interpreter
>>     Py_Finalize();
>>
>
>>     return 0;
>> }
>>

1

rocksolid light 0.9.8
clearnet tor