Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: rosuav@gmail.com (Chris Angelico)
Newsgroups: comp.lang.python
Subject: Re: Help with Streaming and Chunk Processing for Large JSON Data (60
GB) from Kenna API
Date: Tue, 1 Oct 2024 04:46:35 +1000
Lines: 20
Message-ID: <mailman.11.1727722011.3018.python-list@python.org>
References: <CA+hg4RiGjXw3am1s=zVLDpcA-VGS+cWNp_YEyzvS+j2MyDE2Cg@mail.gmail.com>
<CADrxXXmHUwsQbWqNrwzyKWLyTK0J3Hf0z8hAhGwKYoF2PwK7QA@mail.gmail.com>
<CA+hg4Rhn8iX7rp0uC=MbOi+8g73wQ4y4=uV0dU0jHdDUz3jk4w@mail.gmail.com>
<CAJQBtgk122sHzs+=MumYM1HW2DwKm1+i02bqgBKh4oUJYievCg@mail.gmail.com>
<4XHQPG4LzsznVwM@mail.python.org> <Zvrt0RJe5omaFkQq@anomaly>
<CAPTjJmqCz0UthKfs2-sd6E0Jcq23m0r-DgriywwazhWZ381wwg@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de OxbIUFbjuOU//wAtoLJCWw/XMLYTSFJtsljixp9lTxvA==
Cancel-Lock: sha1:P+MtRmTE+6taUmGKV5FWFVZsRdY= sha256:DI2W7d7smkEU0YRnhlNvHU4z9uYgL38t0hfFygue/Ow=
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=IK//9adZ;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.053
X-Spam-Evidence: '*H*': 0.90; '*S*': 0.00; 'subject:API': 0.07; 'dan':
0.09; 'infinite': 0.09; 'received:mail-lj1-x22e.google.com': 0.09;
'2024': 0.16; 'chrisa': 0.16; 'from:addr:rosuav': 0.16;
'from:name:chris angelico': 0.16; 'odd': 0.16; 'parsing': 0.16;
'wrote:': 0.16; 'subject:Help': 0.17; 'tue,': 0.19; 'to:addr
:python-list': 0.20; 'url:wiki': 0.23; 'anything': 0.25;
'examples': 0.25; 'python-list': 0.32; 'message-
id:@mail.gmail.com': 0.32; 'but': 0.32; 'subject:for': 0.33;
'there': 0.33; 'particular': 0.33; 'header:In-Reply-To:1': 0.34;
'received:google.com': 0.34; 'from:addr:gmail.com': 0.35;
'subject:from': 0.37; "it's": 0.37; 'least': 0.39; 'something':
0.40; "there's": 0.61; 'sequence': 0.69; 'url-ip:208.80/16': 0.70;
'url:wikipedia': 0.70; 'subject:Data': 0.71; 'url-ip:208/8': 0.71;
'significant': 0.78; 'left': 0.83; 'significant,': 0.84; 'subject:
\n ': 0.84; 'very,': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1727722008; x=1728326808; darn=python.org;
h=to:subject:message-id:date:from:in-reply-to:references:mime-version
:from:to:cc:subject:date:message-id:reply-to;
bh=Z2BRySl0uaX8R8ST8QwPtm/D88srEcLC82Z1GmJfvaY=;
b=IK//9adZFXbqZvTpWvR5cz12VTQRgeb6DpxtkV4/gl2L1LCn0qwkUWlHraSfGL0xCa
58NzSYGJD/dW6oMY2PzHXOptgVt+tSNU0bxrBaiWoKC0EeAs3Zh//juJ4uFt386SDyx+
nG8GEF8BP7/eaSu92v6QdOeyP1tzWoowtmwn0S32JP9GUZZoNc320e93gxY3vR2vdKEO
NsW81qocbWr1zUxdQ9cJVg8uVWFyovz+/krOfhXNh4kZ+VrWhHyd49NrBRI8IqscscjS
A47pmV9QtsM421nmFvMuZwFzDCfeNDn2rHjEuTLJZ3tw8ajGqFywOLlyre+oY3sGhxXI
wS9g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1727722008; x=1728326808;
h=to:subject:message-id:date:from:in-reply-to:references:mime-version
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=Z2BRySl0uaX8R8ST8QwPtm/D88srEcLC82Z1GmJfvaY=;
b=dBXyaJhJVtuN3hVDi8ldX+aIGxf/aNPt/0KLqk3qe2kpdWG+X6LctYIKeneRRPdvFw
UxLJeA11rFS5G1OlQNx2HMF1CfifwkG7urPHR6JMe6EsIBcz5AfnqLD8kv/uohvLGqBS
zRZRcea10+2gSfTvFbErgNqzS/Q4feTnqfvdhBDvHyhQS636XU5fD2EkVjro/86weII1
2xaX8F86ZPZa1BMICPgaljBlOo2BApZbJ/gZOAdgHxMwMogsRqZd1tq6KsfCTOqKA2Cu
UGy75WN4FbreyuQ7AQjwqX+qSry+xrUpU445jBjrLJdHvwrbHoK3gI4782BaR1FCMM2M
JKAA==
X-Gm-Message-State: AOJu0YwrL1AV7nm6y9lcTUv+4sGr5btS01TyfiRzan6Y+SGAGov9sp6x
AsQ3INR0Tl+1emQCodmlxcRi5aU8LIy78PswpX7Gzw0/8yWIJApLVpJGFp7C9H2wYa4qL4exYIp
llQuLWZna9Tt/pAdWz489GB2AB9WNmsCj
X-Google-Smtp-Source: AGHT+IF8bqcT4NKh0vscxq0nij+8JfI+STzBKYBH+mDC7rHPUdKl1jYFER1uGrqFmBWYm33YchOe24xc/EX1DNt5BQ0=
X-Received: by 2002:a2e:be24:0:b0:2fa:d4ef:f222 with SMTP id
38308e7fff4ca-2fad4eff4b4mr13195291fa.38.1727722007825; Mon, 30 Sep 2024
11:46:47 -0700 (PDT)
In-Reply-To: <Zvrt0RJe5omaFkQq@anomaly>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAPTjJmqCz0UthKfs2-sd6E0Jcq23m0r-DgriywwazhWZ381wwg@mail.gmail.com>
X-Mailman-Original-References: <CA+hg4RiGjXw3am1s=zVLDpcA-VGS+cWNp_YEyzvS+j2MyDE2Cg@mail.gmail.com>
<CADrxXXmHUwsQbWqNrwzyKWLyTK0J3Hf0z8hAhGwKYoF2PwK7QA@mail.gmail.com>
<CA+hg4Rhn8iX7rp0uC=MbOi+8g73wQ4y4=uV0dU0jHdDUz3jk4w@mail.gmail.com>
<CAJQBtgk122sHzs+=MumYM1HW2DwKm1+i02bqgBKh4oUJYievCg@mail.gmail.com>
<4XHQPG4LzsznVwM@mail.python.org> <Zvrt0RJe5omaFkQq@anomaly>