rotten news relay - comp.lang.python.announce - [Python-announce] ANN: First beta of Python-Blosc2 3.0.0

Subject: [Python-announce] ANN: First beta of Python-Blosc2 3.0.0
From: Francesc Alted
Newsgroups: comp.lang.python.announce
Date: Fri, 21 Jun 2024 12:16 UTC

Path: eternal-september.org!news.eternal-september.org!feeder3.eternal-september.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: faltet@gmail.com (Francesc Alted)
Newsgroups: comp.lang.python.announce
Subject: [Python-announce] ANN: First beta of Python-Blosc2 3.0.0
Date: Fri, 21 Jun 2024 14:16:55 +0200
Lines: 89
Approved: python-announce-list@python.org
Message-ID: <CAFrp1vpt+YtvZ4vvbP++9who2fy+Tnp9pR09Y0Rv8O_ahMka0g@mail.gmail.com>
Reply-To: python-list@python.org, faltet@gmail.com
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de dERAfKLL/9rO/6nbpZD9aA4Q1XRSKipjkjNbTArH7DCA==
Cancel-Lock: sha1:ZFuB547/f59ebkxfelasuNc3wH0= sha256:3vK6c5srHUc55PcKyU3pwVnXGnBsehBz05xA9zWhoA8=
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=Jqbwl/v8;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'generated': 0.03; 'url-
ip:140.82/16': 0.03; 'pip': 0.04; 'network.': 0.05; 'pypi': 0.05;
'repository': 0.05; 'arrays': 0.07; 'compressed': 0.09;
'computing': 0.09; 'info,': 0.09; 'meant': 0.09; 'memory.': 0.09;
'numpy': 0.09; 'url:reference': 0.09; 'subject:Python': 0.12;
'url:github': 0.14; 'announce': 0.15; 'memory': 0.15; 'url-
ip:140/8': 0.15; 'backward': 0.16; 'beta': 0.16;
'cc:addr:googlegroups.com': 0.16; 'compression': 0.16;
'computations': 0.16; 'datasets.': 0.16; 'engine,': 0.16;
'expressions': 0.16; 'instances,': 0.16; 'instances.': 0.16;
'jupyter': 0.16; 'leverage': 0.16; 'notebook,': 0.16; 'received
:mail-pj1-x1033.google.com': 0.16; 'subject:ANN': 0.16;
'to:name:discussion of numerical python': 0.16; 'url:doc': 0.16;
'url:ipynb': 0.16; 'url:releases': 0.16; 'python': 0.16; 'api':
0.17; 'github': 0.17; 'it?': 0.19; 'cc:addr:python.org': 0.20;
'advanced': 0.22; 'to:addr:numpy-discussion': 0.22; 'version':
0.23; 'install': 0.23; 'cc:2**1': 0.23; 'feedback': 0.23;
'tutorials': 0.23; 'run': 0.23; 'skip:- 10': 0.25; 'examples':
0.25; 'binary': 0.26; 'notes': 0.26; 'perform': 0.26; 'default':
0.31; 'community': 0.31; '(as': 0.32; 'aims': 0.32; 'feed': 0.32;
'message-id:@mail.gmail.com': 0.32; 'but': 0.32; 'there': 0.33;
'release': 0.34; 'package': 0.34; 'received:google.com': 0.34;
'final': 0.35; 'from:addr:gmail.com': 0.35; 'also,': 0.36;
'built': 0.36; 'functions': 0.36; 'using': 0.37; 'hard': 0.37;
'means': 0.38; 'read': 0.38; 'added': 0.39; 'use': 0.39;
'distributed': 0.39; 'evaluation': 0.39; 'data.': 0.40; 'both':
0.40; 'want': 0.40; 'including': 0.60; 'provide': 0.60; 'team':
0.60; 'services': 0.61; 'follow': 0.62; 'data,': 0.63; 'true':
0.63; 'skip:b 10': 0.63; 'providing': 0.63; 'our': 0.64; 'full':
0.64; 'in:': 0.64; 'your': 0.64; 'look': 0.65; 'now,': 0.67;
'pleased': 0.67; 'addition,': 0.70; 'performance': 0.71;
'operate': 0.75; 'sources': 0.75; 'features': 0.75; 'details.':
0.76; 'site:': 0.76; 'reply-to:no real name:2**0': 0.78; 'major':
0.78; 'header:Reply-To:1': 0.79; 'powerful': 0.84; 'reply-
to:addr:gmail.com': 0.84; '-----------': 0.84; 'license,': 0.84;
'metadata,': 0.84; 'newest': 0.84; 'procedures': 0.84;
'subject:First': 0.84; 'wheels': 0.84; 'opposite': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1718972227; x=1719577027; darn=python.org;
h=cc:to:subject:message-id:date:from:reply-to:mime-version:from:to:cc
:subject:date:message-id:reply-to;
bh=Q/r/YjpLhp5wtL5OAuy1bPFUQVnFJV/8xIp6d1NRzLA=;
b=Jqbwl/v8d8xaJ2xSu33IcoBHv+DsLWmOMZo99YQTG6WmsrPjeeP7ibxNIrkwW4ud3l
MaN+8cDNQK2BhO4IDDsm3BxxOC49A4HlhunL+w9e/K/NXm0pZoDr8VsAYUEr636fxs7o
/tXEZi8YnQmNDPC+ucomXW7zUoGuCnkIFs6ay2H+LPbOyvRuY9EpTYBfYtYCq6X7nBnv
6KmCnt3P1IWqe55vQ0KLR55SqRMAiSNaxYmU9ribol6FrK4gHhpyqefTxxvtbE3W00l8
ZfCwRD38IB8S2CCBy4qANrKvEDpjEH7pZkkIs+qlkogA8M1/rfj5PNoRY/lBvPebpecs
TgOg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1718972227; x=1719577027;
h=cc:to:subject:message-id:date:from:reply-to:mime-version
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=Q/r/YjpLhp5wtL5OAuy1bPFUQVnFJV/8xIp6d1NRzLA=;
b=AKS3/RuM0Y+A3X/M/vBkGbtiD/4OS95zCVoPfv1HcKm+1HfT835SC+3+yKBrP+92VI
vd5XOFX4Nmuy1omraXK2dh0dhH/wPcbtb8FveWDdz22j1HUHfcYTqNwt5NTrkZZHGPSE
09PJVabyozyAI1Q7v/UGybRwZkJhe3e6/1pflVm6mhU80LImEIyTgxGsakF9zOqM4FtZ
W0p0uzia2qx2m0ziQTbPAUxNMqTtejL5WkTjuZef4BEBXd1+86qYqycRLBtZGowf5Pjz
fuuJO4jaPuhLlZgA/AndWeu/nq31UTVEaGLjjzGH5aOF8B66GV/w/HeF3kj4VpVGMR2T
xVrQ==
X-Forwarded-Encrypted: i=1; AJvYcCWalZmb1P38aFnHPLqQGQW2m3yu/NyNaMAq/DOGxT9ANr6TLuAf/LwUgnQg05N9UtqTV9eRspne+pxwrey2quU9WvXXXYK3R+OKfsrhMKpT
X-Gm-Message-State: AOJu0YzSPR6XkgrEqSYU6kgG7Pvh6J+jh/CrMrqjxPjmOpt0HJydLOy6
3oqUqKj5CprYNmLXHnr47/9G4j2HxOZjop4+4OtDSp/MwSJm7FqQA8zvGjRH06KSOPNvhkjAfKM
mEG8fphwldBHZwns9IcNRkrG7RLUbwRZd
X-Google-Smtp-Source: AGHT+IFQFv/CeaHqZaSNv5p05Q+5kNKrghG3fuztkmMuAhaBTiKMBrbJEqpTU+TQ/SIOlNOsDFTQCrVxsBY1cEZPMZM=
X-Received: by 2002:a17:90a:bf13:b0:2bf:8ce5:dc51 with SMTP id
98e67ed59e1d1-2c7b5d5c1b1mr8476085a91.35.1718972227055; Fri, 21 Jun 2024
05:17:07 -0700 (PDT)
X-MailFrom: faltet@gmail.com
X-Mailman-Rule-Hits: emergency
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved
Message-ID-Hash: UQFNZRKR73XEYMKXQZGK3A3QK2LZWYGC
X-Message-ID-Hash: UQFNZRKR73XEYMKXQZGK3A3QK2LZWYGC
X-Mailman-Approved-At: Fri, 21 Jun 2024 09:05:54 -0400
X-Mailman-Version: 3.3.10b1
Precedence: list
List-Id: Announcement-only list for the Python programming language <python-announce-list.python.org>
Archived-At: <https://mail.python.org/archives/list/python-announce-list@python.org/message/UQFNZRKR73XEYMKXQZGK3A3QK2LZWYGC/>
List-Archive: <https://mail.python.org/archives/list/python-announce-list@python.org/>
List-Help: <mailto:python-announce-list-request@python.org?subject=help>
List-Owner: <mailto:python-announce-list-owner@python.org>
List-Post: <mailto:python-announce-list@python.org>
List-Subscribe: <mailto:python-announce-list-join@python.org>
List-Unsubscribe: <mailto:python-announce-list-leave@python.org>

View all headers

Hi,

The Blosc development team is pleased to announce the first beta release of
Python-Blosc2 3.0.0. We have been working hard to provide a new evaluation
engine (based on numexpr) for NDArray instances, and we would like to get
feedback from the community before the final release.

Now, you can evaluate expressions like `a + sin(b) + 1` where `a` and `b`
are NDArray instances. This is a powerful feature that allows for
efficient computations on compressed data, and supports advanced features
like reductions, filters, user-defined functions and broadcasting (still
in beta). See this
`example <https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/eval_expr.py>`_.

Also, we have added support for memory mapping in `SChunk` and
`NDArray` instances.
This allows to map super-chunks stored in disk and access them as if
they were in
memory. When combined with the evaluation engine, this feature allows for very
good performance when working with large datasets. See this
`benchmark <https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/lazyarray-expr.ipynb>`_
(as it is a Jupyter notebook, you can easily run it in your own computer).

Last, but not least, we are using NumPy 2.x as the default for testing
procedures
and builds. This means that our wheels are built against NumPy 2, so
in case you want
to use NumPy 1.x, you will need to use NumPy 1.23.0 or later.

We are providing binary wheels that you can easily install from PyPI with::
pip install blosc2==3.0.0b1

For more info, you can have a look at the release notes in:

https://github.com/Blosc/python-blosc2/releases

More docs and examples are available in the documentation site:

https://www.blosc.org/python-blosc2/python-blosc2.html

What is it?
-----------

`C-Blosc2 <https://github.com/Blosc/c-blosc2>`_ is the new major version of
`C-Blosc <https://github.com/Blosc/c-blosc>`_, and is backward compatible with
both the C-Blosc1 API and its in-memory format. Python-Blosc2 is a Python
package that wraps C-Blosc2, the newest version of the Blosc compressor.

Starting with version 3.0.0, Python-Blosc2 is including a powerful computing
engine that can operate on compressed data that can be either in-memory,
on-disk or on the network. This engine also supports advanced features like
reductions, filters, user-defined functions and broadcasting.

You can read some of our tutorials on how to perform advanced computations at:

* https://github.com/Blosc/python-blosc2/blob/main/doc/getting_started/tutorials/03.lazyarray-expressions.ipynb
* https://github.com/Blosc/python-blosc2/blob/main/doc/getting_started/tutorials/03.lazyarray-udf.ipynb

In addition, Python-Blosc2 aims to leverage the full C-Blosc2 functionality to
support super-chunks
(`SChunk <https://www.blosc.org/python-blosc2/reference/schunk_api.html>`_),
multi-dimensional arrays
(`NDArray <https://www.blosc.org/python-blosc2/reference/ndarray_api.html>`_),
metadata, serialization and other bells and whistles introduced in C-Blosc2.

**Note:** Blosc2 is meant to be backward compatible with Blosc(1) data.
That means that it can read data generated with Blosc, but the opposite
is not true (i.e. there is no *forward* compatibility).

Sources repository
------------------

The sources and documentation are managed through github services at:

https://github.com/Blosc/python-blosc2

Python-Blosc2 is distributed using the BSD license, see
https://github.com/Blosc/python-blosc2/blob/main/LICENSE.txt
for details.

Mastodon feed
-------------

Please follow https://fosstodon.org/@Blosc2 to get informed about the latest
developments.

- Blosc Development Team
Make compression better

Subject	Author
[Python-announce] ANN: First beta of Python-Blosc2 3.0.0	Francesc Alted

Q: Why do WASPs play golf ? A: So they can dress like pimps.

comp / comp.lang.python.announce / [Python-announce] ANN: First beta of Python-Blosc2 3.0.0