[PATCH Added hdf5-1.12-parallel-openmpi] * gnu/packages/maths.scm (hdf5-1.12-parallel-openmpi): New package based on HDF5 1.12.0

---

gnu/packages/maths.scm | 34 +++++++++++++++++++++++++++++++++-

1 file changed, 33 insertions(+), 1 deletion(-)

This needs to be a separate package since (as with all HDF5 major releases) it's not 100% API compatible with HDF5 1.10. It would break other packages, e.g., `petsc-lite`

Hi,

On Sun, 28 Feb 2021 at 07:33, Gerd Heber <gerd.heber@gmail.com> wrote:

Why duplicates? Something like:

seems simpler.

All the best,

simon

Hi,

(Please CC the bug number.)

On Tue, 2 Mar 2021 at 01:32, Gerd Heber <gerd.heber@gmail.com> wrote:

The usual way is to send a v2 which is the patch to apply to the Git

tree (and *not* a patch to apply to a patch to apply to the tree).

However, personally I am not in favor to add to Guix proper too much

versions of variant packages; but that's just my humble opinion.

Instead, it could be submitted to specific channels, for instance

https://gitlab.inria.fr/guix-hpc/guix-hpc or any other listed in

Guix-HPC.

Cheers,

simon

zimoun <zimon.toutoune@gmail.com> skribis:

In this particular case, I think the proposed variant is fine; it’s

consistent with what we’re already doing with hdf5.

Ludo’.

Ludovic, how are you? Thanks for taking the time to reply. This

conversation makes

me wonder what the Guix model for packages such as HDF5 might be.

On the one hand, there should be defaults for typical, i.e., non-HPC users, and

they probably belong into gnu/packages/maths.scm. Once you add MPI to the

mix, things get a little more complicated, and channels, such as

Guix-HPC might be

the better place. (?) Should we run our own channel?

In HDF5's case, we also use versions in a peculiar way, where the expectation is

that 1.8.x, 1.10.x, and 1.12.x, etc. version are not compatible for

end-user applications,

mostly because of API changes.

What's your recommendation? Maybe (hdf5-1.6?), hdf5-1.8, hdf-1.10, and hdf5-1.12

belong into maths.scm, plus the thread-safe builds, but not

hdf5-parallel-openmpi?

I tried to build hdf5-1.8.22-parallel-openmpi, but some of the (MPI)

atomicity tests fail

with the OpenMPI version used in the current hdf5-parallel-openmpi.

And then there is MPICH...

I would also like to see HDFView as a Guix package. We have a Spack package, but

the Java story on Guix (I don't blame you ;-) is a little confusing.

I'm sold on the merits of Guix and you are doing fantastic work.

What's your recommendation, and what can we (The HDF Group) do to help?

Best, G.

On Tue, Mar 2, 2021 at 1:39 PM Ludovic Courtès <ludo@gnu.org> wrote:

Hi Gerd,

On Wed, 03 Mar 2021 at 08:10, Gerd Heber <gerd.heber@gmail.com> wrote:

From my understanding, hdf5 at versions 1.8 and 1.10 are used by other

packages. When those very packages will not use at one of these

versions, I guess the very version will be removed.

hdf5-parallel-openmpi is used by petsc-openmpi for instance. And this

hdf5 variant is built with version 1.10. Since there is no package that

requires hdf5-parallel-openmpi at another version than 1.10, I do not

see the point to include it in Guix proper. Especially when the custom

variant is straightforward to locally create and buildable on a

reasonable amount of time.

However, these words are not totally acceptable. :-) If I take the shoes

of a regular scientist, then they only wants the package at hand and not

necessary RTFM how to do package transformations.

Yes, and we could imagine different versions of openmpi. And then

compiled with different compiler versions, etc…

…and the matrix of combinations is exploding. ;-)

One issue with the channel is to provide substitutes for that channel.

For example, the channel guix-science uses TravisCI to build the package

from GitHub.

That’s said, the cons about channels–and so the pros about include the

hdf5 variant in Guix proper–is to keep consistency and detect breakage:

Guix proper updates a package that becomes incompatible with one the

variant living a channel. All in Guix proper, then Guix CI will detect

it. Some dependencies in Guix proper and hdf variant in a channel, then

the channel CI probably not since nothing changed (from the channel

side) and so nothing triggered a build. I do not know.

Well, let stay pragmatic. :-)

It would be really cool!

Thanks the HDF Group for their interest on Guix.

Where the package definition ends (channel vs Guix proper) is one thing,

maybe a start should to have these hdf5 variant definitions. Then from

a pragmatic point of view, depending on these definitions (number,

length, etc.), they could ends in (gnu packages maths) or maybe its own

module (gnu packages hdf) or maybe in a channel. WDYT?

All the best,

simon

Hi Gerd,

Gerd Heber <gerd.heber@gmail.com> skribis:

As you write, the goal for packages in Guix proper should be to have

“good defaults”, and possibly useful variants, as is currently the case

with HDF5.

We won’t keep all versions and variants in Guix proper because that’d be

too much of a maintenance burden. For “unusual” variants, I recommend

maintaining your own channel. (This is something we did at Inria for

example with an Open MPI variant that includes the MPI1 compatibility

layer.)

Additionally, you can use things like ‘--with-input=openmpi=mpich’. We

may eventually get support for “parameterized packages”, which will

allow users to choose between HDF5 variants from the command line:

https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html

For Guix proper, I think we should keep the number of HDF5 variants

stable. We should remove old versions once they’re no longer needed by

any in-tree package.

That’s a different story :-) but please don’t hesitate to share your

experience, frustration, and questions regarding Java on the mailing

lists, I’m sure you’ll get some guidance.

It’s great to get feedback from upstream; you can certainly help us make

wise(r) decisions regarding HDF5 packaging, in particular in deciding

which variants and versions make sense, what defaults are reasonable,

and how well they work in an HPC and non-HPC setting.

WDYT? What changes would you like to see?

Thanks for your interest!

Ludo’.

Hello,

thanks for your patch, that I only see just now (teams were not yet a thing

in 2021...). This version of hdf5 has been retired, so I am closing this issue.

It might be useful to add a parallel package to hdf5@1.14 instead.

Andreas