import: hackage: Internal libraries are not filtered out of dependency list

  • Done
  • quality assurance status badge
Details
2 participants
  • Lars-Dominik Braun
  • Philip Munksgaard
Owner
unassigned
Submitted by
Philip Munksgaard
Severity
normal
P
P
Philip Munksgaard wrote on 7 Apr 2022 10:18
(address . bug-guix@gnu.org)
842828ff-b80b-4488-aa5b-5e6189e0a607@www.fastmail.com
The attoparsec package on hackage defines multiple internal libraries inside one package, named "attoparsec" and "attoparsec-internal", with the first depending on the latter. Importing attoparsec using `guix import hackage attoparsec` therefore yields the following erroneous package definition:

(define-public ghc-attoparsec
(package
(name "ghc-attoparsec")
(version "0.14.4")
(source
(origin
(method url-fetch)
(uri (hackage-uri "attoparsec" version))
(sha256
(base32 "0v4yjz4qi8bwhbyavqxlhsfb1iv07v10gxi64khmsmi4hvjpycrz"))))
(build-system haskell-build-system)
(inputs (list ghc-scientific ghc-attoparsec-internal))
(native-inputs
(list ghc-quickcheck
ghc-quickcheck-unicode
ghc-tasty
ghc-tasty-quickcheck
ghc-vector))
(arguments
`(#:cabal-revision
("1" "149ihklmwnl13mmixq6iq5gzggkgqwsqrjlg2fshqwwbvbd4nn3r")))
(synopsis "Fast combinator parsing for bytestrings and text")
(description
"This package provides a fast parser combinator library, aimed particularly at
dealing efficiently with network protocols and complicated text/binary file
formats.")
(license license:bsd-3)))

Note that `ghc-attoparsec-internal` is listed as a dependency of `ghc-attoparsec`. I believe that is incorrect, and that we should filter out the internal dependencies from `inputs`, just like we filter out `ghc-attoparsec` from the list of `native-inputs`.
L
L
Lars-Dominik Braun wrote on 22 May 2022 11:37
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760@debbugs.gnu.org)
YooEVtIztcFN2nrI@noor.fritz.box
Hello Philip,

Toggle quote (2 lines)
> The attoparsec package on hackage defines multiple internal libraries inside one package, named "attoparsec" and "attoparsec-internal", with the first depending on the latter. Importing attoparsec using `guix import hackage attoparsec` therefore yields the following erroneous package definition: […]

attached patches should fix this. I tried them with both, attoparsec and
raaz, and internal libraries appear neither in inputs nor native inputs
any more.

Cheers,
Lars
Attachment: 0001-import-cabal-Support-library-names.patch
From 1aa0732e8995c3872ddbb67c1a845d39cc8ce6f9 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <lars@6xq.net>
Date: Sun, 22 May 2022 11:20:07 +0200
Subject: [PATCH 2/2] import: hackage: Filter internal libraries from inputs
and native-inputs.

* guix/import/hackage.scm (filter-dependencies): Support multiple
OWN-NAMES.
(hackage-module->sexp): Filter OWN-NAMES from HACKAGE-DEPENDENCIES and
HACKAGE-NATIVE-DEPENDENCIES.
* tests/hackage.scm (test-cabal-internal-library-ignored): New variable.
("hackage->guix-package test internal libraries are ignored"): New testcase.
---
guix/import/hackage.scm | 17 ++++++++++-------
tests/hackage.scm | 19 +++++++++++++++++++
2 files changed, 29 insertions(+), 7 deletions(-)

Toggle diff (77 lines)
diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm
index 0d6c77e399..6e982366cf 100644
--- a/guix/import/hackage.scm
+++ b/guix/import/hackage.scm
@@ -222,12 +222,13 @@ (define (cabal-custom-setup-dependencies->names cabal)
'())))
(map cabal-dependency-name custom-setup-dependencies)))
-(define (filter-dependencies dependencies own-name)
+(define (filter-dependencies dependencies own-names)
"Filter the dependencies included with the GHC compiler from DEPENDENCIES, a
-list with the names of dependencies. OWN-NAME is the name of the Cabal
-package being processed and is used to filter references to itself."
+list with the names of dependencies. OWN-NAMES is the name of the Cabal
+package being processed and its internal libaries and is used to filter
+references to itself."
(filter (lambda (d) (not (member (string-downcase d)
- (cons own-name ghc-standard-libraries))))
+ (append own-names ghc-standard-libraries))))
dependencies))
(define* (hackage-module->sexp cabal cabal-hash
@@ -248,9 +249,11 @@ (define revision
(define source-url
(hackage-source-url name version))
+ (define own-names (cons (cabal-package-name cabal)
+ (map cabal-library-name (cabal-package-library cabal))))
+
(define hackage-dependencies
- (filter-dependencies (cabal-dependencies->names cabal)
- (cabal-package-name cabal)))
+ (filter-dependencies (cabal-dependencies->names cabal) own-names))
(define hackage-native-dependencies
(lset-difference
@@ -260,7 +263,7 @@ (define hackage-native-dependencies
(cabal-test-dependencies->names cabal)
'())
(cabal-custom-setup-dependencies->names cabal))
- (cabal-package-name cabal))
+ own-names)
hackage-dependencies))
(define dependencies
diff --git a/tests/hackage.scm b/tests/hackage.scm
index 85a5c2115c..a11dd14846 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -368,6 +368,25 @@ (define test-cabal-no-final-newline
(test-assert "hackage->guix-package test without final newline"
(eval-test-with-cabal test-cabal-no-final-newline match-ghc-foo))
+;; Make sure internal libraries will not be part of the dependencies.
+(define test-cabal-internal-library-ignored
+ "name: foo
+version: 1.0.0
+homepage: http://test.org
+synopsis: synopsis
+description: description
+license: BSD3
+executable cabal
+ build-depends:
+ HTTP >= 4000.2.5 && < 4000.3,
+ internal
+library internal
+ build-depends: mtl >= 2.0 && < 3
+")
+
+(test-assert "hackage->guix-package test internal libraries are ignored"
+ (eval-test-with-cabal test-cabal-internal-library-ignored match-ghc-foo))
+
;; Check if-elif-else statements
(define test-cabal-if
"name: foo
--
2.35.1
P
P
Philip Munksgaard wrote on 2 Jun 2022 11:53
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
f9c895e0-28c6-4069-969e-00ace03055bf@www.fastmail.com
Hi Lars,

On Sun, 22 May 2022, at 11:37, Lars-Dominik Braun wrote:
Toggle quote (7 lines)
>> The attoparsec package on hackage defines multiple internal libraries inside one package, named "attoparsec" and "attoparsec-internal", with the first depending on the latter. Importing attoparsec using `guix import hackage attoparsec` therefore yields the following erroneous package definition: […]
>
> attached patches should fix this. I tried them with both, attoparsec and
> raaz, and internal libraries appear neither in inputs nor native inputs
> any more.
>

Indeed, that seems to work for those packages, but it still doesn't seem to work for OneTuple, as mentioned here: https://issues.guix.gnu.org/52152#3

Also, attoparsec still won't build with result of the import, but for different reasons.
P
P
Philip Munksgaard wrote on 2 Jun 2022 11:59
Re: bug#54760: import: hackage: Internal libraries are not filtered out of dependency list
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
dccaf1e9-e2f9-432b-9084-5134597ead2d@www.fastmail.com
On Thu, 2 Jun 2022, at 11:53, Philip Munksgaard wrote:
Toggle quote (3 lines)
> Also, attoparsec still won't build with result of the import, but for
> different reasons.

Actually, that's weird. Didn't attoparsec used to build?
P
P
Philip Munksgaard wrote on 2 Jun 2022 13:31
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
eb3dfbb5-23ef-4092-9e08-0301405a815e@www.fastmail.com
On Thu, 2 Jun 2022, at 11:59, Philip Munksgaard wrote:
Toggle quote (6 lines)
> On Thu, 2 Jun 2022, at 11:53, Philip Munksgaard wrote:
>> Also, attoparsec still won't build with result of the import, but for
>> different reasons.
>
> Actually, that's weird. Didn't attoparsec used to build?

Aha! The upgrade to GHC 9.0.0 caused ghc-hashable to break. I suppose many other packages could be broken as well.
L
L
Lars-Dominik Braun wrote on 2 Jun 2022 13:57
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760@debbugs.gnu.org)
YpiljmAhcSAHsn36@noor.fritz.box
Hey,

Toggle quote (1 lines)
> Indeed, that seems to work for those packages, but it still doesn't seem to work for OneTuple, as mentioned here:
I’m attaching a patch for that, please have a look. I’ll merge these
into master, since they don’t change any builds.

Toggle quote (1 lines)
> Aha! The upgrade to GHC 9.0.0 caused ghc-hashable to break. I suppose many other packages could be broken as well.
I didn’t actually upgrade any packages on wip-haskell yet, so it’s
likely they fail with a newer GHC than 8.10.

Cheers,
Lars
From 261736187d51c85c203ad08fbc1ae89340256a8c Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <lars@6xq.net>
Date: Thu, 2 Jun 2022 13:52:08 +0200
Subject: [PATCH] import: hackage: Filter upper/mixed case dependencies too.

* guix/import/hackage.scm (filter-dependencies): Convert OWN-NAMES
to lowercase before filtering.
(hackage-module->sexp): Remove #f from OWN-NAMES, which is used for
unnamed (default) libraries.
* tests/hackage.scm (test-cabal-internal-library-ignored): Add mismatched
uppercase letters.
---
guix/import/hackage.scm | 8 +++++---
tests/hackage.scm | 7 ++++---
2 files changed, 9 insertions(+), 6 deletions(-)

Toggle diff (54 lines)
diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm
index 6e982366cf..878a7d2f9c 100644
--- a/guix/import/hackage.scm
+++ b/guix/import/hackage.scm
@@ -227,9 +227,10 @@ (define (filter-dependencies dependencies own-names)
list with the names of dependencies. OWN-NAMES is the name of the Cabal
package being processed and its internal libaries and is used to filter
references to itself."
- (filter (lambda (d) (not (member (string-downcase d)
+ (let ((ignored-dependencies (map string-downcase
(append own-names ghc-standard-libraries))))
- dependencies))
+ (filter (lambda (d) (not (member (string-downcase d) ignored-dependencies)))
+ dependencies)))
(define* (hackage-module->sexp cabal cabal-hash
#:key (include-test-dependencies? #t))
@@ -250,7 +251,8 @@ (define source-url
(hackage-source-url name version))
(define own-names (cons (cabal-package-name cabal)
- (map cabal-library-name (cabal-package-library cabal))))
+ (filter (lambda (x) (not (eqv? x #f)))
+ (map cabal-library-name (cabal-package-library cabal)))))
(define hackage-dependencies
(filter-dependencies (cabal-dependencies->names cabal) own-names))
diff --git a/tests/hackage.scm b/tests/hackage.scm
index a11dd14846..ad2ee4b7f9 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -368,7 +368,8 @@ (define test-cabal-no-final-newline
(test-assert "hackage->guix-package test without final newline"
(eval-test-with-cabal test-cabal-no-final-newline match-ghc-foo))
-;; Make sure internal libraries will not be part of the dependencies.
+;; Make sure internal libraries will not be part of the dependencies,
+;; ignore case.
(define test-cabal-internal-library-ignored
"name: foo
version: 1.0.0
@@ -379,8 +380,8 @@ (define test-cabal-internal-library-ignored
executable cabal
build-depends:
HTTP >= 4000.2.5 && < 4000.3,
- internal
-library internal
+ internAl
+library internaL
build-depends: mtl >= 2.0 && < 3
")
--
2.35.1
P
P
Philip Munksgaard wrote on 2 Jun 2022 14:47
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
c565b36c-f4df-4ead-8d00-53f950e295cc@www.fastmail.com
Hi again,

On Thu, 2 Jun 2022, at 13:57, Lars-Dominik Braun wrote:
Toggle quote (6 lines)
> Hey,
>
>> Indeed, that seems to work for those packages, but it still doesn't seem to work for OneTuple, as mentioned here:
> I’m attaching a patch for that, please have a look. I’ll merge these
> into master, since they don’t change any builds.

That works, thanks!

Toggle quote (4 lines)
>> Aha! The upgrade to GHC 9.0.0 caused ghc-hashable to break. I suppose many other packages could be broken as well.
> I didn’t actually upgrade any packages on wip-haskell yet, so it’s
> likely they fail with a newer GHC than 8.10.

Aha. Let me know if there's anything I can do to help.
L
L
Lars-Dominik Braun wrote on 5 Jun 2022 11:43
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760@debbugs.gnu.org)
Ypx6zTU6FRcJdV+O@noor.fritz.box
Hi Philip,

Toggle quote (1 lines)
> Aha. Let me know if there's anything I can do to help.
at this point there’s two options:

1) We manually update all packages coming from Stackage (i.e. run `guix
refresh -t stackage` and adjust #:cabal-revision/inputs by hand)
2) We improve the updater so it automatically adjusts the entire package
– not just version and source – and then run `guix refresh`.

1 is more labor-intensive, but 2 might take more time to figure out.

Which option can you help with?

Thanks,
Lars
L
L
Lars-Dominik Braun wrote on 6 Jun 2022 13:30
Re: import: hackage: Internal libraries are not filtered out of dependency list
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760-done@debbugs.gnu.org)
Yp3lYgoJyrVn5Ej1@noor.fritz.box
Hi,

fix pushed to master as c3fbaee34548fbfb1617dc7fccc94c598efbd7a6 and following.

Cheers,
Lars
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 54760@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 54760
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch