import: hackage: Internal libraries are not filtered out of dependency list

  • Done
  • quality assurance status badge
Details
2 participants
  • Lars-Dominik Braun
  • Philip Munksgaard
Owner
unassigned
Submitted by
Philip Munksgaard
Severity
normal
P
P
Philip Munksgaard wrote on 7 Apr 2022 10:18
(address . bug-guix@gnu.org)
842828ff-b80b-4488-aa5b-5e6189e0a607@www.fastmail.com
The attoparsec package on hackage defines multiple internal libraries inside one package, named "attoparsec" and "attoparsec-internal", with the first depending on the latter. Importing attoparsec using `guix import hackage attoparsec` therefore yields the following erroneous package definition:

(define-public ghc-attoparsec
(package
(name "ghc-attoparsec")
(version "0.14.4")
(source
(origin
(method url-fetch)
(uri (hackage-uri "attoparsec" version))
(sha256
(base32 "0v4yjz4qi8bwhbyavqxlhsfb1iv07v10gxi64khmsmi4hvjpycrz"))))
(build-system haskell-build-system)
(inputs (list ghc-scientific ghc-attoparsec-internal))
(native-inputs
(list ghc-quickcheck
ghc-quickcheck-unicode
ghc-tasty
ghc-tasty-quickcheck
ghc-vector))
(arguments
`(#:cabal-revision
("1" "149ihklmwnl13mmixq6iq5gzggkgqwsqrjlg2fshqwwbvbd4nn3r")))
(synopsis "Fast combinator parsing for bytestrings and text")
(description
"This package provides a fast parser combinator library, aimed particularly at
dealing efficiently with network protocols and complicated text/binary file
formats.")
(license license:bsd-3)))

Note that `ghc-attoparsec-internal` is listed as a dependency of `ghc-attoparsec`. I believe that is incorrect, and that we should filter out the internal dependencies from `inputs`, just like we filter out `ghc-attoparsec` from the list of `native-inputs`.
L
L
Lars-Dominik Braun wrote on 22 May 2022 11:37
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760@debbugs.gnu.org)
YooEVtIztcFN2nrI@noor.fritz.box
Hello Philip,

Toggle quote (2 lines)
> The attoparsec package on hackage defines multiple internal libraries inside one package, named "attoparsec" and "attoparsec-internal", with the first depending on the latter. Importing attoparsec using `guix import hackage attoparsec` therefore yields the following erroneous package definition: […]

attached patches should fix this. I tried them with both, attoparsec and
raaz, and internal libraries appear neither in inputs nor native inputs
any more.

Cheers,
Lars
Attachment: 0001-import-cabal-Support-library-names.patch
From 1aa0732e8995c3872ddbb67c1a845d39cc8ce6f9 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <lars@6xq.net>
Date: Sun, 22 May 2022 11:20:07 +0200
Subject: [PATCH 2/2] import: hackage: Filter internal libraries from inputs
and native-inputs.

* guix/import/hackage.scm (filter-dependencies): Support multiple
OWN-NAMES.
(hackage-module->sexp): Filter OWN-NAMES from HACKAGE-DEPENDENCIES and
HACKAGE-NATIVE-DEPENDENCIES.
* tests/hackage.scm (test-cabal-internal-library-ignored): New variable.
("hackage->guix-package test internal libraries are ignored"): New testcase.
---
guix/import/hackage.scm | 17 ++++++++++-------
tests/hackage.scm | 19 +++++++++++++++++++
2 files changed, 29 insertions(+), 7 deletions(-)

Toggle diff (77 lines)
diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm
index 0d6c77e399..6e982366cf 100644
--- a/guix/import/hackage.scm
+++ b/guix/import/hackage.scm
@@ -222,12 +222,13 @@ (define (cabal-custom-setup-dependencies->names cabal)
'())))
(map cabal-dependency-name custom-setup-dependencies)))
-(define (filter-dependencies dependencies own-name)
+(define (filter-dependencies dependencies own-names)
"Filter the dependencies included with the GHC compiler from DEPENDENCIES, a
-list with the names of dependencies. OWN-NAME is the name of the Cabal
-package being processed and is used to filter references to itself."
+list with the names of dependencies. OWN-NAMES is the name of the Cabal
+package being processed and its internal libaries and is used to filter
+references to itself."
(filter (lambda (d) (not (member (string-downcase d)
- (cons own-name ghc-standard-libraries))))
+ (append own-names ghc-standard-libraries))))
dependencies))
(define* (hackage-module->sexp cabal cabal-hash
@@ -248,9 +249,11 @@ (define revision
(define source-url
(hackage-source-url name version))
+ (define own-names (cons (cabal-package-name cabal)
+ (map cabal-library-name (cabal-package-library cabal))))
+
(define hackage-dependencies
- (filter-dependencies (cabal-dependencies->names cabal)
- (cabal-package-name cabal)))
+ (filter-dependencies (cabal-dependencies->names cabal) own-names))
(define hackage-native-dependencies
(lset-difference
@@ -260,7 +263,7 @@ (define hackage-native-dependencies
(cabal-test-dependencies->names cabal)
'())
(cabal-custom-setup-dependencies->names cabal))
- (cabal-package-name cabal))
+ own-names)
hackage-dependencies))
(define dependencies
diff --git a/tests/hackage.scm b/tests/hackage.scm
index 85a5c2115c..a11dd14846 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -368,6 +368,25 @@ (define test-cabal-no-final-newline
(test-assert "hackage->guix-package test without final newline"
(eval-test-with-cabal test-cabal-no-final-newline match-ghc-foo))
+;; Make sure internal libraries will not be part of the dependencies.
+(define test-cabal-internal-library-ignored
+ "name: foo
+version: 1.0.0
+homepage: http://test.org
+synopsis: synopsis
+description: description
+license: BSD3
+executable cabal
+ build-depends:
+ HTTP >= 4000.2.5 && < 4000.3,
+ internal
+library internal
+ build-depends: mtl >= 2.0 && < 3
+")
+
+(test-assert "hackage->guix-package test internal libraries are ignored"
+ (eval-test-with-cabal test-cabal-internal-library-ignored match-ghc-foo))
+
;; Check if-elif-else statements
(define test-cabal-if
"name: foo
--
2.35.1
P
P
Philip Munksgaard wrote on 2 Jun 2022 11:53
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
f9c895e0-28c6-4069-969e-00ace03055bf@www.fastmail.com
Hi Lars,

On Sun, 22 May 2022, at 11:37, Lars-Dominik Braun wrote:
Toggle quote (7 lines)
>> The attoparsec package on hackage defines multiple internal libraries inside one package, named "attoparsec" and "attoparsec-internal", with the first depending on the latter. Importing attoparsec using `guix import hackage attoparsec` therefore yields the following erroneous package definition: […]
>
> attached patches should fix this. I tried them with both, attoparsec and
> raaz, and internal libraries appear neither in inputs nor native inputs
> any more.
>

Indeed, that seems to work for those packages, but it still doesn't seem to work for OneTuple, as mentioned here: https://issues.guix.gnu.org/52152#3

Also, attoparsec still won't build with result of the import, but for different reasons.
P
P
Philip Munksgaard wrote on 2 Jun 2022 11:59
Re: bug#54760: import: hackage: Internal libraries are not filtered out of dependency list
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
dccaf1e9-e2f9-432b-9084-5134597ead2d@www.fastmail.com
On Thu, 2 Jun 2022, at 11:53, Philip Munksgaard wrote:
Toggle quote (3 lines)
> Also, attoparsec still won't build with result of the import, but for
> different reasons.

Actually, that's weird. Didn't attoparsec used to build?
P
P
Philip Munksgaard wrote on 2 Jun 2022 13:31
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
eb3dfbb5-23ef-4092-9e08-0301405a815e@www.fastmail.com
On Thu, 2 Jun 2022, at 11:59, Philip Munksgaard wrote:
Toggle quote (6 lines)
> On Thu, 2 Jun 2022, at 11:53, Philip Munksgaard wrote:
>> Also, attoparsec still won't build with result of the import, but for
>> different reasons.
>
> Actually, that's weird. Didn't attoparsec used to build?

Aha! The upgrade to GHC 9.0.0 caused ghc-hashable to break. I suppose many other packages could be broken as well.
L
L
Lars-Dominik Braun wrote on 2 Jun 2022 13:57
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760@debbugs.gnu.org)
YpiljmAhcSAHsn36@noor.fritz.box
Hey,

Toggle quote (1 lines)
> Indeed, that seems to work for those packages, but it still doesn't seem to work for OneTuple, as mentioned here:
I’m attaching a patch for that, please have a look. I’ll merge these
into master, since they don’t change any builds.

Toggle quote (1 lines)
> Aha! The upgrade to GHC 9.0.0 caused ghc-hashable to break. I suppose many other packages could be broken as well.
I didn’t actually upgrade any packages on wip-haskell yet, so it’s
likely they fail with a newer GHC than 8.10.

Cheers,
Lars
From 261736187d51c85c203ad08fbc1ae89340256a8c Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <lars@6xq.net>
Date: Thu, 2 Jun 2022 13:52:08 +0200
Subject: [PATCH] import: hackage: Filter upper/mixed case dependencies too.

* guix/import/hackage.scm (filter-dependencies): Convert OWN-NAMES
to lowercase before filtering.
(hackage-module->sexp): Remove #f from OWN-NAMES, which is used for
unnamed (default) libraries.
* tests/hackage.scm (test-cabal-internal-library-ignored): Add mismatched
uppercase letters.
---
guix/import/hackage.scm | 8 +++++---
tests/hackage.scm | 7 ++++---
2 files changed, 9 insertions(+), 6 deletions(-)

Toggle diff (54 lines)
diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm
index 6e982366cf..878a7d2f9c 100644
--- a/guix/import/hackage.scm
+++ b/guix/import/hackage.scm
@@ -227,9 +227,10 @@ (define (filter-dependencies dependencies own-names)
list with the names of dependencies. OWN-NAMES is the name of the Cabal
package being processed and its internal libaries and is used to filter
references to itself."
- (filter (lambda (d) (not (member (string-downcase d)
+ (let ((ignored-dependencies (map string-downcase
(append own-names ghc-standard-libraries))))
- dependencies))
+ (filter (lambda (d) (not (member (string-downcase d) ignored-dependencies)))
+ dependencies)))
(define* (hackage-module->sexp cabal cabal-hash
#:key (include-test-dependencies? #t))
@@ -250,7 +251,8 @@ (define source-url
(hackage-source-url name version))
(define own-names (cons (cabal-package-name cabal)
- (map cabal-library-name (cabal-package-library cabal))))
+ (filter (lambda (x) (not (eqv? x #f)))
+ (map cabal-library-name (cabal-package-library cabal)))))
(define hackage-dependencies
(filter-dependencies (cabal-dependencies->names cabal) own-names))
diff --git a/tests/hackage.scm b/tests/hackage.scm
index a11dd14846..ad2ee4b7f9 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -368,7 +368,8 @@ (define test-cabal-no-final-newline
(test-assert "hackage->guix-package test without final newline"
(eval-test-with-cabal test-cabal-no-final-newline match-ghc-foo))
-;; Make sure internal libraries will not be part of the dependencies.
+;; Make sure internal libraries will not be part of the dependencies,
+;; ignore case.
(define test-cabal-internal-library-ignored
"name: foo
version: 1.0.0
@@ -379,8 +380,8 @@ (define test-cabal-internal-library-ignored
executable cabal
build-depends:
HTTP >= 4000.2.5 && < 4000.3,
- internal
-library internal
+ internAl
+library internaL
build-depends: mtl >= 2.0 && < 3
")
--
2.35.1
P
P
Philip Munksgaard wrote on 2 Jun 2022 14:47
(name . Lars-Dominik Braun)(address . lars@6xq.net)(address . 54760@debbugs.gnu.org)
c565b36c-f4df-4ead-8d00-53f950e295cc@www.fastmail.com
Hi again,

On Thu, 2 Jun 2022, at 13:57, Lars-Dominik Braun wrote:
Toggle quote (6 lines)
> Hey,
>
>> Indeed, that seems to work for those packages, but it still doesn't seem to work for OneTuple, as mentioned here:
> I’m attaching a patch for that, please have a look. I’ll merge these
> into master, since they don’t change any builds.

That works, thanks!

Toggle quote (4 lines)
>> Aha! The upgrade to GHC 9.0.0 caused ghc-hashable to break. I suppose many other packages could be broken as well.
> I didn’t actually upgrade any packages on wip-haskell yet, so it’s
> likely they fail with a newer GHC than 8.10.

Aha. Let me know if there's anything I can do to help.
L
L
Lars-Dominik Braun wrote on 5 Jun 2022 11:43
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760@debbugs.gnu.org)
Ypx6zTU6FRcJdV+O@noor.fritz.box
Hi Philip,

Toggle quote (1 lines)
> Aha. Let me know if there's anything I can do to help.
at this point there’s two options:

1) We manually update all packages coming from Stackage (i.e. run `guix
refresh -t stackage` and adjust #:cabal-revision/inputs by hand)
2) We improve the updater so it automatically adjusts the entire package
– not just version and source – and then run `guix refresh`.

1 is more labor-intensive, but 2 might take more time to figure out.

Which option can you help with?

Thanks,
Lars
L
L
Lars-Dominik Braun wrote on 6 Jun 2022 13:30
Re: import: hackage: Internal libraries are not filtered out of dependency list
(name . Philip Munksgaard)(address . philip@munksgaard.me)(address . 54760-done@debbugs.gnu.org)
Yp3lYgoJyrVn5Ej1@noor.fritz.box
Hi,

fix pushed to master as c3fbaee34548fbfb1617dc7fccc94c598efbd7a6 and following.

Cheers,
Lars
Closed
?