Doc and vignettes bundled from Bioconductor

  • Open
  • quality assurance status badge
Details
One participant
  • zimoun
Owner
unassigned
Submitted by
zimoun
Severity
normal
Z
Z
zimoun wrote on 27 Oct 2020 21:36
(address . bug-guix@gnu.org)
86v9ev2ypw.fsf@gmail.com
Dear,

Let’s take the example:

Toggle snippet (51 lines)
$ guix import cran -a bioconductor diffcyt -r

Starting download of /tmp/guix-file.qJmqja
From https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz...
…8.8.tar.gz 776KiB 585KiB/s 00:01 [##################] 100.0%

Starting download of /tmp/guix-file.qAoQ59
From https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz...
…8.8.tar.gz 776KiB 565KiB/s 00:01 [##################] 100.0%
(define-public r-diffcyt
(package
(name "r-diffcyt")
(version "1.8.8")
(source
(origin
(method url-fetch)
(uri (bioconductor-uri "diffcyt" version))
(sha256
(base32
"1kq3zisgvi6kfra2sm17d01vw3rg0nz0vz5031jp2a96j7dryp98"))))
(properties `((upstream-name . "diffcyt")))
(build-system r-build-system)
(propagated-inputs
`(("r-circlize" ,r-circlize)
("r-complexheatmap" ,r-complexheatmap)
("r-dplyr" ,r-dplyr)
("r-edger" ,r-edger)
("r-flowcore" ,r-flowcore)
("r-flowsom" ,r-flowsom)
("r-limma" ,r-limma)
("r-lme4" ,r-lme4)
("r-magrittr" ,r-magrittr)
("r-multcomp" ,r-multcomp)
("r-reshape2" ,r-reshape2)
("r-s4vectors" ,r-s4vectors)
("r-summarizedexperiment"
,r-summarizedexperiment)
("r-tidyr" ,r-tidyr)))
(native-inputs `(("r-knitr" ,r-knitr)))
(home-page "https://github.com/lmweber/diffcyt")
(synopsis
"Differential discovery in high-dimensional cytometry via high-resolution clustering")
(description
"Statistical methods for differential discovery analyses in
high-dimensional cytometry data (including flow cytometry, mass
cytometry or CyTOF, and oligonucleotide-tagged cytometry), based on
a combination of high-resolution clustering and empirical Bayes
moderated tests adapted from transcriptomics.")
(license expat)))

Well, added to gnu/packages/bioconductor.scm. Then the usual
“./pre-inst-env guix build r-diffcyt” where the relevant parts are:

Toggle snippet (54 lines)
starting phase `unpack'

[...]

diffcyt/inst/NEWS
diffcyt/inst/doc/
diffcyt/inst/doc/diffcyt_workflow.R
diffcyt/inst/doc/diffcyt_workflow.Rmd
diffcyt/inst/doc/diffcyt_workflow.html

[...]

starting phase `install'
* installing *source* package ‘diffcyt’ ...
** using staged installation
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (diffcyt)

[...]

starting phase `check'
Testing examples for package ‘diffcyt’
Running specific tests for package ‘diffcyt’
Running ‘testthat.R’
Running vignettes for package ‘diffcyt’
Running ‘diffcyt_workflow.Rmd’

*** Source Errors ***

File diffcyt_workflow.R:
Cannot find the file(s): "diffcyt.png"

*** Weave Errors ***

File diffcyt_workflow.Rmd:
Cannot find the file(s): "diffcyt.png"
Warning message:
In engine$weave(path, quiet = TRUE, encoding = enc) :
The vignette engine knitr::rmarkdown is not available because the rmarkdown package is not available. Did you forget to add it to Suggests in DESCRIPTION? Please see https://github.com/yihui/knitr/issues/1864 for more information.
phase `check' succeeded after 61.9
[..]


Therefore, trying to address this error in the check phase, I am
puzzled. Initially, I thought it was a change in ’knitr’ [1,2], I
added:

Toggle snippet (10 lines)
(arguments
`(#:phases
(modify-phases %standard-phases
(add-before 'check 'patch-knitr-include_graphics
(lambda _
(substitute* "vignettes/diffcyt_workflow.Rmd"
(("knitr::include_graphics\\(\"diffcyt.png\"\\)")
"knitr::include_graphics(\"diffcyt.png\", error = FALSE)")

and even tried “after ’unpack” phase; idem.


Well, the error is about a missing file, however:

tar -xvf $(./pre-inst-env guix build r-diffcyt -S)

and tree returns:

??? inst
?   ??? doc
?   ?   ??? diffcyt_workflow.html
?   ?   ??? diffcyt_workflow.R
?   ?   ??? diffcyt_workflow.Rmd
?   ??? NEWS

[...]

??? vignettes
??? diffcyt.png
??? diffcyt_workflow.Rmd

and “inst/doc/diff_workflow.html“ is correct, i.e., displaying the logo
(which is diffcyt.png).


The issue for this specific package is that the check is running
inst/doc/diff_workflow.Rmd instead of vignettes/diffcyt_workflow.Rmd;
which one story.


The other story is how “inst/doc/diff_workflow.html” is generated? I am
not sure it is locally generated. To be sure, let fetch the tarball
from upstream:

Toggle snippet (5 lines)
wget https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz
tar xvf diffcyt_1.8.8.tar.gz
tree diffcyt

Z
Z
zimoun wrote on 28 Oct 2020 00:18
(address . bug-guix@gnu.org)
86o8kn2r6v.fsf@gmail.com
On Tue, 27 Oct 2020 at 21:36, zimoun <zimon.toutoune@gmail.com> wrote:

On part of the story,

Toggle quote (5 lines)
> --8<---------------cut here---------------start------------->8---
> (define-public r-diffcyt
> (package
> (name "r-diffcyt")

[...]

Toggle quote (1 lines)
> (native-inputs
`(("r-knitr" ,r-knitr)
("r-rmarkdown" ,r-rmarkdown)
("pandoc" ,pandoc)
("pandoc" ,pandoc-citeproc)
("r-biocstyle" ,r-biocstyle)))

[...]

Toggle quote (3 lines)
> (license expat)))
> --8<---------------cut here---------------end--------------->8---

leads to:

Toggle snippet (18 lines)
starting phase `check'
Testing examples for package ‘diffcyt’
Running specific tests for package ‘diffcyt’
Running ‘testthat.R’
Running vignettes for package ‘diffcyt’
Running ‘diffcyt_workflow.Rmd’

*** Source Errors ***

File diffcyt_workflow.R:
Cannot find the file(s): "diffcyt.png"

*** Weave Errors ***

File diffcyt_workflow.Rmd:
Cannot find the file(s): "diffcyt.png"

and I have no clue what I am missing.


The other part of the story is:

Toggle snippet (9 lines)
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'delete
(lambda _
(delete-file-recursively "inst")
#t)))))

leads to:

Toggle snippet (22 lines)
starting phase `install'
* installing *source* package ‘diffcyt’ ...
** using staged installation
** R
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
Warning in file(con, "w") :
cannot open file '/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/00LOCK-diffcyt/00new/diffcyt/doc/index.html': No such file or directory
Error in file(con, "w") : cannot open the connection
ERROR: installing vignettes failed
* removing ‘/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/diffcyt’
command "R" "CMD" "INSTALL" "--install-tests" "--library=/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/" "--built-timestamp=1970-01-01" "." failed with status 1
builder for `/gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv' failed with exit code 1
build of /gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv failed
View build log at '/var/log/guix/drvs/9d/wqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv.bz2'.
guix build: error: build of `/gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv' failed

and the addition of ’(mkdir-p "inst/doc")’ makes the build pass, but:

file:///gnu/store/…-r-diffcyt-1.8.8/site-library/diffcyt/doc/index.html

shows one HTML link pointing to:

file:///gnu/store/…-r-diffcyt-1.8.8/library/diffcyt/doc/diffcyt_workflow.html

which does not exist.


Another random example is ’r-ebimage’,

Toggle snippet (11 lines)
$ tar xvf $(guix build r-ebimage -S) | grep "inst/doc"
EBImage/inst/doc/
EBImage/inst/doc/EBImage-introduction.R
EBImage/inst/doc/EBImage-introduction.Rmd
EBImage/inst/doc/EBImage-introduction.html

$ md5sum EBImage/inst/doc/EBImage-introduction.html $(guix build r-ebimage)/site-library/EBImage/doc/EBImage-introduction.html
4dabd489660ff7909e4787cb23fe89d7 EBImage/inst/doc/EBImage-introduction.html
4dabd489660ff7909e4787cb23fe89d7 /gnu/store/9lmzhb0palxsm8ynacy2nnh5w8gvbn6j-r-ebimage-4.30.0/site-library/EBImage/doc/EBImage-introduction.html

Well bit-to-bit identical HTML as upstream… strong evidence that the
documentation is not locally generated. And the upstream-upstream


does not contains this HTML, AFAICT.


BTW, going via the Source Repository


i.e., ’git-fetch’ instead of ’url-fetch’ seems the right direction to
fix. Because:

1. it is the “real” source and this source is easily verifiable; for
example it is easy to check the commit hash of the Bioconductor repo
against the one of the upstream GitHub repo.

2. the fallback to SWH works for git-fetch. The tarball is not ready
yet; and who knows when… :-)

All the best,
simon
?