Doc and vignettes bundled from Bioconductor

  • Open
  • quality assurance status badge
Details
One participant
  • zimoun
Owner
unassigned
Submitted by
zimoun
Severity
normal
Z
Z
zimoun wrote on 27 Oct 2020 21:36
(address . bug-guix@gnu.org)
86v9ev2ypw.fsf@gmail.com
Dear,

Let’s take the example:

Toggle snippet (51 lines)
$ guix import cran -a bioconductor diffcyt -r

Starting download of /tmp/guix-file.qJmqja
From https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz...
…8.8.tar.gz 776KiB 585KiB/s 00:01 [##################] 100.0%

Starting download of /tmp/guix-file.qAoQ59
From https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz...
…8.8.tar.gz 776KiB 565KiB/s 00:01 [##################] 100.0%
(define-public r-diffcyt
(package
(name "r-diffcyt")
(version "1.8.8")
(source
(origin
(method url-fetch)
(uri (bioconductor-uri "diffcyt" version))
(sha256
(base32
"1kq3zisgvi6kfra2sm17d01vw3rg0nz0vz5031jp2a96j7dryp98"))))
(properties `((upstream-name . "diffcyt")))
(build-system r-build-system)
(propagated-inputs
`(("r-circlize" ,r-circlize)
("r-complexheatmap" ,r-complexheatmap)
("r-dplyr" ,r-dplyr)
("r-edger" ,r-edger)
("r-flowcore" ,r-flowcore)
("r-flowsom" ,r-flowsom)
("r-limma" ,r-limma)
("r-lme4" ,r-lme4)
("r-magrittr" ,r-magrittr)
("r-multcomp" ,r-multcomp)
("r-reshape2" ,r-reshape2)
("r-s4vectors" ,r-s4vectors)
("r-summarizedexperiment"
,r-summarizedexperiment)
("r-tidyr" ,r-tidyr)))
(native-inputs `(("r-knitr" ,r-knitr)))
(home-page "https://github.com/lmweber/diffcyt")
(synopsis
"Differential discovery in high-dimensional cytometry via high-resolution clustering")
(description
"Statistical methods for differential discovery analyses in
high-dimensional cytometry data (including flow cytometry, mass
cytometry or CyTOF, and oligonucleotide-tagged cytometry), based on
a combination of high-resolution clustering and empirical Bayes
moderated tests adapted from transcriptomics.")
(license expat)))

Well, added to gnu/packages/bioconductor.scm. Then the usual
“./pre-inst-env guix build r-diffcyt” where the relevant parts are:

Toggle snippet (54 lines)
starting phase `unpack'

[...]

diffcyt/inst/NEWS
diffcyt/inst/doc/
diffcyt/inst/doc/diffcyt_workflow.R
diffcyt/inst/doc/diffcyt_workflow.Rmd
diffcyt/inst/doc/diffcyt_workflow.html

[...]

starting phase `install'
* installing *source* package ‘diffcyt’ ...
** using staged installation
** R
** inst
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (diffcyt)

[...]

starting phase `check'
Testing examples for package ‘diffcyt’
Running specific tests for package ‘diffcyt’
Running ‘testthat.R’
Running vignettes for package ‘diffcyt’
Running ‘diffcyt_workflow.Rmd’

*** Source Errors ***

File diffcyt_workflow.R:
Cannot find the file(s): "diffcyt.png"

*** Weave Errors ***

File diffcyt_workflow.Rmd:
Cannot find the file(s): "diffcyt.png"
Warning message:
In engine$weave(path, quiet = TRUE, encoding = enc) :
The vignette engine knitr::rmarkdown is not available because the rmarkdown package is not available. Did you forget to add it to Suggests in DESCRIPTION? Please see https://github.com/yihui/knitr/issues/1864 for more information.
phase `check' succeeded after 61.9
[..]


Therefore, trying to address this error in the check phase, I am
puzzled. Initially, I thought it was a change in ’knitr’ [1,2], I
added:

Toggle snippet (10 lines)
(arguments
`(#:phases
(modify-phases %standard-phases
(add-before 'check 'patch-knitr-include_graphics
(lambda _
(substitute* "vignettes/diffcyt_workflow.Rmd"
(("knitr::include_graphics\\(\"diffcyt.png\"\\)")
"knitr::include_graphics(\"diffcyt.png\", error = FALSE)")

and even tried “after ’unpack” phase; idem.


Well, the error is about a missing file, however:

tar -xvf $(./pre-inst-env guix build r-diffcyt -S)

and tree returns:

??? inst
?   ??? doc
?   ?   ??? diffcyt_workflow.html
?   ?   ??? diffcyt_workflow.R
?   ?   ??? diffcyt_workflow.Rmd
?   ??? NEWS

[...]

??? vignettes
??? diffcyt.png
??? diffcyt_workflow.Rmd

and “inst/doc/diff_workflow.html“ is correct, i.e., displaying the logo
(which is diffcyt.png).


The issue for this specific package is that the check is running
inst/doc/diff_workflow.Rmd instead of vignettes/diffcyt_workflow.Rmd;
which one story.


The other story is how “inst/doc/diff_workflow.html” is generated? I am
not sure it is locally generated. To be sure, let fetch the tarball
from upstream:

Toggle snippet (5 lines)
wget https://bioconductor.org/packages/release/bioc/src/contrib/diffcyt_1.8.8.tar.gz
tar xvf diffcyt_1.8.8.tar.gz
tree diffcyt

Z
Z
zimoun wrote on 28 Oct 2020 00:18
(address . bug-guix@gnu.org)
86o8kn2r6v.fsf@gmail.com
On Tue, 27 Oct 2020 at 21:36, zimoun <zimon.toutoune@gmail.com> wrote:

On part of the story,

Toggle quote (5 lines)
> --8<---------------cut here---------------start------------->8---
> (define-public r-diffcyt
> (package
> (name "r-diffcyt")

[...]

Toggle quote (1 lines)
> (native-inputs
`(("r-knitr" ,r-knitr)
("r-rmarkdown" ,r-rmarkdown)
("pandoc" ,pandoc)
("pandoc" ,pandoc-citeproc)
("r-biocstyle" ,r-biocstyle)))

[...]

Toggle quote (3 lines)
> (license expat)))
> --8<---------------cut here---------------end--------------->8---

leads to:

Toggle snippet (18 lines)
starting phase `check'
Testing examples for package ‘diffcyt’
Running specific tests for package ‘diffcyt’
Running ‘testthat.R’
Running vignettes for package ‘diffcyt’
Running ‘diffcyt_workflow.Rmd’

*** Source Errors ***

File diffcyt_workflow.R:
Cannot find the file(s): "diffcyt.png"

*** Weave Errors ***

File diffcyt_workflow.Rmd:
Cannot find the file(s): "diffcyt.png"

and I have no clue what I am missing.


The other part of the story is:

Toggle snippet (9 lines)
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'delete
(lambda _
(delete-file-recursively "inst")
#t)))))

leads to:

Toggle snippet (22 lines)
starting phase `install'
* installing *source* package ‘diffcyt’ ...
** using staged installation
** R
** tests
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
Warning in file(con, "w") :
cannot open file '/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/00LOCK-diffcyt/00new/diffcyt/doc/index.html': No such file or directory
Error in file(con, "w") : cannot open the connection
ERROR: installing vignettes failed
* removing ‘/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/diffcyt’
command "R" "CMD" "INSTALL" "--install-tests" "--library=/gnu/store/nqymkk1g0zsmf2wwrq5zd6rh7sdhvpjm-r-diffcyt-1.8.8/site-library/" "--built-timestamp=1970-01-01" "." failed with status 1
builder for `/gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv' failed with exit code 1
build of /gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv failed
View build log at '/var/log/guix/drvs/9d/wqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv.bz2'.
guix build: error: build of `/gnu/store/9dwqj8vfn2rd8amprrqw2db511c65az4-r-diffcyt-1.8.8.drv' failed

and the addition of ’(mkdir-p "inst/doc")’ makes the build pass, but:

file:///gnu/store/…-r-diffcyt-1.8.8/site-library/diffcyt/doc/index.html

shows one HTML link pointing to:

file:///gnu/store/…-r-diffcyt-1.8.8/library/diffcyt/doc/diffcyt_workflow.html

which does not exist.


Another random example is ’r-ebimage’,

Toggle snippet (11 lines)
$ tar xvf $(guix build r-ebimage -S) | grep "inst/doc"
EBImage/inst/doc/
EBImage/inst/doc/EBImage-introduction.R
EBImage/inst/doc/EBImage-introduction.Rmd
EBImage/inst/doc/EBImage-introduction.html

$ md5sum EBImage/inst/doc/EBImage-introduction.html $(guix build r-ebimage)/site-library/EBImage/doc/EBImage-introduction.html
4dabd489660ff7909e4787cb23fe89d7 EBImage/inst/doc/EBImage-introduction.html
4dabd489660ff7909e4787cb23fe89d7 /gnu/store/9lmzhb0palxsm8ynacy2nnh5w8gvbn6j-r-ebimage-4.30.0/site-library/EBImage/doc/EBImage-introduction.html

Well bit-to-bit identical HTML as upstream… strong evidence that the
documentation is not locally generated. And the upstream-upstream


does not contains this HTML, AFAICT.


BTW, going via the Source Repository


i.e., ’git-fetch’ instead of ’url-fetch’ seems the right direction to
fix. Because:

1. it is the “real” source and this source is easily verifiable; for
example it is easy to check the commit hash of the Bioconductor repo
against the one of the upstream GitHub repo.

2. the fallback to SWH works for git-fetch. The tarball is not ready
yet; and who knows when… :-)

All the best,
simon
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 44264@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 44264
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch