ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly

  • Done
  • quality assurance status badge
Details
6 participants
  • Leo Famulari
  • Tobias Geerinckx-Rice
  • Mark H Weaver
  • Nathan Dehnel
  • raingloom
  • zimoun
Owner
unassigned
Submitted by
Nathan Dehnel
Severity
normal
N
N
Nathan Dehnel wrote on 15 Sep 2020 13:27
(address . bug-guix@gnu.org)
CAEEhgEtsM4BQ=5uu7VDC9afGubzs0Xg6jPYyH7g4A+xPTMRgzA@mail.gmail.com
[youtube] 7Ijd_iN9Blk: Downloading webpage
[download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been downloaded
[download] 100% of 15.45MiB
ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please install one.
T
T
Tobias Geerinckx-Rice wrote on 15 Sep 2020 14:06
(name . Nathan Dehnel)(address . ncdehnel@gmail.com)
87bli7s0r0.fsf@nckx
Hi Nathan,

Nathan Dehnel ???
Toggle quote (9 lines)
> bash-5.0$ youtube-dl -x
> https://www.youtube.com/watch?v=7Ijd_iN9Blk
> [youtube] 7Ijd_iN9Blk: Downloading webpage
> [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been
> downloaded
> [download] 100% of 15.45MiB
> ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please
> install one.

It works fine without ‘-x’:

[youtube] 7Ijd_iN9Blk: Downloading webpage
[download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
[download] 100% of 37.09MiB in 00:02

This *is* functioning correctly in my view: that advanced options
optionally require run-time dependencies is not a bug but a
feature. The programme clearly explains which package could be
installed to continue, and the user can chose which one they
prefer - including neither, which suffices for the majority of
youtube-dling.

Matters would be different if the error message were less clear,
or perhaps if ffmpeg weren't so insanely great:

λ guix size youtube-dl | tail -n1
total: 186.9 MiB
λ guix size youtube-dl ffmpeg | tail -n1
total: 811.2 MiB

(!)

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCX2CuMw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15Z6QA/1Ia1UABr4VP+trQdAXyxebJhnFckMRa9ARwO2Dm
+gFuAP4m6U+AqKtl0sezgJlNLjw7p+ZujyIzmkNWlO8sSG2eCQ==
=4m3B
-----END PGP SIGNATURE-----

R
R
raingloom wrote on 15 Sep 2020 21:34
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)
20200915213436.1577fed3@riseup.net
On Tue, 15 Sep 2020 14:06:11 +0200
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
wrote:

Toggle quote (40 lines)
> Hi Nathan,
>
> Nathan Dehnel ???
> > bash-5.0$ youtube-dl -x
> > https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been
> > downloaded
> > [download] 100% of 15.45MiB
> > ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please
> > install one.
>
> It works fine without ‘-x’:
>
> λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
> [youtube] 7Ijd_iN9Blk: Downloading webpage
> [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
> [download] 100% of 37.09MiB in 00:02
>
> This *is* functioning correctly in my view: that advanced options
> optionally require run-time dependencies is not a bug but a
> feature. The programme clearly explains which package could be
> installed to continue, and the user can chose which one they
> prefer - including neither, which suffices for the majority of
> youtube-dling.
>
> Matters would be different if the error message were less clear,
> or perhaps if ffmpeg weren't so insanely great:
>
> λ guix size youtube-dl | tail -n1
> total: 186.9 MiB
> λ guix size youtube-dl ffmpeg | tail -n1
> total: 811.2 MiB
>
> (!)
>
> Kind regards,
>
> T G-R

Maybe it's time we added an "optional dependencies" field?
There seems to be a bug report or help request like every week that
just boils down to "this package has an undocumented dependency".
L
L
Leo Famulari wrote on 16 Sep 2020 01:17
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)
20200915231706.GB20807@jasmine.lan
On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
Toggle quote (8 lines)
> Matters would be different if the error message were less clear, or perhaps
> if ffmpeg weren't so insanely great:
>
> λ guix size youtube-dl | tail -n1
> total: 186.9 MiB
> λ guix size youtube-dl ffmpeg | tail -n1
> total: 811.2 MiB

I wonder, should we expect FFmpeg to already be referenced by somebody's
profile if they are using youtube-dl? VLC and mpv both depend on FFmpeg.
The use case of "download video and watch it on another machine (or
never watch it)" seems somewhat esoteric.

On the other hand, youtube-dl has no explicit dependencies — I guess
that everything it needs is from Python. That's beautiful, and I
understand the reluctance to add a dependency, especially on a "kitchen
sink" package like FFmpeg.
N
N
Nathan Dehnel wrote on 16 Sep 2020 01:52
(name . raingloom)(address . raingloom@riseup.net)
CAEEhgEuR+kpafsKaZGF2wekTyBTzYbw6uqfmmntQbKqE3+SThw@mail.gmail.com
Ludovic wanted to add useflags to guix.
I don't know what became of it.

On Tue, Sep 15, 2020 at 2:44 PM raingloom <raingloom@riseup.net> wrote:
Toggle quote (48 lines)
>
> On Tue, 15 Sep 2020 14:06:11 +0200
> Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
> wrote:
>
> > Hi Nathan,
> >
> > Nathan Dehnel ???
> > > bash-5.0$ youtube-dl -x
> > > https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > > [download] Deadmau5 - Hit Save-7Ijd_iN9Blk.webm has already been
> > > downloaded
> > > [download] 100% of 15.45MiB
> > > ERROR: ffprobe/avprobe and ffmpeg/avconv not found. Please
> > > install one.
> >
> > It works fine without ‘-x’:
> >
> > λ youtube-dl https://www.youtube.com/watch?v=7Ijd_iN9Blk
> > [youtube] 7Ijd_iN9Blk: Downloading webpage
> > [download] Destination: Deadmau5 - Hit Save-7Ijd_iN9Blk.mp4
> > [download] 100% of 37.09MiB in 00:02
> >
> > This *is* functioning correctly in my view: that advanced options
> > optionally require run-time dependencies is not a bug but a
> > feature. The programme clearly explains which package could be
> > installed to continue, and the user can chose which one they
> > prefer - including neither, which suffices for the majority of
> > youtube-dling.
> >
> > Matters would be different if the error message were less clear,
> > or perhaps if ffmpeg weren't so insanely great:
> >
> > λ guix size youtube-dl | tail -n1
> > total: 186.9 MiB
> > λ guix size youtube-dl ffmpeg | tail -n1
> > total: 811.2 MiB
> >
> > (!)
> >
> > Kind regards,
> >
> > T G-R
>
> Maybe it's time we added an "optional dependencies" field?
> There seems to be a bug report or help request like every week that
> just boils down to "this package has an undocumented dependency".
M
M
Mark H Weaver wrote on 16 Sep 2020 10:14
(address . ncdehnel@gmail.com)
87mu1q9lyw.fsf@netris.org
Leo Famulari <leo@famulari.name> writes:

Toggle quote (14 lines)
> On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
>> Matters would be different if the error message were less clear, or perhaps
>> if ffmpeg weren't so insanely great:
>>
>> λ guix size youtube-dl | tail -n1
>> total: 186.9 MiB
>> λ guix size youtube-dl ffmpeg | tail -n1
>> total: 811.2 MiB
>
> I wonder, should we expect FFmpeg to already be referenced by somebody's
> profile if they are using youtube-dl? VLC and mpv both depend on FFmpeg.
> The use case of "download video and watch it on another machine (or
> never watch it)" seems somewhat esoteric.

I was just about to disagree with you, on the grounds that GStreamer,
and in particular Totem (a.k.a. GNOME Videos) does not depend on FFmpeg.
It is, in fact, possible to watch videos downloaded using 'youtube-dl',
with only gst-plugins-{base,good} installed, provided that you request
freedom-respecting formats like VP9 and Opus.

How to request freedom-respecting formats? Unfortunately, the obvious
"--prefer-free-formats" usually doesn't work in my experience, but you
can run "youtube-dl -F <URL>" to see a list of available formats, and
then something like "youtube-dl -f 247+251 <URL>" (for non-live youtube
videos of sufficient resolution) to download VP9 and Opus and combine
them into a file.

However, in my recent experience, youtube usually provides the
freedom-respecting formats only as separate audio and video streams
which must be recombined by youtube-dl, and in that case the 'ffmpeg'
command line tool seems to be required.

So, ultimately I tend to agree with Leo that 'youtube-dl' is unlikely to
be useful without 'ffmpeg', and therefore it might as well depend on it
somehow.

I see two possible approaches: ffmpeg could be added to youtube-dl's
propagated-inputs, or it could be instead be added to 'inputs', and we
could substitute hardcoded paths to 'ffmpeg' in the built 'youtube-dl'.
I generally tend to favor the latter approach, but there are valid
arguments for each approach and I don't feel strongly about it.

What do you think?

Mark
T
T
Tobias Geerinckx-Rice wrote on 16 Sep 2020 10:58
(name . Mark H Weaver)(address . mhw@netris.org)
875z8ertcp.fsf@nckx
Leo, Mark,

Mark H Weaver ???
Toggle quote (21 lines)
> Leo Famulari <leo@famulari.name> writes:
>
>> On Tue, Sep 15, 2020 at 02:06:11PM +0200, Tobias Geerinckx-Rice
>> via Bug reports for GNU Guix wrote:
>>> Matters would be different if the error message were less
>>> clear, or perhaps
>>> if ffmpeg weren't so insanely great:
>>>
>>> λ guix size youtube-dl | tail -n1
>>> total: 186.9 MiB
>>> λ guix size youtube-dl ffmpeg | tail -n1
>>> total: 811.2 MiB
>>
>> I wonder, should we expect FFmpeg to already be referenced by
>> somebody's
>> profile if they are using youtube-dl? VLC and mpv both depend
>> on FFmpeg.
>> The use case of "download video and watch it on another machine
>> (or
>> never watch it)" seems somewhat esoteric.

I don't see why my use case should be sidelined as ‘esoteric’.
;-)

Toggle quote (7 lines)
> However, in my recent experience, youtube usually provides the
> freedom-respecting formats only as separate audio and video
> streams
> which must be recombined by youtube-dl, and in that case the
> 'ffmpeg'
> command line tool seems to be required.

This is a good argument. If including ffmpeg by default makes it
easier to request & mux freeër formats, let's do that.

Toggle quote (10 lines)
> I see two possible approaches: ffmpeg could be added to
> youtube-dl's
> propagated-inputs, or it could be instead be added to 'inputs',
> and we
> could substitute hardcoded paths to 'ffmpeg' in the built
> 'youtube-dl'.
> I generally tend to favor the latter approach, but there are
> valid
> arguments for each approach and I don't feel strongly about it.

Which advantages does the former have? I wasn't aware of any.

Thanks!

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCX2HTpg0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15rQAA/A8BhACrbvnqwkyRK+8jNjn6Wzp27A3tFY+ElzlO
w5N9AQComy1HNFEUu34SqG9bkauRJxpYvNEC28mAYdf4Onp7BA==
=rYpc
-----END PGP SIGNATURE-----

T
T
Tobias Geerinckx-Rice wrote on 16 Sep 2020 11:54
[PATCH] gnu: youtube-dl: Refer to ffmpeg.
(address . 43418@debbugs.gnu.org)
20200916095415.13617-1-me@tobias.gr

* gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
[arguments]: Fall back to input ffmpeg instead of searching $PATH.
---
gnu/packages/video.scm | 9 +++++++++
1 file changed, 9 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/video.scm b/gnu/packages/video.scm
index 8ebe7dbcdf..50f751f8da 100644
--- a/gnu/packages/video.scm
+++ b/gnu/packages/video.scm
@@ -2184,6 +2184,13 @@ To load this plugin, specify the following option when starting mpv:
;; 'youtube-dl.bash-completion'.
`(#:tests? #f ; Many tests fail. The test suite can be run with pytest.
#:phases (modify-phases %standard-phases
+ (add-after 'unpack 'default-to-the-ffmpeg-input
+ (lambda _
+ ;; Continue respecting the --ffmpeg-location argument.
+ (substitute* "youtube_dl/postprocessor/ffmpeg.py"
+ (("\\.get\\('ffmpeg_location'\\)" match)
+ (format #f "~a or '~a'" match (which "ffmpeg"))))
+ #t))
(add-before 'install 'fix-the-data-directories
(lambda* (#:key outputs #:allow-other-keys)
(let ((prefix (assoc-ref outputs "out")))
@@ -2207,6 +2214,8 @@ To load this plugin, specify the following option when starting mpv:
(copy-file "youtube-dl.zsh"
(string-append zsh "/_youtube-dl"))
#t))))))
+ (inputs
+ `(("ffmpeg" ,ffmpeg)))
(synopsis "Download videos from YouTube.com and other sites")
(description
"Youtube-dl is a small command-line program to download videos from
--
2.28.0
M
M
Mark H Weaver wrote on 16 Sep 2020 13:30
Re: bug#43418: ffprobe/avprobe and ffmpeg/avconv should be added as dependencies of youtube-dl so it will function correctly
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
87bli69cw9.fsf@netris.org
Hi Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

Toggle quote (11 lines)
> Mark H Weaver ???
>
>> I see two possible approaches: ffmpeg could be added to youtube-dl's
>> propagated-inputs, or it could be instead be added to 'inputs', and
>> we could substitute hardcoded paths to 'ffmpeg' in the built
>> 'youtube-dl'. I generally tend to favor the latter approach, but
>> there are valid arguments for each approach and I don't feel strongly
>> about it.
>
> Which advantages does the former have? I wasn't aware of any.

The former approach is arguably simpler, easier to maintain, and if the
user upgrades only 'ffmpeg' while leaving 'youtube-dl' unchanged, with
the latter approach 'youtube-dl' will continue using the old 'ffmpeg'
with its potential security holes, which users unfamiliar with Guix
might not expect.

I still generally prefer the latter approach for several reasons, most
notably because it means fewer dynamically-determined dependencies,
which are contrary to our goals for reproducible behavior, i.e. that if
the software in a given store item works today for your use case, it
will work the same way tomorrow.

However, it's worth noting that this property is less useful for a
program like 'youtube-dl', because a copy of 'youtube-dl' that works
today might well stop working tomorrow because of changes made on the
server side, regardless of our efforts.

Mark
M
M
Mark H Weaver wrote on 16 Sep 2020 13:31
Re: bug#43418: [PATCH] gnu: youtube-dl: Refer to ffmpeg.
87a6xq9cvh.fsf@netris.org
Hi Tobias,

Toggle quote (5 lines)
>
> * gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
> [arguments]: Fall back to input ffmpeg instead of searching $PATH.

Looks good to me. Thank you!

Mark
T
T
Tobias Geerinckx-Rice wrote on 16 Sep 2020 13:54
(address . 43418-done@debbugs.gnu.org)
87zh5qq6m5.fsf@nckx
Mark, Simon,

Mark H Weaver ???
Toggle quote (8 lines)
>>
>> * gnu/packages/video.scm (youtube-dl)[inputs]: Add ffmpeg.
>> [arguments]: Fall back to input ffmpeg instead of searching
>> $PATH.
>
> Looks good to me. Thank you!

Thanks, pushed.

Toggle quote (2 lines)
> The usage seems more Fixes than Addresses.

I'm still of the opinion that this was no bug but a feature
request, and there was nothing to fix. I promise to add it to the
release notes if you think it's necessary, but would rather keep
those limited to ‘real’ bugs as well. It's not like we're short
on those ;-)

Whatever it was, it's now closed, and we'll be spared of
duplicates from now on.

Thanks all,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCX2H9Ag0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15OVoA/ReNmO3ktaM51oeLkqG86JqI0Qw28MIn/C2nnc19
PSbLAP9F3BDvzSlbd6WBeAIbjqMcBEYK5eWQEdGk+DUd88wUDA==
=qmgq
-----END PGP SIGNATURE-----

Closed
Z
Z
zimoun wrote on 16 Sep 2020 14:07
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
CAJ3okZ0m+foNMcJWCFfohc9gcxMB0U3Km8=CooT1nkGL_1njrA@mail.gmail.com
On Wed, 16 Sep 2020 at 13:54, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

Toggle quote (8 lines)
> > The usage seems more Fixes than Addresses.
>
> I'm still of the opinion that this was no bug but a feature
> request, and there was nothing to fix. I promise to add it to the
> release notes if you think it's necessary, but would rather keep
> those limited to ‘real’ bugs as well. It's not like we're short
> on those ;-)

I am fine with whatever the convention is. Just one that we
collectively share. :-)
It helps to dig with the tandem "git log --grep" and "M-x debbugs", IMHO.
I mean it could make sense to have Fixes for "real" bugs and Addresses
for anything else tracked by debbugs (whishlist, typo, etc.).

Cheers,
simon
Closed
?