Package GNU MediaGoblin as a Guix service

  • Open
  • quality assurance status badge
Details
5 participants
  • Dr. Arne Babenhauserheide
  • Ben Sturmfels
  • Christopher Lemmer Webber
  • jgart
  • Léo Le Bouter
Owner
unassigned
Submitted by
Ben Sturmfels
Severity
normal
B
B
Ben Sturmfels wrote on 19 Mar 2021 13:20
(address . bug-guix@gnu.org)
87eegbz65q.fsf@sturm.com.au
This is a "meta" bug to keep track of the progress of packaging GNU
MediaGoblin, a platform for publishing images/audio/video etc. See

We have a guix-env.scm in the upstream source which should always have
the latest copy of our packaging progress and instructions to run it:


Current plan is:

1. Add OGG support to libsndfile which is needed to package
python-soundfile [patch 47210]

2. Package python-soundfile (see above). After this the test suite
should pass 100% with pytest installed from PyPI [patch 47181]

3. Work out why python-pytest-6/python-pytest-xdist/python-pytest-forked
in Guix seem to be incompatible. After this our test suite should run
100% with only dependencies from Guix!

4. Package MediaGoblin itself. The build process is ./configure/make
which is a bit weird for a Python project.

5. Get a basic Guix service working, with sqlite3 and without the
offloaded media transcoding currently using Celery/RabbitMQ.

6. Rewrite MediaGoblin's JavaScript code not to use jQuery. Maybe
improve the no-bundled-JavaScript video/audio playing experience.

7. Work out why H264 support is missing.

8. Either package RabbitMQ (probably hard) or rewrite MediaGoblin's
processing backend from Celery/RabbitMQ to RQ/Redis. Celery has been
implicated in many bugs anyway, so there may benefits to the project to
doing this anyway.

9. Figure out how to deal with translations.

10. Add a PostgreSQL database to the Guix service instead of sqlite3.

11. We win. Maybe :)
-----BEGIN PGP SIGNATURE-----

iQJFBAEBCgAvFiEEPn825zvdanEG+SAhAjwF4snAaPAFAmBUlxERHGJlbkBzdHVy
bS5jb20uYXUACgkQAjwF4snAaPCddA/9Fy2keoy060sfHYzWy4cFW5X39EbwTUht
VBKRZcGWXnAE2etyMrhTfTeMCFx/9TD0AihVfSarij6m7xfGCLBZYbQYSeeO6DcP
tgRqJjSZ/cU6I00c6rnUbOXqp4C5olZu7ZolUiydMs4hJI0ubhGx13R9qhKEyfgy
YLwyQzyftzu3AodrRcH4ijBByuG7GlQCwm3wHNugI7jOrEsGQVHnLwkuI56V6ufh
fk+XpWaM4UpdPnVk0NYe5yeX1394b3H2hOf2mMa2GIHLAYs4rggWoXxbsQjA1x7t
C4vYYd9WgORTwrTKRSPh9bDuPfDuEs4uZcD2MI+mOTw+XgiyDGMLMatavbl1sZC6
/LPZ57O8QQlw9yKJTqFh/XLHbA3UGqwb6Fh4GKGfxlbgVHBZu8mGKdSnPb1xW0tk
FZGMhulyod5k00e2gMLVVlvk+rm2qEvjgOnL1uzal0iowowFX7t7fWkCh429Xq1q
vxaWSl26jc24FI8C1O/fdY/yOopksppPAmSgkLOWs5Cc9wwdvF5eD3RRRPBIBpae
oK2PRvpRRJBz+MHJxI33oSoLbCpelsDYun+rL4Cm2bXqXNxt/OD/Q+R1DSmQoRVi
LkMLzPYC9G15ahZIr8z1Hh96C4uAij+xtMNz7korneox19w9FuGEl7qCaJm63cet
wk/CT6bsG8s=
=7l7S
-----END PGP SIGNATURE-----

J
(address . 47260@debbugs.gnu.org)
8ce8fb88252149eb789a07514c8419f7@dismail.de
This sounds like a great project. I would love MediaGoblin to be in Guix also.

Toggle quote (3 lines)
> 6. Rewrite MediaGoblin's JavaScript code not to use jQuery. Maybe
> improve the no-bundled-JavaScript video/audio playing experience.

What are your thoughts on rewriting the jquery?

Should MediaGoblin be using vanilla javascript instead?

Some other possibilities could be purescript (https://www.purescript.org) or mint (http://mint-lang.com), although mint and crystal are not in guix yet and mint uses preact (http://preactjs.com) as its' runtime since 0.8.0 (https://www.mint-lang.com/blog/mint-0.8.0).
B
B
Ben Sturmfels wrote on 22 Mar 2021 00:28
Re: bug#47260: Package GNU MediaGoblin as a Guix service
(name . jgart)(address . jgart@dismail.de)(address . 47260@debbugs.gnu.org)
87mtuw9jdp.fsf@sturm.com.au
On Fri, 19 Mar 2021, jgart wrote:

Toggle quote (14 lines)
> This sounds like a great project. I would love MediaGoblin to be in Guix also.
>
>> 6. Rewrite MediaGoblin's JavaScript code not to use jQuery. Maybe
>> improve the no-bundled-JavaScript video/audio playing experience.
>
> What are your thoughts on rewriting the jquery?
>
> Should MediaGoblin be using vanilla javascript instead?
>
> Some other possibilities could be purescript (https://www.purescript.org) or
> mint (http://mint-lang.com), although mint and crystal are not in guix yet and
> mint uses preact (http://preactjs.com) as its' runtime since 0.8.0
> (https://www.mint-lang.com/blog/mint-0.8.0).

Many of the functions we used to use jQuery for are now built into most
browsers from the last 10 years.

By far the most common are element selectors like:

var panel = $('#header-panel')
var arrow = $('.arrow')

which we just change to:

var panel = document.querySelector('#header-panel')
var arrow = document.querySelector('.arrow')

This is all vanilla JavaScript in individual files imported via <script>
without any sort of compilation process. This seems to be the simplest
path to get into distributions like Guix and Debian. The main problem
with purescript and others is that they require more dependencies, build
steps and tooling and are known by less people.

We have a few dependencies which will be harder to deal with such as the
video, audio and 3D model viewers. In the first instance, I think the
answer will be to do without this JavaScript entirely. That should be
feasible for video and audio given modern browser support. Perhaps we
don't ship 3D model support out of the box in distros for now.

Regards,
Ben
D
D
Dr. Arne Babenhauserheide wrote on 22 Mar 2021 08:02
(name . Ben Sturmfels)(address . ben@sturm.com.au)
87lfafn00y.fsf@web.de
Ben Sturmfels via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (41 lines)
> On Fri, 19 Mar 2021, jgart wrote:
>
>> This sounds like a great project. I would love MediaGoblin to be in Guix also.
>>
>>> 6. Rewrite MediaGoblin's JavaScript code not to use jQuery. Maybe
>>> improve the no-bundled-JavaScript video/audio playing experience.
>>
>> What are your thoughts on rewriting the jquery?
>>
>> Should MediaGoblin be using vanilla javascript instead?
>>
>> Some other possibilities could be purescript (https://www.purescript.org) or
>> mint (http://mint-lang.com), although mint and crystal are not in guix yet and
>> mint uses preact (http://preactjs.com) as its' runtime since 0.8.0
>> (https://www.mint-lang.com/blog/mint-0.8.0).
>
> Many of the functions we used to use jQuery for are now built into most
> browsers from the last 10 years.
>
> By far the most common are element selectors like:
>
> var panel = $('#header-panel')
> var arrow = $('.arrow')
>
> which we just change to:
>
> var panel = document.querySelector('#header-panel')
> var arrow = document.querySelector('.arrow')
>
> This is all vanilla JavaScript in individual files imported via <script>
> without any sort of compilation process. This seems to be the simplest
> path to get into distributions like Guix and Debian. The main problem
> with purescript and others is that they require more dependencies, build
> steps and tooling and are known by less people.
>
> We have a few dependencies which will be harder to deal with such as the
> video, audio and 3D model viewers. In the first instance, I think the
> answer will be to do without this JavaScript entirely. That should be
> feasible for video and audio given modern browser support. Perhaps we
> don't ship 3D model support out of the box in distros for now.

If you need support for m3u-playlists, you can use the player I wrote
(that m3u-playlists aren’t supported out of the box in most players is a
strange oversight, the code adds it for video- and audio-tags, License:
GPLv2 or later — just ask me if you need something else)

There’s also an enhanced version for Freenet, but that has lots of
performance-changes to work over high-latency networks and with paranoid
CSP-settings:

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken
-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEE801qEjXQSQPNItXAE++NRSQDw+sFAmBYQSAQHGFybmVfYmFi
QHdlYi5kZQAKCRAT741FJAPD66bnEACT0w91V3N1nhS8N56+9Kz2Z5wBRHpgdqCu
aqaVSGxzPNRuDqUA4VMOw4/dFdelLNaiUXuNHXqrSXd1yh+RRQed+v71h0RZk+E5
DM+1Ef9pBglrfS3ESukyUfWVSri1ekDbHkaK687JVmjphCDRuPUCPYDe78X4RydQ
yvvjDf7NKVa9M5Kb/taJ5Il9goyP+IadnOY1nzR8ccGZq2hUF1SahM0xIQS7ne++
irgPhioeLB4lMfmLQmNG3p2UmpqbDQ9Xvb6/Unv9PpELDFOmIOl/Xz9xVT9SJt5v
Y8eQx2Ah3VCiw/Dz5QoRrKZIAI4KgOpGWo46B41H0A9vPqP0l43wQ4h0q3Uy9Rg/
NxAYTMN3vLo7SEYuwEoJ2wtD2bfZgteZkc71jpr1mO630GwCvNvUF4oUe7frvUMd
qRcnyKaU/9OSAQEgxiHaTIXf5exkat+Ihn7a5sja590/ccS5apuvc/tZphApVlOl
MITpUraq88MbcYj29Jhiy9wKyo+CAexJ8ZYm8lEQozbI3VnP5wTRM0y8n7hwgDT8
MeWnF19VuqTT7wLo6DqWPlS1KUYdLYFEkQzrIwb+2toPjfPQfl0rNFU3UHapq3Kf
xfXr7F4uY/9z5nXZaSVh5MWRyNbk7/IAIxCnhW9/QRaFMnknNlb+OoOTAHjOkrWe
rkT/WLYlZojEBAEBCAAuFiEE3Si95tmHXKvOSosd3M8NswvBBUgFAmBYQSAQHGFy
bmVfYmFiQHdlYi5kZQAKCRDczw2zC8EFSJDqA/4x161lHDcd8vjBptBjc2pmvW18
7g8OQkGzNE7pNVxLyyWyIMRZJNUW7GuwAg5EwVe7F4FUrOAEbCL6C+BVXXX/GF1H
Nr2lGyfpTblE7Uqpzgig/ia29bLKYOwHrl9gkUQT5Gpxz6rz29d4SZgNuY1s1Izl
Tzu9Di/xUNDoeO7L8w==
=hrwr
-----END PGP SIGNATURE-----

C
C
Christopher Lemmer Webber wrote on 22 Mar 2021 18:58
(name . Ben Sturmfels)(address . ben@sturm.com.au)
874kh3m5ot.fsf@dustycloud.org
Ben Sturmfels via Bug reports for GNU Guix writes:

Toggle quote (44 lines)
> On Fri, 19 Mar 2021, jgart wrote:
>
>> This sounds like a great project. I would love MediaGoblin to be in Guix also.
>>
>>> 6. Rewrite MediaGoblin's JavaScript code not to use jQuery. Maybe
>>> improve the no-bundled-JavaScript video/audio playing experience.
>>
>> What are your thoughts on rewriting the jquery?
>>
>> Should MediaGoblin be using vanilla javascript instead?
>>
>> Some other possibilities could be purescript (https://www.purescript.org) or
>> mint (http://mint-lang.com), although mint and crystal are not in guix yet and
>> mint uses preact (http://preactjs.com) as its' runtime since 0.8.0
>> (https://www.mint-lang.com/blog/mint-0.8.0).
>
> Many of the functions we used to use jQuery for are now built into most
> browsers from the last 10 years.
>
> By far the most common are element selectors like:
>
> var panel = $('#header-panel')
> var arrow = $('.arrow')
>
> which we just change to:
>
> var panel = document.querySelector('#header-panel')
> var arrow = document.querySelector('.arrow')
>
> This is all vanilla JavaScript in individual files imported via <script>
> without any sort of compilation process. This seems to be the simplest
> path to get into distributions like Guix and Debian. The main problem
> with purescript and others is that they require more dependencies, build
> steps and tooling and are known by less people.
>
> We have a few dependencies which will be harder to deal with such as the
> video, audio and 3D model viewers. In the first instance, I think the
> answer will be to do without this JavaScript entirely. That should be
> feasible for video and audio given modern browser support. Perhaps we
> don't ship 3D model support out of the box in distros for now.
>
> Regards,
> Ben

The 3d model support isn't MediaGoblin's most heavily used plugin. Not
shipping support for that would be fine, I'd think.

- Chris
B
B
Ben Sturmfels wrote on 30 Mar 2021 06:02
(name . Dr. Arne Babenhauserheide)(address . arne_bab@web.de)(address . 47260@debbugs.gnu.org)
87r1jxqog0.fsf@sturm.com.au
On Mon, 22 Mar 2021, Dr. Arne Babenhauserheide wrote:

Toggle quote (12 lines)
> If you need support for m3u-playlists, you can use the player I wrote
> here: https://www.draketo.de/software/m3u-player
> → https://www.draketo.de/software/m3u-player.js (save as utf-8)
> (that m3u-playlists aren’t supported out of the box in most players is a
> strange oversight, the code adds it for video- and audio-tags, License:
> GPLv2 or later — just ask me if you need something else)
>
> There’s also an enhanced version for Freenet, but that has lots of
> performance-changes to work over high-latency networks and with paranoid
> CSP-settings:
> https://github.com/freenet/fred/pull/721/files#diff-33cbf95723ae7b33eb205cf9adc3411b2098e27ba757e553406f689a4fafb802

Thanks Arne! I've forwarded this on to mediagoblin-devel@gnu.org so we
don't lose track of it.

Regards,
Ben
B
B
Ben Sturmfels wrote on 30 Mar 2021 06:12
(address . 47260@debbugs.gnu.org)
87k0ppqny7.fsf@sturm.com.au
Just an update:

On Fri, 19 Mar 2021, Ben Sturmfels wrote:

Toggle quote (3 lines)
> 1. Add OGG support to libsndfile which is needed to package
> python-soundfile [patch 47210]

lfam pointed out that this has already been done in 46067, which is now
in core-updates awaiting merging into master:


For the time being, we've also copied the updated libsndfile into
MediaGoblin's guix-env.scm.

Toggle quote (3 lines)
> 2. Package python-soundfile (see above). After this the test suite
> should pass 100% with pytest installed from PyPI [patch 47181]

We now have a local copy of python-soundfile in MediaGoblin's
guix-env.scm which is passing all audio tests. Yay!

Toggle quote (4 lines)
> 3. Work out why python-pytest-6/python-pytest-xdist/python-pytest-forked
> in Guix seem to be incompatible. After this our test suite should run
> 100% with only dependencies from Guix!

Discovered we'll also need to upgrade Guix's python-wtforms, but in the
mean time, installing only wtforms, pytest, pytest-xdist, pytest-forked
from PyPI allows us to pass the test suite 100% Getting closer!

Toggle quote (6 lines)
> ...
> 8. Either package RabbitMQ (probably hard) or rewrite MediaGoblin's
> processing backend from Celery/RabbitMQ to RQ/Redis. Celery has been
> implicated in many bugs anyway, so there may benefits to the project to
> doing this anyway.

I learnt that Celery has a Redis backend, so maybe we don't need to
rewrite just yet.

Regards,
Ben
D
D
Dr. Arne Babenhauserheide wrote on 30 Mar 2021 08:40
(name . Ben Sturmfels)(address . ben@sturm.com.au)(address . 47260@debbugs.gnu.org)
87lfa55ekz.fsf@web.de
Ben Sturmfels <ben@sturm.com.au> writes:

Toggle quote (17 lines)
> On Mon, 22 Mar 2021, Dr. Arne Babenhauserheide wrote:
>
>> If you need support for m3u-playlists, you can use the player I wrote
>> here: https://www.draketo.de/software/m3u-player
>> → https://www.draketo.de/software/m3u-player.js (save as utf-8)
>> (that m3u-playlists aren’t supported out of the box in most players is a
>> strange oversight, the code adds it for video- and audio-tags, License:
>> GPLv2 or later — just ask me if you need something else)
>>
>> There’s also an enhanced version for Freenet, but that has lots of
>> performance-changes to work over high-latency networks and with paranoid
>> CSP-settings:
>> https://github.com/freenet/fred/pull/721/files#diff-33cbf95723ae7b33eb205cf9adc3411b2098e27ba757e553406f689a4fafb802
>
> Thanks Arne! I've forwarded this on to mediagoblin-devel@gnu.org so we
> don't lose track of it.

Thank you!

I added one change last week to support mobile browsers which answer
"maybe" to the query `mediaTag.canPlayType('audio/x-mpegurl')` (yes,
seriously, and it is in the spec :-) ).

Also I backported the not freenet specific changes:
- prefetch the next three tracks as blob and keep at most 10 tracks
cached to allow for fast track skipping (and now actually release the
memory)
- adjustments to allow for inlining and survive the non-utf8-encoding.
- continue automatically when fetch succeeded if playback was stopped
because it reached the end (but not if paused).
- minimal mouseover for the back and forward arrows.

When a https-m3u-list refers to a http-file, it falls back from fetching
blobs to rewriting the src-part of the tag (because blobs cannot be
fetched from a less secure resource).



You can use it like this:

<script src="m3u-player.js" defer="defer"></script>
<audio src="m3u-player-example-playlist.m3u" controls="controls">
not supported?
</audio>


To make this bug-report independent of my site, here’s the full code:



// [[file:m3u-player.org::*The script][The script:1]]
// @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2-or-Later
const nodes = document.querySelectorAll("audio,video");
const playlists = {};
const prefetchedTracks = new Map(); // use a map for insertion order, so we can just blow away old entries.
// maximum prefetched blobs that are kept.
const MAX_PREFETCH_KEEP = 10;
// maximum allowed number of entries in a playlist to prevent OOM attacks against the browser with self-referencing playlists
const MAX_PLAYLIST_LENGTH = 1000;
const PLAYLIST_MIME_TYPES = ["audio/x-mpegurl", "audio/mpegurl", "application/vnd.apple.mpegurl","application/mpegurl","application/x-mpegurl"];
function stripUrlParameters(link) {
const url = new URL(link, window.location);
url.search = "";
url.hash = "";
return url.href;
}
function isPlaylist(link) {
const linkHref = stripUrlParameters(link);
return linkHref.endsWith(".m3u") || linkHref.endsWith(".m3u8");
}
function isBlob(link) {
return new URL(link, window.location).protocol == 'blob';
}
function parsePlaylist(textContent) {
return textContent.match(/^(?!#)(?!\s).*$/mg)
.filter(s => s); // filter removes empty strings
}
/**
* Download the given playlist, parse it, and store the tracks in the
* global playlists object using the url as key.
*
* Runs callback once the playlist downloaded successfully.
*/
function fetchPlaylist(url, onload, onerror) {
const playlistFetcher = new XMLHttpRequest();
playlistFetcher.open("GET", url, true);
playlistFetcher.responseType = "blob"; // to get a mime type
playlistFetcher.onload = () => {
if (PLAYLIST_MIME_TYPES.includes(playlistFetcher.response.type)) { // security check to ensure that filters have run
const reader = new FileReader();
const load = onload; // propagate to inner scope
reader.addEventListener("loadend", e => {
playlists[url] = parsePlaylist(reader.result);
onload();
});
reader.readAsText(playlistFetcher.response);
} else {
console.error("playlist must have one of the playlist MIME type '" + PLAYLIST_MIME_TYPES + "' but it had MIME type '" + playlistFetcher.response.type + "'.");
onerror();
}
};
playlistFetcher.onerror = onerror;
playlistFetcher.abort = onerror;
playlistFetcher.send();
}
function prefetchTrack(url, onload) {
if (prefetchedTracks.has(url)) {
return;
}
// first cleanup: kill the oldest entries until we're back at the allowed size
while (prefetchedTracks.size > MAX_PREFETCH_KEEP) {
const key = prefetchedTracks.keys().next().value;
const track = prefetchedTracks.get(key);
prefetchedTracks.delete(key);
}
// first set the prefetched to the url so we will never request twice
prefetchedTracks.set(url, url);
// now start replacing it with a blob
const xhr = new XMLHttpRequest();
xhr.open("GET", url, true);
xhr.responseType = "blob";
xhr.onload = () => {
prefetchedTracks.set(url, xhr.response);
if (onload) {
onload();
}
};
xhr.send();
}
function updateSrc(mediaTag, callback) {
const playlistUrl = mediaTag.getAttribute("playlist");
const trackIndex = mediaTag.getAttribute("track-index");
// deepcopy playlists to avoid shared mutation
let playlist = [...playlists[playlistUrl]];
let trackUrl = playlist[trackIndex];
// download and splice in playlists as needed
if (isPlaylist(trackUrl)) {
if (playlist.length >= MAX_PLAYLIST_LENGTH) {
// skip playlist if we already have too many tracks
changeTrack(mediaTag, +1);
} else {
// do not use the cached playlist here, though it is tempting: it might genuinely change to allow for updates
fetchPlaylist(
trackUrl,
() => {
playlist.splice(trackIndex, 1, ...playlists[trackUrl]);
playlists[playlistUrl] = playlist;
updateSrc(mediaTag, callback);
},
() => callback());
}
} else {
let url = prefetchedTracks.has(trackUrl)
? prefetchedTracks.get(trackUrl) instanceof Blob
? URL.createObjectURL(prefetchedTracks.get(trackUrl))
: trackUrl : trackUrl;
const oldUrl = mediaTag.getAttribute("src");
mediaTag.setAttribute("src", url);
// replace the url when done, because a blob from an xhr request
// is more reliable in the media tag;
// the normal URL caused jumping prematurely to the next track.
if (url == trackUrl) {
prefetchTrack(trackUrl, () => {
if (mediaTag.paused) {
if (url == mediaTag.getAttribute("src")) {
if (mediaTag.currentTime === 0) {
mediaTag.setAttribute("src", URL.createObjectURL(
prefetchedTracks.get(url)));
}
}
}
});
}
// allow releasing memory
if (isBlob(oldUrl)) {
URL.revokeObjectURL(oldUrl);
}
// update title
mediaTag.parentElement.querySelector(".m3u-player--title").title = trackUrl;
mediaTag.parentElement.querySelector(".m3u-player--title").textContent = trackUrl;
// start prefetching the next three tracks.
for (const i of [1, 2, 3]) {
if (playlist.length > Number(trackIndex) + i) {
prefetchTrack(playlist[Number(trackIndex) + i]);
}
}
callback();
}
}
function changeTrack(mediaTag, diff) {
const currentTrackIndex = Number(mediaTag.getAttribute("track-index"));
const nextTrackIndex = currentTrackIndex + diff;
const tracks = playlists[mediaTag.getAttribute("playlist")];
if (nextTrackIndex >= 0) { // do not collapse the if clauses with double-and, that does not survive inlining
if (tracks.length > nextTrackIndex) {
mediaTag.setAttribute("track-index", nextTrackIndex);
updateSrc(mediaTag, () => mediaTag.play());
}
}
}

/**
* Turn a media tag into playlist player.
*/
function initPlayer(mediaTag) {
mediaTag.setAttribute("playlist", mediaTag.getAttribute("src"));
mediaTag.setAttribute("track-index", 0);
const url = mediaTag.getAttribute("playlist");
const wrapper = mediaTag.parentElement.insertBefore(document.createElement("div"), mediaTag);
const controls = document.createElement("div");
const left = document.createElement("span");
const title = document.createElement("span");
const right = document.createElement("span");
controls.appendChild(left);
controls.appendChild(title);
controls.appendChild(right);
left.classList.add("m3u-player--left");
right.classList.add("m3u-player--right");
title.classList.add("m3u-player--title");
title.style.overflow = "hidden";
title.style.textOverflow = "ellipsis";
title.style.whiteSpace = "nowrap";
title.style.opacity = "0.3";
title.style.direction = "rtl"; // for truncation on the left
title.style.paddingLeft = "0.5em";
title.style.paddingRight = "0.5em";
controls.style.display = "flex";
controls.style.justifyContent = "space-between";
const styleTag = document.createElement("style");
styleTag.innerHTML = ".m3u-player--left:hover, .m3u-player--right:hover {color: wheat; background-color: DarkSlateGray}";
wrapper.appendChild(styleTag);
wrapper.appendChild(controls);
controls.style.width = mediaTag.getBoundingClientRect().width.toString() + "px";
// appending the media tag to the wrapper removes it from the outer scope but keeps the event listeners
wrapper.appendChild(mediaTag);
left.innerHTML = "&lt;"; // not textContent, because we MUST escape
// the tag here and textContent shows the
// escaped version
left.onclick = () => changeTrack(mediaTag, -1);
right.innerHTML = "&gt;";
right.onclick = () => changeTrack(mediaTag, +1);
fetchPlaylist(
url,
() => {
updateSrc(mediaTag, () => null);
mediaTag.addEventListener("ended", event => {
if (mediaTag.currentTime >= mediaTag.duration) {
changeTrack(mediaTag, +1);
}
});
},
() => null);
// keep the controls aligned to the media tag
mediaTag.resizeObserver = new ResizeObserver(entries => {
controls.style.width = entries[0].contentRect.width.toString() + "px";
});
mediaTag.resizeObserver.observe(mediaTag);
}
function processTag(mediaTag) {
const canPlayClaim = mediaTag.canPlayType('audio/x-mpegurl');
let supportsPlaylists = !!canPlayClaim;
if (canPlayClaim == 'maybe') { // yes, seriously: specced as you only know when you try
supportsPlaylists = false;
}
if (!supportsPlaylists) {
if (isPlaylist(mediaTag.getAttribute("src"))) {
initPlayer(mediaTag);
}
}
}
document.addEventListener('DOMContentLoaded', () => {
const nodes = document.querySelectorAll("audio,video");
nodes.forEach(processTag);
});
// @license-end
// The script:1 ends here




Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken
-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEE801qEjXQSQPNItXAE++NRSQDw+sFAmBix+4QHGFybmVfYmFi
QHdlYi5kZQAKCRAT741FJAPD62HfEACj05Xrkc+b59LGbHm50jqhuYdHRJy6yXD2
H1KMGt+9DjySbIzMDvELnAMXmSlAnfrMgunzXn8Ky7TGM1Vbjn43s9BwPYfMwcMm
KnQpRxw2yMBLmg2kCk6yuYDopj6dABD4qmOQs3XOjoZ/H1G9TqdMB3g5LkHj/jlE
ypGXkMHVTgsG30NhprO4p5N45AvGwtRf2WkN2HoJiv29chjQzA0CMWDYezYZrTyw
JDAEwWuwEDa01zskMJcp6avZZloPV7d5CUif2QCYUllKcQObfSsdnbQhVRe1R1CU
8+evfCer5uKvdrlfI0gTqTkg7ic/jZNEZAM55jawi4Pfui4dkGHfeJRirP8vG1pB
dsVF3vtUjq3omQrk1w8fKlhrhhuxapJ5/E24jmg8FrtSug5NrIiKVfTJaY247KoC
CW9xiOaepc4PWlivaoBJrVZFLUsGeTXf+sEWb9pF5j2JLTRk/pSIJJzYYWPtGOUT
MvKiF1JAXr9qy4EU28zWkPQhPppJ1YO3i4TcaBoZZbGtwTslbDjIGE7yss/m+mEv
LyJQ4YGMposXSNUgMrgIFoRV6DGVvl9WOFhz5ebmXo0jLfIMDQ6OWZa5TEnZNiCS
q3iB25HpwrcEb5u9EG1kw2NcO5ICHVKgcor7C0jB9FukhOu6EEjQG7hw/s8r4sdy
2wST8dc804jEBAEBCAAuFiEE3Si95tmHXKvOSosd3M8NswvBBUgFAmBix+4QHGFy
bmVfYmFiQHdlYi5kZQAKCRDczw2zC8EFSO9CA/4i6k14FE2X2EUzw+nH2k4vWqjP
BzhLUrbk8wSsWdBLLUQBy7OKTwrwSnFgpSNu+cg7QxU9FKuK3YO8ofSl0/hwhEz8
4mOHWtiJy9p83fRsOVdqSvq0du64qXx9a0aY9RdSchbcARGbCtGRbNIH514B/mNn
fs98CJmES7Yo37WvvA==
=t966
-----END PGP SIGNATURE-----

B
B
Ben Sturmfels wrote on 30 Mar 2021 14:13
(address . 47260@debbugs.gnu.org)
87y2e4q1pp.fsf@sturm.com.au
On Tue, 30 Mar 2021, Ben Sturmfels wrote:

Toggle quote (8 lines)
>> 3. Work out why python-pytest-6/python-pytest-xdist/python-pytest-forked
>> in Guix seem to be incompatible. After this our test suite should run
>> 100% with only dependencies from Guix!
>
> Discovered we'll also need to upgrade Guix's python-wtforms, but in the
> mean time, installing only wtforms, pytest, pytest-xdist, pytest-forked
> from PyPI allows us to pass the test suite 100% Getting closer!

Pytest issues have now been resolved. We now need ZERO packages from
PyPI for a basic install and full test suite run. :)

Regards,
Ben
B
B
Ben Sturmfels wrote on 1 Apr 2021 04:03
(address . 47260@debbugs.gnu.org)
87tuoqsqw3.fsf@sturm.com.au
On Tue, 30 Mar 2021, Ben Sturmfels wrote:

Toggle quote (10 lines)
> On Fri, 19 Mar 2021, Ben Sturmfels wrote:

>> 8. Either package RabbitMQ (probably hard) or rewrite MediaGoblin's
>> processing backend from Celery/RabbitMQ to RQ/Redis. Celery has been
>> implicated in many bugs anyway, so there may benefits to the project to
>> doing this anyway.
>
> I learnt that Celery has a Redis backend, so maybe we don't need to
> rewrite just yet.

It turns out that MediaGoblin's Celery-based media processing backend
work out of the box by simply configuring:

[celery]
BROKER_URL = "redis://"

(There seems to be an unrelated bug where media is marked as failed after
restarting Celery, possibly tied to sqlite. We've had reports of this
with a RabbitMQ broker too though.)


This means our shorter to-do list is now:

1. Upstream our new python-soundfile Guix package from guix-env.scm when
core-updates is merged.

2. Upstream our upgraded python-wtforms package.

6. Convert MediaGoblin's jQuery-based JavaScript to use vanilla JS.
Video and audio are essentially functional without the NPM installed
players. Some later refinements perhaps.

4. Package MediaGoblin itself. The build process is ./configure/make
which is a bit weird for a Python project.

5. Get a basic Guix service working, with sqlite3 and without the
offloaded media transcoding currently using Celery task queue with a
Redis broker.

7. Work out why H264 support is missing.

8. Figure out how to deal with translations.

9. Add a PostgreSQL database to the Guix service instead of sqlite3.

Regards,
Ben
B
B
Ben Sturmfels wrote on 5 Apr 2021 16:17
(address . 47260@debbugs.gnu.org)
871rbon7dk.fsf@sturm.com.au
On Thu, 01 Apr 2021, Ben Sturmfels wrote:

Toggle quote (2 lines)
> 7. Work out why H264 support is missing.

This is now fixed MediaGoblin's master branch guix-env.scm by adding
gst-libav to propagated inputs.
L
L
Léo Le Bouter wrote on 5 Apr 2021 17:50
9a64a244a3692342389cc8e1ce998cacfc6ea8b5.camel@zaclys.net
On Tue, 2021-04-06 at 00:17 +1000, Ben Sturmfels wrote:
Toggle quote (7 lines)
> On Thu, 01 Apr 2021, Ben Sturmfels wrote:
>
> > 7. Work out why H264 support is missing.
>
> This is now fixed MediaGoblin's master branch guix-env.scm by adding
> gst-libav to propagated inputs.

Hello!

I suggest not using propagated-inputs because they are likely to cause
conflicts in profiles when used. I suggest intead creating wrapper
scripts that append to the PATH variable.

Examples of such wrapper scripts can be found:

-
(PYTHONPATH more so)
-
(Append xdg-utils to PATH)

Let me know if there's any questions about creating such wrappers.

Léo
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEFIvLi9gL+xax3g6RRaix6GvNEKYFAmBrMdEACgkQRaix6GvN
EKZ3TQ//S0ncDrMzatdhXyGW/u+PHoY+pHX+s6FKQ1K1uxC78K1MVH95af8EHBH8
Ah7e5ZNLNzTIPBv54nW9fe/YuaDO35NAM/r6Vd5743HSA5zxPe4hNm4G5bnoN25Z
yf7dXV3WkP5YguM82l4TMfto+Byn/6xwpRdJxWY0vmK4E+EJ8Ossr1gG8zNQ4az0
3c0bX8rwfk6w1XjdptuvxeNAa5Kos1FzyEgKad0O3431kWfQtFVTK3ZouG1P8q+n
hlK3l2k7xvg/xc43hepeQB+YZfyCDAgSxilJ+vN9vdvuioh5VEejrNNUcmdWXRfY
MmopYJ0pS4rPsGHpvOK4kAc52+VphFWsrkfZm5mT8zphwklYNAKQ7ju6pgsphOeJ
1YLMLAJCzMyEWkbwu1tlBu7VrkZMAQGuHuOM2YzoDiLhCS9rAP7pN0/PRS8uQg57
fsjdcir8hB5zC/zd8QflVHfElkCmolTeHYP0TA9JXqKy41cfxSba3fXmtaFPNu68
UiC+QdKLm0FwROxZ8ZndX8lRwyZW1B6zSTwWgK9NRmE/7maxbm+1k7vAcRu4MhP5
asyaCkYA99h3qwrVPomz6UHE4NJOMRwQHcaiG0MMmM8tegtKzdgANtfQtGsrILK0
OvbMnZCI/m/13uK2x5K17Ghko2CekvIGbqGv9UhDi8tBgBXJz60=
=A3qa
-----END PGP SIGNATURE-----


B
B
Ben Sturmfels wrote on 6 Apr 2021 14:01
(address . 47260@debbugs.gnu.org)
87wntfsjt7.fsf@sturm.com.au
On Thu, 01 Apr 2021, Ben Sturmfels wrote:

Toggle quote (4 lines)
> 5. Get a basic Guix service working, with sqlite3 and without the
> offloaded media transcoding currently using Celery task queue with a
> Redis broker.

Woo! After a lot of trial and error, I finally have a basic MediaGoblin
running entirely under Guix with no virtualenv trickery!

After applying the attached patch to my guix repo, I grab a copy of the
basic config files and enable audio and video:

echo "[[mediagoblin.media_types.audio]]" >> mediagoblin.ini
echo "[[mediagoblin.media_types.video]]" >> mediagoblin.ini

Build MediaGoblin, which downloads from our master branch and runs the
full test suite successfully:

~/ws/guix/pre-inst-env guix build mediagoblin

Then install MediaGoblin in a container (not working in a non-container
guix environment or without explicit "python"):

~/ws/guix/pre-inst-env guix environment --container --network --share=$HOME/.bash_history --ad-hoc mediagoblin python

Create an sqlite3 database and add a user:

gmg dbupdate
gmg adduser --username admin --password a --email admin@example.com
gmg changepw admin a

Upload an image, audio and video via CLI:

gmg addmedia admin image.jpg
gmg addmedia admin audio.wav
gmg addmedia admin video.mp4

Start the web interface:

CELERY_ALWAYS_EAGER=true paster serve paste.ini

The web interface is working. Looks like we're missing some CSS
(probably due to files not being included in the setuptools package),
but that's a minor issue.

Getting there!

Regards,
Ben
B
B
Ben Sturmfels wrote on 6 Apr 2021 14:05
(name . Léo Le Bouter)(address . lle-bout@zaclys.net)
87tuojsjnk.fsf@sturm.com.au
On Mon, 05 Apr 2021, Léo Le Bouter wrote:

Toggle quote (25 lines)
> On Tue, 2021-04-06 at 00:17 +1000, Ben Sturmfels wrote:
>> On Thu, 01 Apr 2021, Ben Sturmfels wrote:
>>
>> > 7. Work out why H264 support is missing.
>>
>> This is now fixed MediaGoblin's master branch guix-env.scm by adding
>> gst-libav to propagated inputs.
>
> Hello!
>
> I suggest not using propagated-inputs because they are likely to cause
> conflicts in profiles when used. I suggest intead creating wrapper
> scripts that append to the PATH variable.
>
> Examples of such wrapper scripts can be found:
>
> -
> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=373e5fc96724fd38bb1263e4af90932ea36f596b
> (PYTHONPATH more so)
> -
> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=00c1793ce8e2210e48b18422ea3e76da10541874
> (Append xdg-utils to PATH)
>
> Let me know if there's any questions about creating such wrappers.

Thanks Léo, I'll look into this!

Regards,
Ben
B
B
Ben Sturmfels wrote on 7 Apr 2021 15:15
(address . 47260@debbugs.gnu.org)
87r1jms0aa.fsf@sturm.com.au
On Tue, 06 Apr 2021, Ben Sturmfels wrote:

Toggle quote (11 lines)
> On Thu, 01 Apr 2021, Ben Sturmfels wrote:
>
>> 5. Get a basic Guix service working, with sqlite3 and without the
>> offloaded media transcoding currently using Celery task queue with a
>> Redis broker.
>
> Woo! After a lot of trial and error, I finally have a basic MediaGoblin
> running entirely under Guix with no virtualenv trickery!
>
> After applying the attached patch to my guix repo...

Even simpler, I've now created a Guix channel for our work-in-progress
MediaGoblin package/service. See the README for instructions on enabling
the channel and installing MediaGoblin:


Once the channel is enabled, you can `guix install mediagoblin` and run
`gmg` commands and the web interface. All documented in the above README.

Regards,
Ben
D
D
Dr. Arne Babenhauserheide wrote on 4 May 2021 22:58
(name . Ben Sturmfels)(address . ben@sturm.com.au)
87y2cu2p47.fsf@web.de
Hi,

I just added non-flickering video-change to the m3u-player. Attaching
the file. I thought that could be useful for MediaGoblin. The file is
attached.
Attachment: m3u-player.js
Best wishes,
Arne


Ben Sturmfels via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (51 lines)
> On Tue, 30 Mar 2021, Ben Sturmfels wrote:
>
>> On Fri, 19 Mar 2021, Ben Sturmfels wrote:
>
>>> 8. Either package RabbitMQ (probably hard) or rewrite MediaGoblin's
>>> processing backend from Celery/RabbitMQ to RQ/Redis. Celery has been
>>> implicated in many bugs anyway, so there may benefits to the project to
>>> doing this anyway.
>>
>> I learnt that Celery has a Redis backend, so maybe we don't need to
>> rewrite just yet.
>
> It turns out that MediaGoblin's Celery-based media processing backend
> work out of the box by simply configuring:
>
> [celery]
> BROKER_URL = "redis://"
>
> (There seems to be an unrelated bug where media is marked as failed after
> restarting Celery, possibly tied to sqlite. We've had reports of this
> with a RabbitMQ broker too though.)
>
>
> This means our shorter to-do list is now:
>
> 1. Upstream our new python-soundfile Guix package from guix-env.scm when
> core-updates is merged.
>
> 2. Upstream our upgraded python-wtforms package.
>
> 6. Convert MediaGoblin's jQuery-based JavaScript to use vanilla JS.
> Video and audio are essentially functional without the NPM installed
> players. Some later refinements perhaps.
>
> 4. Package MediaGoblin itself. The build process is ./configure/make
> which is a bit weird for a Python project.
>
> 5. Get a basic Guix service working, with sqlite3 and without the
> offloaded media transcoding currently using Celery task queue with a
> Redis broker.
>
> 7. Work out why H264 support is missing.
>
> 8. Figure out how to deal with translations.
>
> 9. Add a PostgreSQL database to the Guix service instead of sqlite3.
>
> Regards,
> Ben


--
Unpolitisch sein
heißt politisch sein
ohne es zu merken
-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEE801qEjXQSQPNItXAE++NRSQDw+sFAmCRtVsQHGFybmVfYmFi
QHdlYi5kZQAKCRAT741FJAPD6xiQD/sFMA4qFqYt9Rmh0f5WgtBxsB+k3GPDQAoZ
C10DCBPMQt8XkvhdrTYS9L9+PWZmO1sOq8V6OA+6W+pP3aIwoEQeUpoluv8gNdI9
kKFKFS8Sp5gKeh6piD35Rpp3Iq9XjWRBMSVcI/ylz/XDOOzRVgDmw4IgHxDjsRxZ
dmFME2X+HNoz4jIEzLxwiRyVW0QF5ky2u6ZnNwFWEW0x5KJUYnOIVjifxJfvFgAc
aJQDKj2JYTJb2tN2wwis2UKaQ/PZpBsY6plXe+wFHmqvAGRyBv+CW9LVIK87LeYK
U+/spYbBukZS4YaGah5deX3Imum1yq/MFX7iZM7jqeWjkySKZZjNeQNybmeo7iNP
11au/q678pv0TnzN46Sr20tgqLIUFvZBrRuwNS8tzln3YsQXLIjxJWpw6zd3Sz3B
X6GZd5C4ym4aD4JNFd58NjwQDYlQQAuI6g2queUhGQzBsJL7Fa8Qz6Ts61u+TEQa
GFx2zTc6YFzHIYlkS4cklNg0mrAY4wGo16QJiJPwrNC612P5NYtP6JCPVT6pCjPs
5nmqlxniIqd5dWw4H9puX6FT+IeNupSQ2G10Bs4J4nRDaaZ1NKM1tInHN8f05f+9
MMCm4zM536ddXXtrBbuC7O9+LFxrXCf9Lus4E4wvuw1Pi57U5jxHaMzGZB2+poeL
rt477/6ELojEBAEBCAAuFiEE3Si95tmHXKvOSosd3M8NswvBBUgFAmCRtV4QHGFy
bmVfYmFiQHdlYi5kZQAKCRDczw2zC8EFSFDgBACCqyRkasALL4XeTkqxZeMebaNe
6x62njO4+y1Lt+5mqs+moVNJa0LjRVyxd8oocZQ/owGNkmgsi9kozpNiw9mbgf7z
BxOa41/GM6PQna0dCuQwBY2WVSysZB/XwErOI8GZt3+bS5N52Dw0rlQqaNcIS/h3
tov+84bphpjxS10I6g==
=KSx5
-----END PGP SIGNATURE-----

B
B
Ben Sturmfels wrote on 6 May 2021 02:49
(name . Dr. Arne Babenhauserheide)(address . arne_bab@web.de)
87o8doelel.fsf@sturm.com.au
Thanks for the update Arne. This issue is specifically about Guix
packaging, so to save us losing track of your update, please post it to
the dedicated mediagoblin-devel@gnu.org thread we started a couple of
months back:


Thanks again,
Ben

On Wed, 05 May 2021, Arne Babenhauserheide wrote:

Toggle quote (62 lines)
> Hi,
>
> I just added non-flickering video-change to the m3u-player. Attaching
> the file. I thought that could be useful for MediaGoblin. The file is
> attached.
>
>
> Best wishes,
> Arne
>
>
> Ben Sturmfels via Bug reports for GNU Guix <bug-guix@gnu.org> writes:
>
>> On Tue, 30 Mar 2021, Ben Sturmfels wrote:
>>
>>> On Fri, 19 Mar 2021, Ben Sturmfels wrote:
>>
>>>> 8. Either package RabbitMQ (probably hard) or rewrite MediaGoblin's
>>>> processing backend from Celery/RabbitMQ to RQ/Redis. Celery has been
>>>> implicated in many bugs anyway, so there may benefits to the project to
>>>> doing this anyway.
>>>
>>> I learnt that Celery has a Redis backend, so maybe we don't need to
>>> rewrite just yet.
>>
>> It turns out that MediaGoblin's Celery-based media processing backend
>> work out of the box by simply configuring:
>>
>> [celery]
>> BROKER_URL = "redis://"
>>
>> (There seems to be an unrelated bug where media is marked as failed after
>> restarting Celery, possibly tied to sqlite. We've had reports of this
>> with a RabbitMQ broker too though.)
>>
>>
>> This means our shorter to-do list is now:
>>
>> 1. Upstream our new python-soundfile Guix package from guix-env.scm when
>> core-updates is merged.
>>
>> 2. Upstream our upgraded python-wtforms package.
>>
>> 6. Convert MediaGoblin's jQuery-based JavaScript to use vanilla JS.
>> Video and audio are essentially functional without the NPM installed
>> players. Some later refinements perhaps.
>>
>> 4. Package MediaGoblin itself. The build process is ./configure/make
>> which is a bit weird for a Python project.
>>
>> 5. Get a basic Guix service working, with sqlite3 and without the
>> offloaded media transcoding currently using Celery task queue with a
>> Redis broker.
>>
>> 7. Work out why H264 support is missing.
>>
>> 8. Figure out how to deal with translations.
>>
>> 9. Add a PostgreSQL database to the Guix service instead of sqlite3.
>>
>> Regards,
>> Ben
B
B
Ben Sturmfels wrote on 12 Sep 2021 04:38
(address . 47260@debbugs.gnu.org)
874kaqplh3.fsf@sturm.com.au
Just a progress update - Raghav and Jgart have now packaged and updated
python-wtforms, thank you both!

All that remains now for a passing test suite (and OGG Vorbis support)
is the modified libsndfile that's waiting in Guix's core-updates branch.
Very close now.

Regards,
Ben
B
B
Ben Sturmfels wrote on 13 Sep 2021 06:06
(address . 47260@debbugs.gnu.org)
875yv5t904.fsf@sturm.com.au
I've now written up all the progress in our MediaGoblin Guix channel README:


In short MediaGoblin can be installed as a Guix package with no external
dependencies or Python virtualenvs. After some slightly clumsy static
files configuration the web interface runs successfully based on an
SQLite database. The Celery backend run successfully. Images and video
can be uploaded and viewed successfully. Video plays via the stock
browser player but doesn't allow selection of video quality. Audio is
broken pending merge of updated libsndfile from Guix's "core-updates"
branch.

To try it out, follow the instructions in "Install via load-path",
followed by "Run MediaGoblin" from the README mentioned above.

Regards,
Ben
B
B
Ben Sturmfels wrote on 17 Sep 2021 16:20
(address . 47260@debbugs.gnu.org)
8735q3i8r3.fsf@sturm.com.au
On Mon, 13 Sep 2021, Ben Sturmfels wrote:

I've now updated MediaGoblin to find static assets relative to the code,
and hacked in two new entrypoints `gmg serve` and `gmg celery`. Together
these changes mean that we can now mostly use `inputs` rather than
`propagated-inputs` and that the CSS and images are showing up when
installed as a user package (not service)

Here's the updated channel:


I think we're close to being able to propose a patch for a MediaGoblin
package that can be run standalone. Haven't yet worked on a service.
J
J
jgart wrote on 5 Oct 2021 06:34
Wrapping binaries in MediaGoblin Guix Package
(address . mediagoblin-devel@gnu.org)(address . 47260@debbugs.gnu.org)
20211005003420.GB20658@gac.attlocal.net
Just a note that we'll probably have to wrap the binaries that are in this
module when we finish making the guix package:

mediagoblin/media_types/pdf/processing.py

I see that pdftocairo, pdfinfo, and unoconv probably need to be wrapped to point
to the executable that is in /gnu/store.

See this issue for a similar discussion about wrapping binaries with guix:

B
B
Ben Sturmfels wrote on 5 Oct 2021 07:34
(name . jgart)(address . jgart@dismail.de)
87a6jo11cn.fsf@sturm.com.au
On Tue, 05 Oct 2021, jgart wrote:

Toggle quote (12 lines)
> Just a note that we'll probably have to wrap the binaries that are in this
> module when we finish making the guix package:
>
> mediagoblin/media_types/pdf/processing.py
>
> I see that pdftocairo, pdfinfo, and unoconv probably need to be wrapped to point
> to the executable that is in /gnu/store.
>
> See this issue for a similar discussion about wrapping binaries with guix:
>
> https://issues.guix.gnu.org/50833

Thanks, I've made a note in guix-env.scm to remind me when we look at
PDF support in Guix.
?