guix does not resume download on FTP timeouts

OpenSubmitted by Sergei Trofimovich.
Details
3 participants
  • Leo Famulari
  • Ludovic Courtès
  • Sergei Trofimovich
Owner
unassigned
Severity
normal
S
S
Sergei Trofimovich wrote on 7 May 2017 10:31
(address . bug-guix@gnu.org)
20170507093159.170a100b@sf
I have slow internet :)
It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].
Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout.
Thanks!
[1]: $ while :; do ./pre-inst-env guix build sbcl --no-substitutes; done
@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2
Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xzFrom ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz... …3b-texmf.tar.xz 1.93GiB 213KiB/s 02:11:32 [################ ] 83.0%output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `062qbbhlin1ys9lihax34kq503ig1fl55br84j94vwawbk04p6j0'@ build-failed /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - 1 output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `062qbbhlin1ys9lihax34kq503ig1fl55br84j94vwawbk04p6j0'@ build-started /gnu/store/in6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv - x86_64-linux /var/log/guix/drvs/in//6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv.bz2cannot build derivation `/gnu/store/q9s2wdsaqvrllkcsf8w1a8424zd9ybc1-texlive-20160523b-texmf.tar.xz.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/b7abk0aarqv0ylxp2g4s2sf9j75xdy04-texlive-texmf-2016.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/kpjy0c1pl8vzxbkys6p45ij3za6a92gj-texlive-2016.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv': 1 dependencies couldn't be builtguix build: error: build failed: build of `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv' failed@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2
Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xzFrom ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz... …3b-texmf.tar.xz 1.93GiB 213KiB/s 02:11:46 [################ ] 83.3%output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `1l296b4ypphffv1zpamp01g6a633kqlmfbf9l4x9411rlblb1hfd'@ build-failed /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - 1 output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `1l296b4ypphffv1zpamp01g6a633kqlmfbf9l4x9411rlblb1hfd'@ build-started /gnu/store/in6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv - x86_64-linux /var/log/guix/drvs/in//6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv.bz2cannot build derivation `/gnu/store/q9s2wdsaqvrllkcsf8w1a8424zd9ybc1-texlive-20160523b-texmf.tar.xz.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/b7abk0aarqv0ylxp2g4s2sf9j75xdy04-texlive-texmf-2016.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/kpjy0c1pl8vzxbkys6p45ij3za6a92gj-texlive-2016.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv': 1 dependencies couldn't be builtguix build: error: build failed: build of `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv' failed@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2
Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xzFrom ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz... …3b-texmf.tar.xz 1.93GiB 213KiB/s 02:11:31 [################ ] 83.1%output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `143zan8sh0lw2jjhdpvc7w9yjh7cw75w1ry60qq4g1n3hjmfrwi3'@ build-failed /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - 1 output path `/gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz' should have sha256 hash `1dv8vgfzpczqw82hv9g7a8djhhyzywljmrarlcyy6g2qi5q51glr', instead has `143zan8sh0lw2jjhdpvc7w9yjh7cw75w1ry60qq4g1n3hjmfrwi3'@ build-started /gnu/store/in6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv - x86_64-linux /var/log/guix/drvs/in//6vixj9fb9c8f9z3jf42zz9v40dbjxi-serf-1.3.8.drv.bz2cannot build derivation `/gnu/store/q9s2wdsaqvrllkcsf8w1a8424zd9ybc1-texlive-20160523b-texmf.tar.xz.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/b7abk0aarqv0ylxp2g4s2sf9j75xdy04-texlive-texmf-2016.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/kpjy0c1pl8vzxbkys6p45ij3za6a92gj-texlive-2016.drv': 1 dependencies couldn't be builtcannot build derivation `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv': 1 dependencies couldn't be builtguix build: error: build failed: build of `/gnu/store/clnamqpbkkn2k0ji1clbnznx2fyyig2c-sbcl-1.3.7.drv' failed@ build-started /gnu/store/ywpxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv - x86_64-linux /var/log/guix/drvs/yw//pxpl2gp6z3sjvm7sn8gs6n4mhm2f0g-texlive-20160523b-texmf.tar.xz.drv.bz2
--
Sergei
-----BEGIN PGP SIGNATURE-----
iF0EARECAB0WIQSZKa0VG5avZRlY01hxoe52YR/zqgUCWQ7bfwAKCRBxoe52YR/zqgEqAJ9tlNFeHTMr59uH+ueIEs4JkNhdJgCeNpv+DeEteGnirZH3GWGxma658/g==QAGe-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 7 May 2017 20:25
(name . Sergei Trofimovich)(address . slyfox@inbox.ru)(address . 26811@debbugs.gnu.org)
20170507182543.GB6096@jasmine
On Sun, May 07, 2017 at 09:31:59AM +0100, Sergei Trofimovich wrote:
Toggle quote (7 lines)> I have slow internet :)> > It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.> Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].> > Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout.
In the meantime, you can download the files with a resumable tool likewget, and then use `guix download file:///path/to/file` to work aroundthis missing feature. Note that both the client *and* the server mustsupport resuming downloads for this to work.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlkPZqcACgkQJkb6MLrKfwi9mhAA2DCfs3cCsz/B+6CquWTR8irPhT1Yt3ozcj1sGUVyYtEo4sZlWPkX4Ez9q+jZhymrPoQ2QrViKNTY911ZPitcN2KvJbD0Qs8gFEqj3Nzh2E7IKyn3D+jVQaYZd+r7ih5LsLXL1Hn2c9aVOFQuvX2SQHQS3iQ0ZBCIfrHC3QgZWbuqGMwfN3Btf3XEQkaHIlMxX793WxMxKxDYRELsglmDnRbrg5CKPNxNNqG7hoLYEuVAAhxBBwUP+7WX9pHI6r14u6jMHjHt6VK8EYe+QkfCgxA5Trs2KeQ8clKLe5fzLk/pHhpjPDzJf9Psvjla5VzKAZaB1iymI9u8214n765iCmBXmc1VUd+lG8VnN1sxeXCad5aRCjBX4DDbPJ/MkagnbOMLZGfrkiJvS/Sb4TyamQFiTMyzVGS5k/IV71IxCk7RDB3skBmJhjtlCYfzgmNcgJ1XX/+1xASZczch2fzCgUfzS44zlR5QYVdVIKm7rKhNJF6SR6Q/auIx/CGt3FOPlSn6PAOpXx+7sgo91b3w0AyUzvDhOYUqGXkF2oMoUSI5EhxA+FHsIMrmVNKsqQENrtCQxNrjl4skTYirG7iWxDzY6lTEjCOIHXgx/Gpsip03qmQ05vDTKcjvWG9jpMrNTKuxipCYFP2vBW1E8Ln/NSuAavoX4HyPbu0Gio3i/ok==pO2o-----END PGP SIGNATURE-----

S
S
Sergei Trofimovich wrote on 7 May 2017 20:49
(name . Leo Famulari)(address . leo@famulari.name)(address . 26811@debbugs.gnu.org)
20170507194932.213380f8@sf
On Sun, 7 May 2017 14:25:43 -0400Leo Famulari <leo@famulari.name> wrote:
Toggle quote (13 lines)> On Sun, May 07, 2017 at 09:31:59AM +0100, Sergei Trofimovich wrote:> > I have slow internet :)> > > > It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.> > Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].> > > > Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout. > > In the meantime, you can download the files with a resumable tool like> wget, and then use `guix download file:///path/to/file` to work around> this missing feature. Note that both the client *and* the server must> support resuming downloads for this to work.
The workaround worked.
I've noticed it took guix-daemon 2GB of RAM to process this file locally.Does guix keep all the file in RAM until download finishes?
Thank you!
--
Sergei
-----BEGIN PGP SIGNATURE-----
iF0EARECAB0WIQSZKa0VG5avZRlY01hxoe52YR/zqgUCWQ9sPAAKCRBxoe52YR/zqkq6AJ9AcgPoLkED6BqIgeH1DoCoQRv5awCfbL4QzW39v89l8wiXRCUsm1LVV7k==8cHI-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 7 May 2017 20:58
(name . Sergei Trofimovich)(address . slyfox@inbox.ru)(address . 26811@debbugs.gnu.org)
20170507185817.GB27949@jasmine
On Sun, May 07, 2017 at 07:49:32PM +0100, Sergei Trofimovich wrote:
Toggle quote (21 lines)> On Sun, 7 May 2017 14:25:43 -0400> Leo Famulari <leo@famulari.name> wrote:> > > On Sun, May 07, 2017 at 09:31:59AM +0100, Sergei Trofimovich wrote:> > > I have slow internet :)> > > > > > It looks like ftp://tug.org has a FTP session timeout of 2 hours 11 minutes, or something like that.> > > Over night I tried to build sbcl and failed. Note how download silently stopped at ~80% [1].> > > > > > Would be nice if guix had a mechanism to reconnect to FTP and continue download after timeout. > > > > In the meantime, you can download the files with a resumable tool like> > wget, and then use `guix download file:///path/to/file` to work around> > this missing feature. Note that both the client *and* the server must> > support resuming downloads for this to work.> > The workaround worked.> > I've noticed it took guix-daemon 2GB of RAM to process this file locally.> Does guix keep all the file in RAM until download finishes?
I haven't looked, but that's how I'd expect it be coded. The OS canhandle paging things in and out as needed, right?
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlkPbkgACgkQJkb6MLrKfwjY2BAAuj9CyqbTveJQGTVTugc9TaOBSoBgIRV6tFzU7KYobEes5WRUSsrYWVqqLhxl+CsKaKiTQ7Gb3rfNb5hlnAfLKcfpCeSZ0M7+oWPJdB1t/hEYIoSFKmjpse1+wUHMBVHnnQMOxtvE7HGxIq1/56e8A4SgvxGVNiSM1oTIL6+XyW+BJqiI3qedrkwiCvAUNIoHjaqIND/b88TYIF4jstawyudk6kq3aXo3VgFoIQwQzK79K9B/MX3VWbqFMy9QSgNY4G866vsFmhp4pWcJ6lanC51lYB3GqTdVmR7rZHhv9VzVDr9f78Suyv3pAtZFbrs/ihcQoFs7TidtTb3gQZmA67za3zqpRPegK75yfR5cH6U4qwwOD6roueIqnJ1H6+w1lEl0029a0Pw9h7adqh2sXdS8yYF4YR6afLG/nnbl79a5Sho8670rpsboiyi3Ev37bSZJdEbcaBxfrkBZdemabBly0CvWeCopj5EaT9mQeyTwYRKTxMG1kuqyC/kVwGqeT15St8llyzFPimepW9N7w8mK/RpJ+mhaaXqCTuHXiCniFt/ewB36ClDboMVoK877nR59CCYQJqoQHRUBZ/GvIPVH9ikQLm/zKLIn+GtCygp0MZISsK+AuIHqhJ5gwa6iOOPpYaNawgpcKNfULpVJmBBm8HTc9ec3Sh7NOyYHDX8==Z9/j-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 8 May 2017 16:27
(name . Sergei Trofimovich)(address . slyfox@inbox.ru)(address . 26811@debbugs.gnu.org)
8760hbmnk5.fsf@gnu.org
Sergei Trofimovich <slyfox@inbox.ru> skribis:
Toggle quote (3 lines)> Starting download of /gnu/store/xdjl5wxdjghq85nbc9qacgyzh8d54f6y-texlive-20160523b-texmf.tar.xz> From ftp://tug.org/historic/systems/texlive/2016/texlive-20160523b-texmf.tar.xz...
Fundamentally, I think we should just fix our ‘texlive’ package suchthat it doesn’t download 4G at once…
Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 26811@debbugs.gnu.org