Apache mirrors are not available anymore

Open

Details

5 participants

Greg Hogan
Julian Flake
Giovanni Biscuolo
glaz
Simon Tournier

Owner: unassigned

Submitted by: Julian Flake

Severity: normal

Julian Flake wrote on 25 Apr 2023 14:01

Recipients:(address . bug-guix@gnu.org)

Message-ID:87sfcof9v5.fsf@uni-koblenz.de

- The apache mirrors defined in guix/download.scm aren't available 
  anymore.
- Compare the info at https://www.apache.org/mirrors
- (At least Some of the) packages are available at 
  https://dlcdn.apache.org/

Julian Flake wrote on 28 Apr 2023 22:16

[PATCH] fixed apache mirrors

Recipients:(address . 63068@debbugs.gnu.org)(name . Julian Flake)(address . flake@uni-koblenz.de)

Message-ID:20230428201622.7008-1-flake@uni-koblenz.de

Maybe(!) this allows for installing newer versions of packages that are downloaded from Apache servers (e.g. Maven). Can anyone test and confirm?

---

guix/download.scm | 15 ++-------------

1 file changed, 2 insertions(+), 13 deletions(-)

Toggle diff (30 lines)diff --git a/guix/download.scm b/guix/download.scm
index 561a893eee..3abbf2f10c 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -134,19 +134,8 @@ (define %mirrors
        "http://mirror.linux.org.au/"
        "https://mirrors.edge.kernel.org/pub/"
        "ftp://ftp.funet.fi/pub/mirrors/ftp.kernel.org/pub/")
-      (apache             ; from http://www.apache.org/mirrors/dist.html
-       "http://www.eu.apache.org/dist/"
-       "http://www.us.apache.org/dist/"
-       "https://ftp.nluug.nl/internet/apache/"
-       "http://apache.mirror.iweb.ca/"
-       "http://mirrors.ircam.fr/pub/apache/"
-       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
-       "http://apache-mirror.rbc.ru/pub/apache/"
-       "ftp://ftp.osuosl.org/pub/apache/"
-       "http://mirrors.ibiblio.org/apache/"
-
-       ;; As a last resort, try the archive.
-       "http://archive.apache.org/dist/")
+      (apache
+       "https://dlcdn.apache.org/")
       (xorg               ; from http://www.x.org/wiki/Releases/Download
        "http://www.x.org/releases/" ; main mirrors
        "http://mirror.csclub.uwaterloo.ca/x.org/" ; North America

base-commit: 960647f0161c2c99ea3417d129662181e882cfeb
-- 
2.39.2

Julian Flake wrote on 28 Apr 2023 22:36

What I was trying to do

Recipients:(address . 63068@debbugs.gnu.org)

Message-ID:877ctvhgx5.fsf@uni-koblenz.de

What I tried is

$ guix shell --with-version=maven=3.9.1 maven

(or any other version that exists) and got:

guix shell: warning: could not find version 3.9.1 of 'maven'

upstream

The patch above resolves the origin uri in gnu/packages/maven.scm

(definition of maven-pom) to something available,

e.g. https://dlcdn.apache.org/maven/maven-3/3.9.1/source/apache-maven-3.9.1-src.tar.gz

Simon Tournier wrote on 3 May 2023 18:56

Recipients:

Message-ID:86mt2lz74j.fsf@gmail.com

Hi,

On Fri, 28 Apr 2023 at 22:36, Julian Flake <flake@uni-koblenz.de> wrote:

Toggle quote (2 lines)

> $ guix shell --with-version=maven=3.9.1 maven

Just to be sure, are you using the Guix where you applied the patch?

I mean, from the Guix repository where you applied you patch, I am

expecting:

$ ./pre-inst-env guix shell --with-version=maven=3.9.1 maven

If that’s the case, it means the issue is elsewhere. :-)

Cheers,

simon

Julian Flake wrote on 3 May 2023 20:53

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)(address . 63068@debbugs.gnu.org)

Message-ID:874jotmeee.fsf@uni-koblenz.de

Hi,

Toggle quote (5 lines)

>> $ guix shell --with-version=maven=3.9.1 maven

> Just to be sure, are you using the Guix where you applied the

> patch?

No, the proposed patch is just an untested guess. I didn't test

the patch yet, because I didn't manage to build (make) my guix

working copy yet.

Best Regards,

Julian

On Wed, May 03 2023, Simon Tournier wrote:

Toggle quote (22 lines)> Hi,
>
> On Fri, 28 Apr 2023 at 22:36, Julian Flake 
> <flake@uni-koblenz.de> wrote:
>
>> $ guix shell --with-version=maven=3.9.1 maven
>
> Just to be sure, are you using the Guix where you applied the 
> patch?
>
> I mean, from the Guix repository where you applied you patch, I 
> am
> expecting:
>
>     $ ./pre-inst-env guix shell --with-version=maven=3.9.1 maven
>
> If that’s the case, it means the issue is elsewhere. :-)
>
> Cheers,
> simon

-- 
Dipl.-Inf. Julian Flake

Universität Koblenz
Fachbereich Informatik
Institut für Softwaretechnik
Postfach 20 16 02 | D-56016 Koblenz

Tel.: +49 261 287 2787
E-Mail: flake@uni-koblenz.de
Website: https://www.uni-koblenz.de/~flake

Giovanni Biscuolo wrote on 4 May 2023 09:52

Re: bug#63068: [PATCH] fixed apache mirrors

Recipients:(name . Julian Flake)(address . flake@uni-koblenz.de)

Message-ID:87cz3gjzyi.fsf@xelera.eu

Hello Julian,

thank you for the patch

Julian Flake <flake@uni-koblenz.de> writes:

Toggle quote (28 lines)> Maybe(!) this allows for installing newer versions of packages that are downloaded from Apache servers (e.g. Maven). Can anyone test and confirm?
>
> ---
>  guix/download.scm | 15 ++-------------
>  1 file changed, 2 insertions(+), 13 deletions(-)
>
> diff --git a/guix/download.scm b/guix/download.scm
> index 561a893eee..3abbf2f10c 100644
> --- a/guix/download.scm
> +++ b/guix/download.scm
> @@ -134,19 +134,8 @@ (define %mirrors
>         "http://mirror.linux.org.au/"
>         "https://mirrors.edge.kernel.org/pub/"
>         "ftp://ftp.funet.fi/pub/mirrors/ftp.kernel.org/pub/")
> -      (apache             ; from http://www.apache.org/mirrors/dist.html
> -       "http://www.eu.apache.org/dist/"
> -       "http://www.us.apache.org/dist/"
> -       "https://ftp.nluug.nl/internet/apache/"
> -       "http://apache.mirror.iweb.ca/"
> -       "http://mirrors.ircam.fr/pub/apache/"
> -       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
> -       "http://apache-mirror.rbc.ru/pub/apache/"
> -       "ftp://ftp.osuosl.org/pub/apache/"
> -       "http://mirrors.ibiblio.org/apache/"
> -
> -       ;; As a last resort, try the archive.
> -       "http://archive.apache.org/dist/")

The archive is still working, there is no need to remove it

Toggle quote (8 lines)

> + (apache

> + "https://dlcdn.apache.org/")

> (xorg ; from http://www.x.org/wiki/Releases/Download

> "http://www.x.org/releases/" ; main mirrors

> "http://mirror.csclub.uwaterloo.ca/x.org/" ; North America

> base-commit: 960647f0161c2c99ea3417d129662181e882cfeb

Please can you send a new version of this patch, also with a proper

commit message?

Thanks! Gio'

Giovanni Biscuolo

Xelera IT Infrastructures

Julian Flake wrote on 4 May 2023 10:21

[PATCH] gnu: apache mirror URLs substituted by CDN URL

Recipients:(address . 63068@debbugs.gnu.org)(name . Julian Flake)(address . flake@uni-koblenz.de)

Message-ID:20230504082158.30320-1-flake@uni-koblenz.de

* guix/download.scm (Apache mirrors are substituted by CDN):

---

guix/download.scm | 13 ++-----------

1 file changed, 2 insertions(+), 11 deletions(-)

Toggle diff (28 lines)diff --git a/guix/download.scm b/guix/download.scm
index 561a893eee..ce29e12ae8 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -134,17 +134,8 @@ (define %mirrors
        "http://mirror.linux.org.au/"
        "https://mirrors.edge.kernel.org/pub/"
        "ftp://ftp.funet.fi/pub/mirrors/ftp.kernel.org/pub/")
-      (apache             ; from http://www.apache.org/mirrors/dist.html
-       "http://www.eu.apache.org/dist/"
-       "http://www.us.apache.org/dist/"
-       "https://ftp.nluug.nl/internet/apache/"
-       "http://apache.mirror.iweb.ca/"
-       "http://mirrors.ircam.fr/pub/apache/"
-       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
-       "http://apache-mirror.rbc.ru/pub/apache/"
-       "ftp://ftp.osuosl.org/pub/apache/"
-       "http://mirrors.ibiblio.org/apache/"
-
+      (apache ; see https://apache.org/history/mirror-history.html
+       "https://dlcdn.apache.org/"
        ;; As a last resort, try the archive.
        "http://archive.apache.org/dist/")
       (xorg               ; from http://www.x.org/wiki/Releases/Download

base-commit: 7a0a186a32524d4156bf96786d708fab323cebff
-- 
2.39.2

Julian Flake wrote on 4 May 2023 10:22

Re: bug#63068: [PATCH] fixed apache mirrors

Recipients:(name . Giovanni Biscuolo)(address . g@xelera.eu)(address . 63068@debbugs.gnu.org)

Message-ID:87zg6kld0a.fsf@uni-koblenz.de

Hello Giovanni,

Toggle quote (4 lines)

> Please can you send a new version of this patch, also with a

> proper

> commit message?

see above (or https://issues.guix.gnu.org/63068#6).I didn't send

it to guix-patches@gnu.org, since I still didn't test the

patch. Furthermore I am not sure, how a proper commit message

looks like, when it is not related to a package (there is no

YASnippet for that, right?).

Best Regards,

Julian

On Thu, May 04 2023, Giovanni Biscuolo wrote:

Toggle quote (64 lines)> [[PGP Signed Part:Undecided]]
> Hello Julian,
>
> thank you for the patch
>
> Julian Flake <flake@uni-koblenz.de> writes:
>
>> Maybe(!) this allows for installing newer versions of packages 
>> that are downloaded from Apache servers (e.g. Maven). Can 
>> anyone test and confirm?
>>
>> ---
>>  guix/download.scm | 15 ++-------------
>>  1 file changed, 2 insertions(+), 13 deletions(-)
>>
>> diff --git a/guix/download.scm b/guix/download.scm
>> index 561a893eee..3abbf2f10c 100644
>> --- a/guix/download.scm
>> +++ b/guix/download.scm
>> @@ -134,19 +134,8 @@ (define %mirrors
>>         "http://mirror.linux.org.au/"
>>         "https://mirrors.edge.kernel.org/pub/"
>>         "ftp://ftp.funet.fi/pub/mirrors/ftp.kernel.org/pub/")
>> -      (apache             ; from 
>> http://www.apache.org/mirrors/dist.html
>> -       "http://www.eu.apache.org/dist/"
>> -       "http://www.us.apache.org/dist/"
>> -       "https://ftp.nluug.nl/internet/apache/"
>> -       "http://apache.mirror.iweb.ca/"
>> -       "http://mirrors.ircam.fr/pub/apache/"
>> -       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
>> -       "http://apache-mirror.rbc.ru/pub/apache/"
>> -       "ftp://ftp.osuosl.org/pub/apache/"
>> -       "http://mirrors.ibiblio.org/apache/"
>> -
>> -       ;; As a last resort, try the archive.
>> -       "http://archive.apache.org/dist/")
>
> The archive is still working, there is no need to remove it
>
>> +      (apache
>> +       "https://dlcdn.apache.org/")
>>        (xorg               ; from 
>>        http://www.x.org/wiki/Releases/Download
>>         "http://www.x.org/releases/" ; main mirrors
>>         "http://mirror.csclub.uwaterloo.ca/x.org/" ; North 
>>         America
>>
>> base-commit: 960647f0161c2c99ea3417d129662181e882cfeb
>
> Please can you send a new version of this patch, also with a 
> proper
> commit message?
>
> Thanks! Gio'
>
> -- 
> Giovanni Biscuolo
>
> Xelera IT Infrastructures
>
> [[End of PGP Signed Part]]

-- 
Dipl.-Inf. Julian Flake

Universität Koblenz
Fachbereich Informatik
Institut für Softwaretechnik
Postfach 20 16 02 | D-56016 Koblenz

Tel.: +49 261 287 2787
E-Mail: flake@uni-koblenz.de
Website: https://www.uni-koblenz.de/~flake

Giovanni Biscuolo wrote on 5 May 2023 08:32

Re: bug#63068: What I was trying to do

Recipients:

Message-ID:87a5yjjnje.fsf@xelera.eu

Hello Julian and Simon,

I'm trying to test this patch but I find a problem with package version

specification

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (6 lines)

> Hi,

> On Fri, 28 Apr 2023 at 22:36, Julian Flake <flake@uni-koblenz.de> wrote:

>> $ guix shell --with-version=maven=3.9.1 maven

If I try to _build_ maven with my current guix specifying a version I

get this:

Toggle snippet (7 lines)

$ LC_ALL=C guix build -n --no-substitutes maven --with-version=maven=3.8.6

guix build: warning: could not find version 3.8.6 of 'maven' upstream

The following derivations would be built:

Am I doing something wrong?

Toggle quote (7 lines)

> Just to be sure, are you using the Guix where you applied the patch?

> I mean, from the Guix repository where you applied you patch, I am

> expecting:

> $ ./pre-inst-env guix shell --with-version=maven=3.9.1 maven

I applied the patch and in my guix dev env I try to build maven:

Toggle snippet (6 lines)

giovanni@roquette [genv]\: ./pre-inst-env guix build -n --no-substitutes maven

/gnu/store/9cx7sijbc9n1fvq528qysh33v6aqb3d9-maven-3.8.6

How can I "force" the build of a package already stored in /gnu/store?

Obviously building with --with-version does not work here, also:

Toggle snippet (7 lines)

giovanni@roquette [genv]\: ./pre-inst-env guix build -n --no-substitutes maven --with-version=maven=3.8.6

guix build: warning: could not find version 3.8.6 of 'maven' upstream

/gnu/store/9cx7sijbc9n1fvq528qysh33v6aqb3d9-maven-3.8.6

[...]

What am I missing?

Thanks! Gio'

Giovanni Biscuolo

Xelera IT Infrastructures

Simon Tournier wrote on 5 May 2023 08:59

Recipients:(name . Giovanni Biscuolo)(address . g@xelera.eu)

Message-ID:CAJ3okZ16N819S8U-S7ZgSgxgJE+=+8ZbonoWhE-nNrrLA57KRw@mail.gmail.com

Hi,

A quick answer for one specific question. :-)

On Fri, 5 May 2023 at 08:32, Giovanni Biscuolo <g@xelera.eu> wrote:

Toggle quote (2 lines)

> How can I "force" the build of a package already stored in /gnu/store?

See option --check.

For the rest, I will give a look later, but do not hold your breathe. ;-)

Cheers,

simon

Greg Hogan wrote on 7 Aug 2023 22:10

Re: bug#63068: [PATCH] gnu: apache mirror URLs substituted by CDN URL

Recipients:(name . Julian Flake)(address . flake@uni-koblenz.de)(address . 63068@debbugs.gnu.org)

Message-ID:CA+3U0Zmvfy8LX4kOPF2stiwM8E3ypdnMAgUgM4z-nq5-vO7+Ww@mail.gmail.com

On Thu, May 4, 2023 at 4:23?AM Julian Flake <flake@uni-koblenz.de> wrote:

Toggle quote (28 lines)>
> * guix/download.scm (Apache mirrors are substituted by CDN):
> ---
>  guix/download.scm | 13 ++-----------
>  1 file changed, 2 insertions(+), 11 deletions(-)
>
> diff --git a/guix/download.scm b/guix/download.scm
> index 561a893eee..ce29e12ae8 100644
> --- a/guix/download.scm
> +++ b/guix/download.scm
> @@ -134,17 +134,8 @@ (define %mirrors
>         "http://mirror.linux.org.au/"
>         "https://mirrors.edge.kernel.org/pub/"
>         "ftp://ftp.funet.fi/pub/mirrors/ftp.kernel.org/pub/")
> -      (apache             ; from http://www.apache.org/mirrors/dist.html
> -       "http://www.eu.apache.org/dist/"
> -       "http://www.us.apache.org/dist/"
> -       "https://ftp.nluug.nl/internet/apache/"
> -       "http://apache.mirror.iweb.ca/"
> -       "http://mirrors.ircam.fr/pub/apache/"
> -       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
> -       "http://apache-mirror.rbc.ru/pub/apache/"
> -       "ftp://ftp.osuosl.org/pub/apache/"
> -       "http://mirrors.ibiblio.org/apache/"
> -
> +      (apache ; see https://apache.org/history/mirror-history.html
> +       "https://dlcdn.apache.org/"

The link referenced in

https://infra.apache.org/release-distribution.html#channelsis

"downloads.apache.org" rather than dlcdn.apache.org.

Toggle quote (7 lines)>         ;; As a last resort, try the archive.
>         "http://archive.apache.org/dist/")
>        (xorg               ; from http://www.x.org/wiki/Releases/Download
>
> base-commit: 7a0a186a32524d4156bf96786d708fab323cebff
> --
> 2.39.2

glaz wrote on 2 Dec 2023 20:08

Apache mirrors are not available anymore

Recipients:(address . 63068@debbugs.gnu.org)

Message-ID:593593ec62d0e9fa0cb2fe3f4e9ad53f@zho.pa

Good evening.

Here's another version of patch that changes download URL's for Apache

Foundation products.

What I have changed:

* Remove Apache mirror list from download.scm, since all Apache

projects now use it's own CDN instead of mirrors

* Set proper download URL to dlcdn.apache.org, not download.apache.org

mentioned earlier in this discussion. download.apache.org doesn't work.

All download URLs on apache.org are set to dlcdn.apache.org

* Use archive.apache.org for outdated httpd-2.4.52 package. The 'last

resort' trick in the mirror list doesn't work because some mirrors

return empty file when you try to download from them, guix returns 'sha

missmatch' error.

Dmitry

Attachment: apache_dlcdn.patch (.03 MiB)

glaz wrote on 10 Dec 2023 08:52

Recipients:(address . 63068@debbugs.gnu.org)

Message-ID:569c2255edd61b7aed8c5e6fe7b4f0bf@zho.pa

Howdy ho, fellow guixers.

I did some minor research on my patch. Looks like I was wrong. There are 
more than one package, that is not available via DLCDN, but only on 
archive.apache.org. So the best solutions seems to be to change the list 
of mirrors in download.scm as described earlier.

-      (apache             ; from 
http://www.apache.org/mirrors/dist.html
-       "http://www.eu.apache.org/dist/"
-       "http://www.us.apache.org/dist/"
-       "https://ftp.nluug.nl/internet/apache/"
-       "http://apache.mirror.iweb.ca/"
-       "http://mirrors.ircam.fr/pub/apache/"
-       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
-       "http://apache-mirror.rbc.ru/pub/apache/"
-       "ftp://ftp.osuosl.org/pub/apache/"
-       "http://mirrors.ibiblio.org/apache/"
-
+      (apache ; see https://apache.org/history/mirror-history.html
+       "https://dlcdn.apache.org/"

Dmitry

Simon Tournier wrote on 12 Jan 13:01 +0100

Recipients:

Message-ID:87frz2wyjq.fsf@gmail.com

Hi,

On Sun, 10 Dec 2023 at 10:52, glaz--- via Bug reports for GNU Guix <bug-guix@gnu.org> wrote:

Toggle quote (20 lines)> I did some minor research on my patch. Looks like I was wrong. There are 
> more than one package, that is not available via DLCDN, but only on 
> archive.apache.org. So the best solutions seems to be to change the list 
> of mirrors in download.scm as described earlier.
>
> -      (apache             ; from 
> http://www.apache.org/mirrors/dist.html
> -       "http://www.eu.apache.org/dist/"
> -       "http://www.us.apache.org/dist/"
> -       "https://ftp.nluug.nl/internet/apache/"
> -       "http://apache.mirror.iweb.ca/"
> -       "http://mirrors.ircam.fr/pub/apache/"
> -       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
> -       "http://apache-mirror.rbc.ru/pub/apache/"
> -       "ftp://ftp.osuosl.org/pub/apache/"
> -       "http://mirrors.ibiblio.org/apache/"
> -
> +      (apache ; see https://apache.org/history/mirror-history.html
> +       "https://dlcdn.apache.org/"

I think it is best to tweak the list of mirrors as you suggest here.
However, why remove all the others?  Are they officially down?

What I would suggest is the list:

      (apache ; see https://apache.org/history/mirror-history.html
       "https://dlcdn.apache.org/"
       "http://www.eu.apache.org/dist/"
       "http://www.us.apache.org/dist/"
       "https://ftp.nluug.nl/internet/apache/"
       "http://apache.mirror.iweb.ca/"
       "http://mirrors.ircam.fr/pub/apache/"
       "http://apache.mirrors.ovh.net/ftp.apache.org/dist/"
       "http://apache-mirror.rbc.ru/pub/apache/"
       "ftp://ftp.osuosl.org/pub/apache/"
       "http://mirrors.ibiblio.org/apache/"

Hence the first item will be tried first.  And if it fails, the other
will be checked.  Then if it still fails, Guix will try other sources as
Webarchive, Software Heritage, etc.

Cheers,
simon

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 63068@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date