substitute servers should be preferred according to their coverage rate

> When using substitute servers discovery, I've noticed that if one of the

> substitute servers doesn't have any substitutes available, it'll keep

> getting tried instead of others, leading to a slide-show of substitutes

> updates such as:

>

> normalized load on machine '127.0.0.1' is 0.04

> building /gnu/store/ajd0hx104702jpz2ycdwgrnyrv8jsp6d-xorg-server-21.1.0.tar.xz.drv...

> process 9195 acquired build slot '/var/guix/offload/127.0.0.1:6666/1'

> normalized load on machine '127.0.0.1' is 0.04

> building /gnu/store/49rqi3wpvdm5pv6in9pamzdvg0wscrl8-xorgproto-2021.5.drv...

> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

> We should implement some scheme to prefer querying high-substitute

> servers first, instead of wasting time querying servers always failed

> queries; this would greatly improve performance when using substitute

> discovery for example combined with low coverage.

> Hi,

>

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

>

>> When using substitute servers discovery, I've noticed that if one of the

>> substitute servers doesn't have any substitutes available, it'll keep

>> getting tried instead of others, leading to a slide-show of substitutes

>> updates such as:

>>

>> normalized load on machine '127.0.0.1' is 0.04

>> building /gnu/store/ajd0hx104702jpz2ycdwgrnyrv8jsp6d-xorg-server-21.1.0.tar.xz.drv...

>> process 9195 acquired build slot '/var/guix/offload/127.0.0.1:6666/1'

>> normalized load on machine '127.0.0.1' is 0.04

>> building /gnu/store/49rqi3wpvdm5pv6in9pamzdvg0wscrl8-xorgproto-2021.5.drv...

>> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

>> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

>> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

>> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

>> substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0%

>

> We’d need to check why this particular server is checked repeatedly.

> The fact that it displays “0.0%” doesn’t mean that the server lacks

> substitutes, but that it does not reply to ‘GET /xyz.narinfo’ requests,

> for example because it’s off-line (see

> <https://issues.guix.gnu.org/48808>.)

>

>> We should implement some scheme to prefer querying high-substitute

>> servers first, instead of wasting time querying servers always failed

>> queries; this would greatly improve performance when using substitute

>> discovery for example combined with low coverage.

>

> There are several problems with that. First one is that you can’t tell

> what substitute coverage is until you’ve actually made those GET

> requests. Second one is that substitute coverage varies and it’s not an

> absolute measure; for example, if a server provides substitutes for only

> 0.1% of all the packages, but that’s precisely the 0.1% you care about,

> it’s more valuable than the one that has 99% of the packages but lacks

> those you want.

>

> There are other issues such as the fact that current semantics is to

> respect the order of substitute URLs, which is presumably chosen by the

> user according to their own criteria: download speed, bandwidth usage,

> etc.

>

> I hope this makes sense!