Poor profile generation performance on spinning disks

  • Open
  • quality assurance status badge
Details
5 participants
  • Joshua Branson
  • Ludovic Courtès
  • Luis Felipe
  • Maxim Cournoyer
  • zimoun
Owner
unassigned
Submitted by
Maxim Cournoyer
Severity
normal
M
M
Maxim Cournoyer wrote on 18 Oct 2020 05:35
(name . bug-guix)(address . bug-guix@gnu.org)
87zh4k435a.fsf@gmail.com
Hello!

I've noticed on multiple occasions that using Guix on traditional
spinning drives can be quite slow.

On my home machine, will is still relying on 2 x 1 TB spinning drives in
RAID1, rebuilding my user profile, which contains 182 entries, takes on
average about 20 minutes, even when there are no packages to be built:

Toggle snippet (22 lines)
guix package -i perl --max-jobs=1 The following package will be
upgraded: perl (dependencies or package changed)

The following derivation will be built:
/gnu/store/lhywla1z2zcz16df4hbvvvngr9zmswr7-profile.drv

building CA certificate bundle...
building fonts directory...
generating GLib schema cache...
creating GTK+ icon theme cache...
building cache files for GTK+ input methods...
building directory of Info manuals...
building database for manual pages...
building XDG desktop file cache...
building XDG MIME database...
building profile with 182 packages...

real 19m0.126s
user 0m5.648s
sys 0m0.333s

Most of the time remains spent after the message 'building profile with
182 package...'. That part seems IO-bound, with the spinning disks
grinding heavily and the CPU mostly idling. The rest of the time (3
minutes), was used by the profile hooks.

The same operation on a second, more modern machine equipped with M2
SSDs does much better and takes about 1 minute to accomplish the same,
so it seems the bad performance can be mostly attributed to the much
slower disk seek times of the spinning disks.

On the older machine, two profile hooks are also sticking out w.r.t. the
time they take (they take more than one minute opposed to a few
seconds):

Toggle snippet (26 lines)
The following profile hook will be built:
/gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
building database for manual pages...
Creating manual page database...
[2139/2139] building list of man-db entries...
175322 entries processed in 95.1 s
successfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
successfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv
/gnu/store/wzp4mk2r7r4ysciw74gqbfkyai0zmrcc-manual-database

real 1m36.378s
user 0m1.674s
sys 0m0.108s

The following profile hook will be built:
/gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
building XDG MIME database...
successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
/gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database

real 1m7.344s
user 0m1.331s
sys 0m0.053s

So we should profile what's going on while generating the profile (no
pun intended) and try to improve this at first since this is where most
of the time is spent on spinning drives (17 minutes out of the 20 in the
above example).

After that we could look into the two above profile hooks.

Thanks,

Maxim
L
L
Luis Felipe wrote on 19 Oct 2020 01:05
(name . 44053@debbugs.gnu.org)(address . 44053@debbugs.gnu.org)
yuID-gVO5clS0T8k81R_AzM7g91lm_6kDlH9rJFE8OOv5XMxEsdFo6hCH_QGnJGMW4nyDfuduHS_cXzKOm0qov8BalGmi8NPGKwR_eInGFk=@protonmail.com
I've never used SDDs, so I've been experiencing this since I installed Guix (~5 years ago). And yes, building the profile seems to be the more resource intensive operation. I usually can't do anything else while that happens because the computer becomes unresponsive.

Running the same command Maxim mentioned took the following time in my case:


$ time guix package -i perl --max-jobs=1
Se instalará el siguiente paquete:
   perl 5.30.2
Se construirá la siguiente derivación:
   /gnu/store/86g48nh8dhgdhzb8r9bxjk6mmszlssss-profile.drv
construyendo empaquetado de certificados de CA...
construyendo el directorio de tipografías...
generando la caché de esquemas de GLib...
creando la caché de temas de iconos de GTK+...
construyendo los ficheros de caché para los métodos de entrada de GTK+...
construyendo el directorio de manuales Info...
construyendo la base de datos de páginas de manual...
construyendo la caché de ficheros desktop XDG...
construyendo la base de datos MIME XDG...
construyendo perfil con 87 paquetes...


real    8m3,043s
user    0m2,656s
sys    0m0,239s


My computer:

Intel® Core™ i3-8100 CPU @ 3.60GHz × 4
RAM: 4 GiB
1 TB HDD
Z
Z
zimoun wrote on 19 Oct 2020 10:18
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 44053@debbugs.gnu.org)
CAJ3okZ1AMUteo3qpAHb0m6RS3ApawCrdfQwVNkDg6Xtc515LcA@mail.gmail.com
Hi Maxim,

On Sun, 18 Oct 2020 at 05:37, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (4 lines)
> On my home machine, will is still relying on 2 x 1 TB spinning drives in
> RAID1, rebuilding my user profile, which contains 182 entries, takes on
> average about 20 minutes, even when there are no packages to be built:

The database build time seems correlated to the number of packages in
the very profile. Well, I have not benchmarked to see if it is
linear, quadratic; or worse or better.


Toggle quote (1 lines)
> --8<---------------cut here---------------start------------->8---
[...]
Toggle quote (10 lines)
> building XDG MIME database...
> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
> /gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database
>
> real 1m7.344s
> user 0m1.331s
> sys 0m0.053s
> --8<---------------cut here---------------end--------------->8---

Well, it is all clear to me, but I do not know if we can do better
because these 2 XDG updates seem relying on the external binaries
"/bin/update-desktop-database" and "/bin/update-mime-database". I
have not timed the function 'xdg-desktop-database' but all the time
should be spent on these, I bet. :-)


All the best,
simon
M
M
Maxim Cournoyer wrote on 19 Oct 2020 20:18
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 44053@debbugs.gnu.org)
87d01e5bbe.fsf@gmail.com
Hello Simon,

zimoun <zimon.toutoune@gmail.com> writes:

Toggle quote (12 lines)
> Hi Maxim,
>
> On Sun, 18 Oct 2020 at 05:37, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> On my home machine, will is still relying on 2 x 1 TB spinning drives in
>> RAID1, rebuilding my user profile, which contains 182 entries, takes on
>> average about 20 minutes, even when there are no packages to be built:
>
> The database build time seems correlated to the number of packages in
> the very profile. Well, I have not benchmarked to see if it is
> linear, quadratic; or worse or better.

It is. Small profiles operations are much faster than large profiles.
Attached is a slightly trimmed version of the manifest I currently use
for my user profile, if you'd like to benchmark it on your system.
(use-modules (gnu packages) (gnu packages emacs) (guix build-system emacs) (guix profiles)) (concatenate-manifests (list ;;; Emacs packages. (specifications->manifest '("emacs" "emacs-auctex" "emacs-bash-completion" "emacs-bbdb" "emacs-cmake-mode" "emacs-company" "emacs-company-quickhelp" "emacs-counsel" "emacs-csv-mode" "emacs-debbugs" "emacs-diff-hl" "emacs-el-mock" "emacs-elpy" "emacs-emms" "emacs-ggtags" "emacs-go-mode" "emacs-grep-a-lot" "emacs-guix" "emacs-htmlize" "emacs-ivy" "emacs-magit" "emacs-org" "emacs-org-reveal" "emacs-paredit" "emacs-pdf-tools" "emacs-qml-mode" "emacs-realgud" "emacs-string-inflection" "emacs-swiper" "emacs-w3m" "emacs-ws-butler" "emacs-yasnippet" "emacs-yasnippet-snippets")) ;; Other software. (specifications->manifest '("adb" "acpi" "alsa-utils" "anthy" "arc-icon-theme" "arc-theme" "aspell" "aspell-dict-en" "aspell-dict-fr" "autoconf" "automake" "autossh" "bash" "bc" "beep" "bind:utils" ;for 'dig' "bluez" "bridge-utils" "cheese" "compsize" "cqfd" "cryptsetup" "curl" "dbus" "dconf" "ddcutil" "diffoscope" "docker-cli" "dosfstools" "evince" "file" "font-adobe-source-han-sans" "font-dejavu" "font-google-roboto" "font-hack" "gcc-toolchain" "gdb" "geeqie" "ghostscript-with-x" "gimp" "git" "git:send-email" "glibc-locales" "global" "gnome-bluetooth" "gnome-boxes" "adwaita-icon-theme" "hicolor-icon-theme" "gnu-standards" "gnucash" "gnucash:doc" "gnupg" "graphviz" "gtk-engines" "guile" "guile-readline" "guile-sqlite3" "guile-ssh" "hackneyed-x11-cursors" "hicolor-icon-theme" "hunspell" "hunspell-dict-fr" "ibus" "ibus-anthy" "icecat" "imagemagick" "inetutils" "inkscape" "iotop" "jack" "jami" "keepassxc" "libjpeg" "libmtp" "libpcap" "libreoffice" "libssh" "libx11" "linphoneqt" "lm-sensors" "lsof" "ltrace" "lvm2" ;for dmsetup "make" "man-pages" "mesa-utils" "mpv" "mtr" "nmap" "openssh" "openvpn" "parted" "pavucontrol" "perl" "pinentry" "pkg-config" "poppler" "pulseaudio" "pv" "python" "python-wrapper" "qemu" "recutils" "rsync" "rtorrent" "screen" "setxkbmap" "shepherd" "sicp" "smartmontools" "spacefm" "stow" "strace" "sysstat" ;for iostat "tcpdump" "the-silver-searcher" ;ag "time" ;aliased to time+ "transmission" "transmission:gui" "tree" "unzip" "vinagre" "vorbis-tools" "weechat" "wget" "workrave" "wpa-supplicant" "xdpyinfo" "xdg-utils" "xev" "xmodmap" "xournal" "xrandr" "xrdb" "xsetroot" "yelp" "gxtuner" "shellcheck")) ;; Others. (specifications->manifest '("docker-compose" "emacs-adoc-mode" "emacs-clang-format" "emacs-clang-rename" "emacs-feature-mode" "picocom" "python-git-review" "sshpass" "ungoogled-chromium" "ddrescue"))))
I'm betting that the part to optimize is:

Toggle snippet (6 lines)
;; Make the symlinks.
(union-build output inputs
#:symlink symlink
#:log-port (%make-void-port "w"))

from the 'build-profile' procedure in (guix build profiles).

Toggle quote (19 lines)
>
>> --8<---------------cut here---------------start------------->8---
> [...]
>> building XDG MIME database...
>> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
>> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv
>> /gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database
>>
>> real 1m7.344s
>> user 0m1.331s
>> sys 0m0.053s
>> --8<---------------cut here---------------end--------------->8---
>
> Well, it is all clear to me, but I do not know if we can do better
> because these 2 XDG updates seem relying on the external binaries
> "/bin/update-desktop-database" and "/bin/update-mime-database". I
> have not timed the function 'xdg-desktop-database' but all the time
> should be spent on these, I bet. :-)

They do, and this is indeed what takes time. But, this doesn't stop
motivated wizards from implementing our own database generator in Scheme
that would do the same. Ludovic had done just that for the man-db
database generator. The custom code in Guix can do it much faster that
it initially could back when it was relying on man-db to do that (that's
also probably the reason why we have issues such as
https://issues.guix.gnu.org/38838;perhaps the generated database is
slightly different -- but that's another topic :-)).

Thanks,

Maxim
J
J
Joshua Branson wrote on 28 Oct 2020 00:14
Re: ‘xdg-mime-database’ profile hook is slow
(name . Ludovic Courtès)(address . ludo@gnu.org)
87ft5zqn2a.fsf@dismail.de
I am just posting my performance numbers. Hopefully you all find it helpful.
I am running Guix System on a Lenovo Thinkpad T400 with 8GB on a hard drive.

#+BEGIN_SRC sh :results output :exports both
guix describe
#+END_SRC

#+RESULTS:
#+begin_example
Generation 49 Oct 20 2020 01:15:33 (current)
my-personal-packages 547d900
branch: master
commit: 547d900c25d2bb9490946162e0fcb315d6bace68
guix 93c2518
branch: master
commit: 93c251818dcd2bf5f60f42e595504293ee83371a

GUIX_PACKAGE_PATH="/home/joshua/prog/gnu/guix/guix-packages/"
#+end_example

The "XDG MIME database" takes a while.

#+BEGIN_SRC sh :results output :exports both
time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)
#+END_SRC

#+RESULTS:
: The following profile hook will be built:
: /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
: building XDG MIME database...
: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
: /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database
:
: real 0m43.716s
: user 0m3.626s
: sys 0m0.258s

Cold cache:

#+BEGIN_SRC sh :results output :exports both
drv="$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)"
sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'

time guix build --check "$drv"
#+END_SRC

#+RESULTS:
: The following profile hook will be built:
: /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
: building XDG MIME database...
: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
: /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database
:
: real 1m29.606s
: user 0m1.401s
: sys 0m0.238s

I have 64 packages in my profile.

#+BEGIN_SRC sh :results output :exports both
guix package -I | wc --lines
#+END_SRC

#+RESULTS:
: 64

If you want to see all packages:

#+BEGIN_SRC sh :results output :exports both
guix package -I
#+END_SRC

#+RESULTS:
#+begin_example
guile-colorized 0.1 out /gnu/store/5h0jfssgyx49cq82wrmw57an7ngb7kkz-guile-colorized-0.1
font-hack 3.003 out /gnu/store/52r8anazd4rnkq9m3vxk700jga5h0i74-font-hack-3.003
font-dejavu 2.37 out /gnu/store/7y3lvk3xf4im8n44337mc6y0ccysvfia-font-dejavu-2.37
font-adobe-source-code-pro 2.030R-ro-1.050R-it out /gnu/store/l5m74158njcan3p784gqhkir6zyk0bcp-font-adobe-source-code-pro-2.030R-ro-1.050R-it
sicp 20170703-1.225c172 out /gnu/store/x2safq6fvcbhp2rljny89qiq74y1xd13-sicp-20170703-1.225c172
unzip 6.0 out /gnu/store/1zx8knb7h79q6cr4kwhbz6z6v609309w-unzip-6.0
font-google-noto 20171025 out /gnu/store/g2szydnbvs7qqy2nf7qylba0rapajmd8-font-google-noto-20171025
font-awesome 4.7.0 out /gnu/store/gknn01i1nr80apmcx42w36cqcz3zzri2-font-awesome-4.7.0
texinfo 6.7 out /gnu/store/w8k9hcigvhzrlrblv8lgqj77sm3833rs-texinfo-6.7
gcc-toolchain 10.2.0 out /gnu/store/pknm43xsza6nlc7bn27djip8fis92akd-gcc-toolchain-10.2.0
recutils 1.8 out /gnu/store/qlx179h9mfdv4zywx2ias2jmkygvs834-recutils-1.8
guile-readline 3.0.2 out /gnu/store/va6l1ivclww22fi38w5h99pb4ndn99hg-guile-readline-3.0.2
gtypist 2.9.5 out /gnu/store/03wd707lyz49f2r14cm9ybyzh7vfvi0j-gtypist-2.9.5
scsh 0.0.0-1.1144324 out /gnu/store/57frd08q6gyd7ysq3aqwlxxmify3w53h-scsh-0.0.0-1.1144324
make 4.3 out /gnu/store/4k33n2nhsnnaxk2ip75gj7xiqdjns5hq-make-4.3
isync 1.3.3 out /gnu/store/745qmx9mmnwq0j3zrd50qcdd2bgv5clz-isync-1.3.3
jmacs 27.1 out /gnu/store/0hqdddkvhi2acxjdssdi5as3wj6cgvnz-jmacs-27.1
glibc-locales 2.31 out /gnu/store/x9c4dk4zc9mi5j9hm3lq1q539kkdhkln-glibc-locales-2.31
guile 3.0.4 out /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4
guile-json 4.3.2 out /gnu/store/12y1s0p3585ydf336gq85kjh5217pnzp-guile-json-4.3.2
guile-syntax-highlight 0.1 out /gnu/store/dwjqscckpypqfq38hc97x5ipc9fplpp9-guile-syntax-highlight-0.1
haunt 0.2.4 out /gnu/store/ccvqa9a6yclgkpi9izhpgf1vgjaghqvr-haunt-0.2.4
gnutls 3.6.14 out /gnu/store/0f6agk6xk8rcz99907d39nd1vzmc4abq-gnutls-3.6.14
wget 1.20.3 out /gnu/store/d7b16jqrgb1jz65hl7vnz261dbxmjh3s-wget-1.20.3
gnupg 2.2.23 out /gnu/store/89cjqnar4aacg58wl6cm1h41rym9mv4l-gnupg-2.2.23
sassc 3.6.1 out /gnu/store/6pqn2i0r12ypss13mdvsypskkcpaagji-sassc-3.6.1
qterminal 0.15.0 out /gnu/store/y7qrfa78y785xfajb3gvvhwcm2d02l81-qterminal-0.15.0
dmenu 5.0 out /gnu/store/393frjc5xjaab6nvp3wxpajb1rc6gynr-dmenu-5.0
openssh 8.4p1 out /gnu/store/k8lnv9cqk6m15k18hhzg7w696r1vxx1v-openssh-8.4p1
grim 1.3.1 out /gnu/store/9v5i9q7kwcmq1q31qri0icjkhh3csh9r-grim-1.3.1
evince 3.36.5 out /gnu/store/arwvpbrzsch4w6afkpqln6625dfl0bz8-evince-3.36.5
pulseaudio 13.0 out /gnu/store/19wcjfwdr3hzq1a4wcpld1zdrjichck0-pulseaudio-13.0
libreoffice 6.4.6.2 out /gnu/store/7rbk52nzqmakd5whds5a6f07xrd2k870-libreoffice-6.4.6.2
termite 15 out /gnu/store/8lqfy0mxqnq3xg24gwsxvzpxnvv6firh-termite-15
mpv 0.32.0 out /gnu/store/2i2gzn7fa8qq0f1jkqhiyl0xw63fjr4w-mpv-0.32.0
pinentry 1.1.0 out /gnu/store/bdsk9yq48lx1f7y7xr9sv4yrmm4ync9b-pinentry-1.1.0
glib 2.62.6 doc /gnu/store/ql2226knbh4q04bbdiqcppvs5rjyl3ga-glib-2.62.6-doc
pidgin 2.13.0 out /gnu/store/fd06njmc5maid7b474v224iazy7bc20h-pidgin-2.13.0
gimp 2.10.20 out /gnu/store/m397hvwhsd0l6v5d0vqxxkm4iyp0p9w3-gimp-2.10.20
sway 1.4 out /gnu/store/27ggd92in0hpgmkgk79n2iy4angjxwvl-sway-1.4
wf-recorder 0.2.1 out /gnu/store/glc6y75l4czpfyrgbrbgxg38ig004v1p-wf-recorder-0.2.1
wofi 1.1.2 out /gnu/store/qaylx0k6wg3awsamn2rqr41q9c8iwypg-wofi-1.1.2
linphoneqt 4.1.1 out /gnu/store/wlcviacpagxqnw1737x131if0fj7kzjv-linphoneqt-4.1.1
qbittorrent 4.2.5 out /gnu/store/29cjbdmz809pdvmylbx307l2p7zv0ma9-qbittorrent-4.2.5
pavucontrol 4.0 out /gnu/store/0ilm4lncr6gpg4h3zdvaxg1pibkm6934-pavucontrol-4.0
git 2.28.0 send-email /gnu/store/g5x559qk5yxs19jcxqplqv8b9y32vak2-git-2.28.0-send-email
flatpak 1.8.2 out /gnu/store/ywfsnccvlr0j6afb5vnl3z4ismvcfbz6-flatpak-1.8.2
xdg-desktop-portal 1.7.2 out /gnu/store/jkjx5dd7s3dp58p2hwsm8n2rm15z90ys-xdg-desktop-portal-1.7.2
xdg-desktop-portal-gtk 1.7.1 out /gnu/store/zhssisyp66vggjnqi5y8nb1xnk55vnw1-xdg-desktop-portal-gtk-1.7.1
xeyes 1.1.2 out /gnu/store/4jx8i19xd94d421n4acy2c1p0nn9h6d0-xeyes-1.1.2
emacs-polymode 0.2 out /gnu/store/gmm1y1xahg44yw9ghsf0kwd40w86q7mf-emacs-polymode-0.2
notmuch 0.31 out /gnu/store/0js5dwj8nxj0x0hxfllgzz4zxlsmyrz4-notmuch-0.31
msmtp 1.8.12 out /gnu/store/s1g4nskvnsx9ww0032xg23548h7mnwvq-msmtp-1.8.12
youtube-viewer 3.7.9 out /gnu/store/jm7q53lpda3a4rs2z7xv7cgig7adpncb-youtube-viewer-3.7.9
guile-curl 0.6 out /gnu/store/9nnclxjxakwx9yc381arqrk1nhld21b5-guile-curl-0.6
qemu 5.1.0 out /gnu/store/qzi36f60d2zlrdj9mg6b5y4i00w1fzfq-qemu-5.1.0
icecat 78.3.1-guix0-preview1 out /gnu/store/71fkykdajhkshsvzwrnqm4lsc6qgycg2-icecat-78.3.1-guix0-preview1
git 2.28.0 out /gnu/store/j66k3x1wz0bc6nh09dwa6i4jwahqcabi-git-2.28.0
guix 1.1.0-29.4e3ed9b out /gnu/store/irirj169rncank4y7i6n8qyiiqbs76qr-guix-1.1.0-29.4e3ed9b
ffmpeg 4.3.1 out /gnu/store/sg4yq9i0v1lphvs1vx33dw9mn801adrd-ffmpeg-4.3.1
pavucontrol-qt 0.15.0 out /gnu/store/9d0zp4wps5kpbqsl4yvjb96spb5xwz9b-pavucontrol-qt-0.15.0
clementine 1.3.1-2.4619a4c out /gnu/store/5392fysgmmi2jmn98maz9ihzn9ahzhw9-clementine-1.3.1-2.4619a4c
mako 1.4 out /gnu/store/k83v9wz6szf6k2p2sz1amvwqm687j3h2-mako-1.4
ungoogled-chromium 86.0.4240.75-0.c34a56d out /gnu/store/m0s9mrbirbssis5nn9agx8k4sl2ck0y9-ungoogled-chromium-86.0.4240.75-0.c34a56d
#+end_example


Thanks,

--
Joshua Branson
Sent from Emacs and Gnus
"You can have whatever you want, as long as you help enough other people get what they want." - Zig Ziglar
L
L
Luis Felipe wrote on 28 Oct 2020 22:48
(name . Joshua Branson)(address . jbranso@dismail.de)
teuOd2Q7Dbrx6oqUwBMU0LqyBVxb35-URBXKJ0OS0xjTrTxUYpoeKt5hTCxbfNWxDVq94Ez3Z2X0K-hPxzHJJczuP8PT8zL-SvDNYu5Bpu0=@protonmail.com
??????? Original Message ???????
On Tuesday, October 27, 2020 11:14 PM, Joshua Branson <jbranso@dismail.de> wrote:

[...]

Toggle quote (6 lines)
>
> #+BEGIN_SRC sh :results output :exports both
> time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)
> #+END_SRC


I haven't run this command because I don't know what it does (how will it affect my profile?), but if you think the results from this command would be more useful than what I posted to issue #44053, please let me know and I'll run it.
L
L
Ludovic Courtès wrote on 1 Nov 2020 11:28
control message for bug #44053
(address . control@debbugs.gnu.org)
87y2jlqsl6.fsf@gnu.org
severity 44053 important
quit
L
L
Ludovic Courtès wrote on 2 Nov 2020 12:12
Re: bug#44053: ‘xdg-mime-database’ profile hook is slow
(name . zimoun)(address . zimon.toutoune@gmail.com)
87sg9sm2qf.fsf@gnu.org
Hi,

Joshua Branson <jbranso@dismail.de> skribis:

Toggle quote (19 lines)
> The "XDG MIME database" takes a while.
>
> #+BEGIN_SRC sh :results output :exports both
> time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)
> #+END_SRC
>
>
> #+RESULTS:
> : The following profile hook will be built:
> : /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
> : building XDG MIME database...
> : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
> : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv
> : /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database
> :
> : real 0m43.716s
> : user 0m3.626s
> : sys 0m0.258s

I found that the MIME database is computed only over the subset of the
packages in your profile that provide ‘share/mime/packages’, plus
‘shared-mime-info’.

In my profile, only LibreOffice provides that directory, so the union is
made over these two directories and that’s what ‘update-mime-database’
works on.

Apart from LibreOffice, I found that ‘share/mime/packages’ is provided
by at least: hugin, gcr, fontforge. Most GUI packages don’t have it.
So in practice, we’re often rebuilding the exact same database.

Here’s the time taken by ‘update-mime-database’ alone:

Toggle snippet (44 lines)
ludo@ribbon ~$ mkdir -p /tmp/mime/share/mime/packages
ludo@ribbon ~$ cd /tmp/mime/share/mime/packages
ludo@ribbon /tmp/mime/share/mime/packages$ for i in $(guix build libreoffice ^C
ludo@ribbon /tmp/mime/share/mime/packages$ for i in ~/.guix-profile/share/mime/packages/* ; do ln -s $i ; done
ludo@ribbon /tmp/mime/share/mime/packages$ ls -l
totalo 8
lrwxrwxrwx 1 ludo users 64 Nov 2 11:46 freedesktop.org.xml -> /home/ludo/.guix-profile/share/mime/packages/freedesktop.org.xml
lrwxrwxrwx 1 ludo users 60 Nov 2 11:46 libreoffice.xml -> /home/ludo/.guix-profile/share/mime/packages/libreoffice.xml
ludo@ribbon /tmp/mime/share/mime/packages$ cd /tmp/mime/
ludo@ribbon /tmp/mime$ time update-mime-database -V /tmp/mime/share/mime
Updating MIME database in /tmp/mime/share/mime...

Parsing source file /tmp/mime/share/mime/packages/freedesktop.org.xml...
Parsing source file /tmp/mime/share/mime/packages/libreoffice.xml...
Wrote 1124 strings at 2c - 6310
Wrote aliases at 6310 - 6be4
Wrote parents at 6be4 - 8468
Wrote literal globs at 8468 - 855c
Wrote suffix globs at 855c - 13514
Wrote full globs at 13514 - 13554
Wrote magic at 13554 - 2065c
Wrote namespace list at 2065c - 20798
Wrote icons list at 20798 - 2079c
Wrote generic icons list at 2079c - 213b8
Wrote types list at 213b8 - 22020

Note that '/tmp/mime/share' is not in the search path
set by the XDG_DATA_HOME and XDG_DATA_DIRS
environment variables, so applications may not
be able to find it until you set them. The
directories currently searched are:

- /home/ludo/.local/share
- /home/ludo/.guix-profile/share
- /run/current-system/profile/share
- /home/ludo/.guix-profile/share
- /run/current-system/profile/share


real 0m2.166s
user 0m0.278s
sys 0m0.056s

To be compared with:

Toggle snippet (13 lines)
$ drv="$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)"
$ time guix build --check $drv
The following profile hook will be built:
/gnu/store/jrmwsxpz3wmq37zx29lvb0r9nvcmdviz-xdg-mime-database.drv
building XDG MIME database...
successfully built /gnu/store/jrmwsxpz3wmq37zx29lvb0r9nvcmdviz-xdg-mime-database.drv
/gnu/store/npvj2sr9kxx48znh7zc8zmzwzs6brc90-xdg-mime-database

real 0m3.670s
user 0m1.428s
sys 0m0.029s

Thus, the build time itself is entirely taken by ‘update-mime-database’
(the 1.4s above is “overhead” in ‘guix build’ it would seem).

Ludo’.
L
L
Ludovic Courtès wrote on 2 Nov 2020 13:53
(name . zimoun)(address . zimon.toutoune@gmail.com)
87ft5rncn5.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (4 lines)
> Apart from LibreOffice, I found that ‘share/mime/packages’ is provided
> by at least: hugin, gcr, fontforge. Most GUI packages don’t have it.
> So in practice, we’re often rebuilding the exact same database.

On closer inspection, the time-consuming bit is processing
‘share/mime/packages/freedesktop.org.xml’ (from ‘shared-mime-info’),
which is quite large and leads to the creation of hundreds of file. We
end up re-processing it every time. This is particularly wasteful
because the ‘shared-mime-info’ package already contains the result of
applying ‘update-mime-database’ to itself.

Unfortunately, AIUI,

update-mime-database(X ∪ Y) ≠ update-mime-database(X) ∪ update-mime-database(Y)

(For example, the files
‘share/mime/{globs,magic,XMLnamespaces,subclasses,aliases,types,generic-icons,icons,treemagic,mime.cache}’
concatenate info from X and Y.)

So it would seem we cannot simply used the pre-built database from
‘shared-mime-info’ and merge it with that of the other packages, at
least not without changing ‘update-mime-database’ or re-implementing
parts of it on our side.

Ludo’.
L
L
Ludovic Courtès wrote on 2 Nov 2020 15:04
(name . zimoun)(address . zimon.toutoune@gmail.com)
87blgflush.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (13 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Apart from LibreOffice, I found that ‘share/mime/packages’ is provided
>> by at least: hugin, gcr, fontforge. Most GUI packages don’t have it.
>> So in practice, we’re often rebuilding the exact same database.
>
> On closer inspection, the time-consuming bit is processing
> ‘share/mime/packages/freedesktop.org.xml’ (from ‘shared-mime-info’),
> which is quite large and leads to the creation of hundreds of file. We
> end up re-processing it every time. This is particularly wasteful
> because the ‘shared-mime-info’ package already contains the result of
> applying ‘update-mime-database’ to itself.

Based on these observations, I added a fast path to the
‘xdg-mime-database’ hook:


It’s down to 0s in the common case where the profile doesn’t contain any
packages providing ‘share/mime/packages’.

Ludo’.
Z
Z
zimoun wrote on 4 Nov 2020 00:32
(name . Ludovic Courtès)(address . ludo@gnu.org)
86blgeggo8.fsf@gmail.com
Hi,

Toggle quote (2 lines)
Clever workaround! What are now the performances on previous examples
(same profiles and same packages)?


Toggle quote (4 lines)
> Unfortunately, AIUI,
>
> update-mime-database(X ∪ Y) ≠ update-mime-database(X) ∪ update-mime-database(Y)

Hum? Interesting… well weird! If it is not a “morphism” then it is not
really an updater. Anyway.

I am curious about the comparison:

time(update-mime-database(X ∪ Y))
vs
time(update-mime-database(X)) + time(update-mime-database(Y))
vs
time(update-mime-database(X ∩ Y))


Toggle quote (5 lines)
> So it would seem we cannot simply used the pre-built database from
> ‘shared-mime-info’ and merge it with that of the other packages, at
> least not without changing ‘update-mime-database’ or re-implementing
> parts of it on our side.

’shared-mime-info’ is simply a package, right? So what does it means:
«Find a way to avoid reprocessing 'shared-mime-info'» in:

Toggle snippet (8 lines)
;; the database. TODO: Find a way to avoid reprocessing
;; 'shared-mime-info', which is the most expensive one.
[...]
(invoke #+(file-append shared-mime-info
"/bin/update-mime-database")
destdir)))))))

? Well, I miss why and how ’shared-mime-info’ could be the culprit here
or what could be improved with “shared-mime-info”? Maybe replace the
“lazy reference” by a strict one if it makes sense.


All the best,
simon
L
L
Luis Felipe wrote on 5 Nov 2020 15:47
(name . zimoun)(address . zimon.toutoune@gmail.com)
wfCkfOh1kZkBBqR2DYrffbz9xGwrXpB5lNfNNbLywaXO2i620ONaFST7pkyF_vskC20eUnLuV36YyO6t8v6__7oyVbOzaCoba1muCHDP0S4=@protonmail.com
??????? Original Message ???????
On Tuesday, November 3, 2020 11:32 PM, zimoun <zimon.toutoune@gmail.com> wrote:

Toggle quote (7 lines)
> Hi,
>
> > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54
>
> Clever workaround! What are now the performances on previous examples
> (same profiles and same packages)?

In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).


$ time guix package -i perl --max-jobs=1
Se instalará el siguiente paquete:
perl 5.30.2

Se construirá la siguiente derivación:
/gnu/store/48imlmfi2hjmp4ksyidvy99c31ahj44z-profile.drv

Se descargarán 0,6 MB
construyendo empaquetado de certificados de CA...
freetype-2.10.4 555KiB 707KiB/s 00:01 [##################] 100.0%

injertando 5 paquetes en /gnu/store/s4s7xzswar962p9w1nvf0hfcgmnxyfsh-at-spi2-core-2.34.0.drv ...
injertando 1 paquete en /gnu/store/80n340f7bnba0ivp00z10a32rz8g8wsx-fontconfig-2.13.1.drv ...
injertando 4 paquetes en /gnu/store/hm3a895l251mckzb9bhiglcwfb6xli5n-at-spi2-atk-2.34.1.drv ...
injertando 6 paquetes en /gnu/store/9vwl8s2h7rbfhvimsarzc1q4gnclwpk2-cairo-1.16.0.drv ...
injertando 6 paquetes en /gnu/store/k6gajkzfq4drpcrimbwxk1ls0h7z1nzw-cairo-1.16.0.drv ...
injertando 4 paquetes en /gnu/store/klq5sp427s80d0mhz3b5x48xpmhf3ila-gdk-pixbuf-2.40.0.drv ...
injertando 5 paquetes en /gnu/store/ijkpcwanfpj02i0rxk4ys0kfdpj6sz69-ghostscript-with-cups-9.52.drv ...
generando la caché de esquemas de GLib...
injertando 2 paquetes en /gnu/store/kg8ca3wz01ibhbm5wk707r44jjnrxdg1-harfbuzz-2.6.4.drv ...
construyendo el directorio de manuales Info...
injertando 4 paquetes en /gnu/store/97bgi57qs077pcqggvr7di0vhchlshk2-libxft-2.3.3.drv ...
construyendo la base de datos de páginas de manual...
injertando 1 paquete en /gnu/store/gb85hxzczf4m0crzwqpw2imxpr1d67nx-mkfontscale-1.2.1.drv ...
injertando 9 paquetes en /gnu/store/ddiqkw9r0p1lhvqmdlcyrzf8v900pm4r-pango-1.42.4.drv ...
injertando 1 paquete en /gnu/store/xd0ca96gvh3vn9bp3bcmczczcxm0bynd-mkfontdir-1.0.7.drv ...
injertando 11 paquetes en /gnu/store/s53s5qx304b90l0dp734lqi42whwd0vd-librsvg-2.40.21.drv ...
construyendo el directorio de tipografías...
injertando 5 paquetes en /gnu/store/qg628jlnjmiyv964c0i4s7bbkwy32c6c-gdk-pixbuf+svg-2.40.0.drv ...
injertando 8 paquetes en /gnu/store/1mj8cjf43nqd9r5151lqrf1sdd1sjg16-pango-1.44.7.drv ...
injertando 8 paquetes en /gnu/store/d42vnhn9grcsyiibdgkiryqb7rxhb89j-poppler-0.86.1.drv ...
construyendo la caché de ficheros desktop XDG...
injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ...
construyendo la base de datos MIME XDG...
injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ...
injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ...
injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ...
creando la caché de temas de iconos de GTK+...
construyendo los ficheros de caché para los métodos de entrada de GTK+...
construyendo perfil con 86 paquetes...


real 8m38,121s
user 0m2,742s
sys 0m0,338s
Z
Z
zimoun wrote on 5 Nov 2020 17:10
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
86imajg4xs.fsf@gmail.com
Hi,

On Thu, 05 Nov 2020 at 14:47, Luis Felipe <luis.felipe.la@protonmail.com> wrote:

Toggle quote (8 lines)
>> Clever workaround! What are now the performances on previous examples
>> (same profiles and same packages)?
>
> In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).
>
>
> $ time guix package -i perl --max-jobs=1

[...]

Toggle quote (4 lines)
> real 8m38,121s
> user 0m2,742s
> sys 0m0,338s

Thanks. But it is hard to conclude because the previous timing is
missing. The right experiment should be:

Toggle snippet (13 lines)
old=c73496f433044a76003b33c3855bb35ecd0df87f
new=5e7cf66fb35780f930ad0bc5fe21ac330df4411d
guix time-machine --commit=$old -- help
guix time-machine --commit=$new -- help

drv=$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)

guix package -I |wc -l
time guix time-machine --commit=$old -- build --check $drv
time guix time-machine --commit=$new -- build --check $drv


And personally, I get:

18

Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
The following profile hook will be built:
/gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drv
building XDG MIME database...
successfully built /gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drv
/gnu/store/hrcj1h0hl9f6hm1akh682zj1mhpwbx4c-xdg-mime-database

real 0m8.063s
user 0m6.226s
sys 0m0.282s

Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
The following profile hook will be built:
/gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drv
building XDG MIME database...
successfully built /gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drv
/gnu/store/hrcj1h0hl9f6hm1akh682zj1mhpwbx4c-xdg-mime-database

real 0m8.016s
user 0m6.223s
sys 0m0.243s

Well, I have only 18 packages in my ~/.guix-profile. That’s why the
difference is not visible and the “time-machine” is totally dominating
here.


All the best,
simon
L
L
Luis Felipe wrote on 5 Nov 2020 18:59
(name . zimoun)(address . zimon.toutoune@gmail.com)
mOI7DET2ltI7DDsuW_psxIqhFvJxKjWOWmGpwEJKTL5rN49dnI-dvMMxquvNl33w9N9gxqJT8KA8hanP03RENzWZTwUY4Y7cNvctDfucOik=@protonmail.com
??????? Original Message ???????
On Thursday, November 5, 2020 4:10 PM, zimoun <zimon.toutoune@gmail.com> wrote:

Toggle quote (19 lines)
> Hi,
>
> On Thu, 05 Nov 2020 at 14:47, Luis Felipe luis.felipe.la@protonmail.com wrote:
>
> > > Clever workaround! What are now the performances on previous examples
> > > (same profiles and same packages)?
> >
> > In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).
> > $ time guix package -i perl --max-jobs=1
>
> [...]
>
> > real 8m38,121s
> > user 0m2,742s
> > sys 0m0,338s
>
> Thanks. But it is hard to conclude because the previous timing is
> missing.

I don't understand, I posted my original timing in https://issues.guix.gnu.org/44053#1.



Toggle quote (16 lines)
> The right experiment should be:
>
> --8<---------------cut here---------------start------------->8---
> old=c73496f433044a76003b33c3855bb35ecd0df87f
> new=5e7cf66fb35780f930ad0bc5fe21ac330df4411d
> guix time-machine --commit=$old -- help
> guix time-machine --commit=$new -- help
>
> drv=$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)
>
> guix package -I |wc -l
> time guix time-machine --commit=$old -- build --check $drv
> time guix time-machine --commit=$new -- build --check $drv
> --8<---------------cut here---------------end--------------->8---


I get this:

yo@gnu: ~? guix package -I |wc -l
85

yo@gnu: ~? time guix time-machine --commit=$old -- build --check $drv
Actualizando el canal 'guix' del repositorio Git en 'https://git.savannah.gnu.org/git/guix.git'...
Actualizando el canal 'luflac-x' del repositorio Git en 'git@gitlab.com:luis-felipe/guix-channel-x.git'...
Se construirá la siguiente extensión del perfil:
/gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drv
construyendo la base de datos MIME XDG...
/gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drv construido satisfactoriamente
/gnu/store/i4qslq2517pwkwb5dyivrg8idsjd8n11-xdg-mime-database

real 0m56,245s
user 0m4,324s
sys 0m0,218s


yo@gnu: ~? time guix time-machine --commit=$new -- build --check $drv
Actualizando el canal 'guix' del repositorio Git en 'https://git.savannah.gnu.org/git/guix.git'...
Actualizando el canal 'luflac-x' del repositorio Git en 'git@gitlab.com:luis-felipe/guix-channel-x.git'...
Se construirá la siguiente extensión del perfil:
/gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drv
construyendo la base de datos MIME XDG...
/gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drv construido satisfactoriamente
/gnu/store/i4qslq2517pwkwb5dyivrg8idsjd8n11-xdg-mime-database

real 0m43,272s
user 0m4,214s
sys 0m0,200s
L
L
Ludovic Courtès wrote on 6 Nov 2020 10:10
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
87blga3l7e.fsf@gnu.org
Hi,

Luis Felipe <luis.felipe.la@protonmail.com> skribis:

Toggle quote (12 lines)
> ??????? Original Message ???????
> On Tuesday, November 3, 2020 11:32 PM, zimoun <zimon.toutoune@gmail.com> wrote:
>
>> Hi,
>>
>> > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54
>>
>> Clever workaround! What are now the performances on previous examples
>> (same profiles and same packages)?
>
> In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).

Please note that the change above addresses only one specific source of
slowness, the ‘xdg-mime-database’ hook, and only in specific cases.

It’s good to look at the overall timing of ‘guix install’, because
that’s what matters in the end, but as we work on optimizing it, we have
to look at specific aspects of it.

Toggle quote (2 lines)
> $ time guix package -i perl --max-jobs=1

[...]

Toggle quote (14 lines)
> injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ...
> construyendo la base de datos MIME XDG...
> injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ...
> injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ...
> injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ...
> creando la caché de temas de iconos de GTK+...
> construyendo los ficheros de caché para los métodos de entrada de GTK+...
> construyendo perfil con 86 paquetes...
>
>
> real 8m38,121s
> user 0m2,742s
> sys 0m0,338s

Here it’s likely that grafting is what’s taking the most time on a
spinning disk.

We should hack (guix status) so it can optionally prefix each event with
a timestamp.

As far as ‘xdg-mime-database’ is concerned, it should be down to 0s,
unless your profile contains one of the packages I cited (libreoffice,
gcr, hugin, etc.).

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 6 Nov 2020 10:12
(name . zimoun)(address . zimon.toutoune@gmail.com)
875z6i3l38.fsf@gnu.org
Hello,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (15 lines)
>> So it would seem we cannot simply used the pre-built database from
>> ‘shared-mime-info’ and merge it with that of the other packages, at
>> least not without changing ‘update-mime-database’ or re-implementing
>> parts of it on our side.
>
> ’shared-mime-info’ is simply a package, right? So what does it means:
> «Find a way to avoid reprocessing 'shared-mime-info'» in:
>
> ;; the database. TODO: Find a way to avoid reprocessing
> ;; 'shared-mime-info', which is the most expensive one.
> [...]
> (invoke #+(file-append shared-mime-info
> "/bin/update-mime-database")
> destdir)))))))

‘shared-mime-info’ contains ‘share/mime/freedesktop.org.xml’, which is
by far where ‘update-mime-database’ spends most of its time.

But it’s wasteful because ‘shared-mime-info’ already contains the result
of running ‘update-mime-database’ on itself.

HTH!

Ludo’.
Z
Z
zimoun wrote on 6 Nov 2020 13:55
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
864km2fxux.fsf@gmail.com
Hi,

On Thu, 05 Nov 2020 at 17:59, Luis Felipe <luis.felipe.la@protonmail.com> wrote:

Therefore, it is nice improvement! :-)

Toggle quote (8 lines)
> real 0m56,245s
> user 0m4,324s
> sys 0m0,218s

> real 0m43,272s
> user 0m4,214s
> sys 0m0,200s

Because, I bet that most of the time is spent in the “time-machine”
part. I mean, compare with:

time guix time-machine --commit=$new -- help


All the best,
simon
L
L
Luis Felipe wrote on 6 Nov 2020 14:57
(name . Ludovic Courtès)(address . ludo@gnu.org)
bMZ76QeMfjltWvLabLrXiBZyPPfTxrDHumnzqmIyDYRiU4BpAXwS8Z0zo4fnvHlbFLP23GpP27GJBS6Q6YZYDJ7sXXRzMOFJGtvujtprDNo=@protonmail.com
??????? Original Message ???????
On Friday, November 6, 2020 9:10 AM, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (26 lines)
> Hi,
>
> Luis Felipe luis.felipe.la@protonmail.com skribis:
>
> > ??????? Original Message ???????
> > On Tuesday, November 3, 2020 11:32 PM, zimoun zimon.toutoune@gmail.com wrote:
> >
> > > Hi,
> > >
> > > > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54
> > >
> > > Clever workaround! What are now the performances on previous examples
> > > (same profiles and same packages)?
> >
> > In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).
>
> Please note that the change above addresses only one specific source of
> slowness, the ‘xdg-mime-database’ hook, and only in specific cases.
>
> It’s good to look at the overall timing of ‘guix install’, because
> that’s what matters in the end, but as we work on optimizing it, we have
> to look at specific aspects of it.
>
> > $ time guix package -i perl --max-jobs=1


Yeah, sorry I was more focused on the general issue (#44053), but I understand.


Toggle quote (17 lines)
> [...]
>
> > injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ...
> > construyendo la base de datos MIME XDG...
> > injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ...
> > injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ...
> > injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ...
> > creando la caché de temas de iconos de GTK+...
> > construyendo los ficheros de caché para los métodos de entrada de GTK+...
> > construyendo perfil con 86 paquetes...
> > real 8m38,121s
> > user 0m2,742s
> > sys 0m0,338s
>
> Here it’s likely that grafting is what’s taking the most time on a
> spinning disk.

It does take some time, but since I can see the output change from grafting to grafting, I at least can tell guix is doing something, so I just let it be.

Compared to grafting, the last step "construyendo perfil con X paquetes..." ("building profile with X packages..."), just stays there without change for several minutes, so it actually seems slower to me. Initially, I thought that guix had frozen.

Also, even though, the "building profile" step has a throbber (| / - \) to indicate that something is being done, it frequently stops in one of the frames of the sequence and stays there until the end.


Toggle quote (7 lines)
> We should hack (guix status) so it can optionally prefix each event with
> a timestamp.
>
> As far as ‘xdg-mime-database’ is concerned, it should be down to 0s,
> unless your profile contains one of the packages I cited (libreoffice,
> gcr, hugin, etc.).

Yes, I have Libreoffice installed.
L
L
Ludovic Courtès wrote on 8 Nov 2020 17:31
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
87pn4nvmh0.fsf@gnu.org
Hi,

Luis Felipe <luis.felipe.la@protonmail.com> skribis:

Toggle quote (4 lines)
> Compared to grafting, the last step "construyendo perfil con X paquetes..." ("building profile with X packages..."), just stays there without change for several minutes, so it actually seems slower to me. Initially, I thought that guix had frozen.
>
> Also, even though, the "building profile" step has a throbber (| / - \) to indicate that something is being done, it frequently stops in one of the frames of the sequence and stays there until the end.

Interesting, so we should profile that step and see what can be done. I
suspect it’s I/O-bound, but maybe we can at least improve feedback.

Thanks,
Ludo’.
Z
Z
zimoun wrote on 23 Mar 2022 13:38
Re: bug#44053: Poor profile generation performance on spinning disks
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 44053@debbugs.gnu.org)
86tubozvw6.fsf@gmail.com
Hi Maxim,

About performance reported by [1],


the commit [2],


improves the situation. Although I agree the performance are poor for
spinning disk, what is the next actionable step associated to this
report?

Maybe we can close, WDYT?


Cheers,
simon
M
M
Maxim Cournoyer wrote on 23 Mar 2022 17:17
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 44053@debbugs.gnu.org)
87ils4y773.fsf@gmail.com
Hi Simon,

zimoun <zimon.toutoune@gmail.com> writes:

Toggle quote (16 lines)
> Hi Maxim,
>
> About performance reported by [1],
>
> 1: <http://issues.guix.gnu.org/issue/44053>
>
> the commit [2],
>
> 2: <https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54>
>
> improves the situation. Although I agree the performance are poor for
> spinning disk, what is the next actionable step associated to this
> report?
>
> Maybe we can close, WDYT?

This issue is more about the time it takes to generate the file
hierarchy of the profile, rather than with the profile hooks. The
profile hooks should be tolerably fast at this point.

There is perhaps more optimization that can be done at the level of
populating the symbolic links of a profile, so I'd like to leave this
open until someone gets around to profiling what is slow and whether
something more can be done.

Thanks,

Maxim
Z
Z
zimoun wrote on 23 Mar 2022 17:54
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 44053@debbugs.gnu.org)
CAJ3okZ3YV9d25YiW2FzjE585ZZfU18s1no1xq-4_bRorcQ0pDQ@mail.gmail.com
Hi Maxim,

On Wed, 23 Mar 2022 at 17:17, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (4 lines)
> This issue is more about the time it takes to generate the file
> hierarchy of the profile, rather than with the profile hooks. The
> profile hooks should be tolerably fast at this point.

Yeah, I have seen your profiling about man-db, cool!

Toggle quote (5 lines)
> There is perhaps more optimization that can be done at the level of
> populating the symbolic links of a profile, so I'd like to leave this
> open until someone gets around to profiling what is slow and whether
> something more can be done.

Ok. So we need to provide a "configuration". For instance, a typical
manifest and Guix revision to compare with. WDYT?


Cheers,
simon
J
J
jbranso wrote on 27 Aug 2023 00:11
Re: Poor profile generation performance on spinning disks
1b6d61fc282998c93915f26914308edd@dismail.de
Shall we go ahead and close this bug report?

I do not recall a recent time that someone complained about Guix System's slow profile generation on hard drives. I currently use guix system on an SSD...but I could install Guix on a Dell Optiplex 7010 on a hard drive to test if you all would like.

Thanks,

Joshua
Attachment: file
L
L
Luis Felipe wrote on 27 Aug 2023 19:59
(address . jbranso@dismail.de)
dzkxWsiO2j2SlWZLTruNGXIELUDvT0_JRUOCJEnE2zlG13OyB1x6ohriYL8-mUHgg2mjtLoOXKASNm-bRVLw6tgkUrVgDH46PaNxdT4oXm8=@protonmail.com
Hi,

On Saturday, August 26th, 2023 at 22:11, jbranso@dismail.de <jbranso@dismail.de> wrote:

Toggle quote (4 lines)
>

> Shall we go ahead and close this bug report?

I'm okay with that.


Toggle quote (2 lines)
> I do not recall a recent time that someone complained about Guix System's slow profile generation on hard drives. I currently use guix system on an SSD...but I could install Guix on a Dell Optiplex 7010 on a hard drive to test if you all would like.

I still use the same computer with the same hard drive, and performance has improved when installing packages. Running

$ time guix package -i perl --max-jobs=1

initially took

(87 packages in profile)
real 8m3,043s
user 0m2,656s
sys 0m0,239s

But now (guix 17fadbb), it takes

(96 packages in profile)
real 3m51,058s
user 0m7,490s
sys 0m0,788s

Whether that's acceptable or not when using spinning disks, I don't know. I'll step aside :)
Attachment: publickey - luis.felipe.la@protonmail.com - 0x12DE1598.asc
Attachment: signature.asc
S
S
Simon Tournier wrote on 28 Aug 2023 14:52
control message for bug #44053
(address . control@debbugs.gnu.org)
87r0nnpbvh.fsf@gmail.com
severity 44053 normal
quit
L
L
Ludovic Courtès wrote on 29 Aug 2023 11:19
Re: Poor profile generation performance on spinning disks
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
874jkiw6gz.fsf@gnu.org
Hi Luis,

Luis Felipe <luis.felipe.la@protonmail.com> skribis:

Toggle quote (18 lines)
> I still use the same computer with the same hard drive, and performance has improved when installing packages. Running
>
> $ time guix package -i perl --max-jobs=1
>
> initially took
>
> (87 packages in profile)
> real 8m3,043s
> user 0m2,656s
> sys 0m0,239s
>
> But now (guix 17fadbb), it takes
>
> (96 packages in profile)
> real 3m51,058s
> user 0m7,490s
> sys 0m0,788s

I think it’s not good.

Could you time just profile generation itself?

To do that, you need to find the profile generation and then to rebuild
it, along these lines:

DRV=$(guix gc --derivers $(readlink -f ~/.guix-profile))
time guix build --check $DRV

The thread contains an analysis of specific sources of slowness, such as
the cost of building the XDG MIME database:


I think we’re not done; we can probably do better by first figuring out
which parts take time (profiling) and then addressing specific issues.

Ludo’.
L
L
Luis Felipe wrote on 29 Aug 2023 23:51
(name . Ludovic Courtès)(address . ludo@gnu.org)
mSj2EYYJvpknifEjHBEvx96jRolchzWAitRR1aPwzJ_5vubGRYl9TIPoBxZrcf0tmN-bcXiXfkpXTL-vNy-S2ekb9K4_D6somthxsXamoYQ=@protonmail.com
Hey Ludo, everyone,

------- Original Message -------
On Tuesday, August 29th, 2023 at 09:19, Ludovic Courtès <ludo@gnu.org> wrote:


Toggle quote (45 lines)
> Hi Luis,
>

> Luis Felipe luis.felipe.la@protonmail.com skribis:
>

> > I still use the same computer with the same hard drive, and performance has improved when installing packages. Running
> >

> > $ time guix package -i perl --max-jobs=1
> >

> > initially took
> >

> > (87 packages in profile)
> > real 8m3,043s
> > user 0m2,656s
> > sys 0m0,239s
> >

> > But now (guix 17fadbb), it takes
> >

> > (96 packages in profile)
> > real 3m51,058s
> > user 0m7,490s
> > sys 0m0,788s
>

>

> I think it’s not good.
>

> Could you time just profile generation itself?
>

> To do that, you need to find the profile generation and then to rebuild
> it, along these lines:
>

> DRV=$(guix gc --derivers $(readlink -f ~/.guix-profile))
> time guix build --check $DRV

The above results in

real 1m28,841s
user 0m2,169s
sys 0m0,450s

The complete output follows (several collisions are reported):

The following derivation will be built:
/gnu/store/mjscvlgfpncscqp2bp92g0nmgbqisdzi-profile.drv
construyendo perfil con 96 paquetes...

warning: collision encountered:
/gnu/store/sy5ya0694zxk07miamhdmzizx999440v-gdk-pixbuf-loaders-cache-file/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/p154yhnkrf6hc6x5r2src7ns5nqz1ain-emacs-next-29.0.92/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/pcz2lw1ycp28s2jzxh4rbnbzi1w037kv-swaynotificationcenter-0.8.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/f22hqlkbpb8v29l3x61yiqh21nj4bjgw-gedit-44.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/fhswqik2r3zb23cis8h1gxqfasmjydnr-ibus-libpinyin-1.15.2/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/6hjr0dwwm3fgfi91hifalf97rkhca05g-dconf-editor-43.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/dmm1yzf8z9ydvvdrr49443xg24803gvv-devhelp-43.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/1w3x53l7qf07xqp3gnb8np1qhj1sd7ff-easytag-2.4.3/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/d25s0w8wqz51499j7ahhs5a2jil9hlz0-flatpak-1.14.4/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/vgc2460l5flvrfqynvjr3iarrxci549d-gfeeds-2.2.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/yf6dy9pfjnfss1hjz2mjv7ls555hlg2h-glade-3.40.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/ibg7cibwpj70pr2rz1isrckrjk9g206w-gnome-music-42.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/p7p8ri4dvnn2cp2pj4s828fsd7l9vj8v-gnumeric-1.12.52/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/z5h07miv53809z225y6m7205w1780jfm-ibus-1.5.27/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/cx9xxn3m85pryr40nxaan47pgfwmcv3r-ibus-anthy-1.5.14/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/w9b0g252l1av1yrn1lkq3qlq61qzp3wq-ibus-speech-to-text-0.4.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/95iqnynknlasnaclji1bkqab7r7f9wgd-inkscape-1.2.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/1027d5nq0i9k5balfnl157gj0vqi8ai2-peek-1.5.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/rna20hbk5b7pdmxk8lv3q43v86fqzb6g-sysprof-3.45.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/18w4iisx7fkyx4whzhypn46g1dkdb1mv-gtk-4.8.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/7qj504g7vp6rni85h8p2wnd2ip0rr5l5-yelp-42.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/cy5s40wfc0glx34578ic8frjx4nh2r2g-dconf-0.40.0/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/bb8ijpv1y3wpppfqd7r0pkk25xckag19-librsvg-2.54.5/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
/gnu/store/1ig678c8vqxvr60x8swmc5wriga7sjf3-gdk-pixbuf-2.42.8/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
warning: choosing /gnu/store/sy5ya0694zxk07miamhdmzizx999440v-gdk-pixbuf-loaders-cache-file/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache

warning: collision encountered:
/gnu/store/b2nmvdcgp0dq1a2i2699c338yf88jsh9-ungoogled-chromium-112.0.5615.165-1/lib/libEGL.so
/gnu/store/qiw0rpaj4rlgpldh12xac03n8rf301zm-mesa-23.1.4/lib/libEGL.so
warning: choosing /gnu/store/b2nmvdcgp0dq1a2i2699c338yf88jsh9-ungoogled-chromium-112.0.5615.165-1/lib/libEGL.so

warning: collision encountered:
/gnu/store/b2nmvdcgp0dq1a2i2699c338yf88jsh9-ungoogled-chromium-112.0.5615.165-1/lib/libGLESv2.so
/gnu/store/qiw0rpaj4rlgpldh12xac03n8rf301zm-mesa-23.1.4/lib/libGLESv2.so
warning: choosing /gnu/store/b2nmvdcgp0dq1a2i2699c338yf88jsh9-ungoogled-chromium-112.0.5615.165-1/lib/libGLESv2.so

warning: collision encountered:
/gnu/store/p154yhnkrf6hc6x5r2src7ns5nqz1ain-emacs-next-29.0.92/share/info/transient.info.gz
/gnu/store/j1qf5zl8pysg608l7szwan0qk25925wg-emacs-transient-0.4.1/share/info/transient.info.gz
warning: choosing /gnu/store/p154yhnkrf6hc6x5r2src7ns5nqz1ain-emacs-next-29.0.92/share/info/transient.info.gz

warning: collision encountered:
/gnu/store/9qxlhkq84jiw0g2g0ywd9v827291rzl1-xdg-desktop-database/share/applications/mimeinfo.cache
/gnu/store/f22hqlkbpb8v29l3x61yiqh21nj4bjgw-gedit-44.1/share/applications/mimeinfo.cache
/gnu/store/ibg7cibwpj70pr2rz1isrckrjk9g206w-gnome-music-42.1/share/applications/mimeinfo.cache
/gnu/store/1027d5nq0i9k5balfnl157gj0vqi8ai2-peek-1.5.1/share/applications/mimeinfo.cache
warning: choosing /gnu/store/9qxlhkq84jiw0g2g0ywd9v827291rzl1-xdg-desktop-database/share/applications/mimeinfo.cache

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/subclasses
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/subclasses
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/subclasses
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/subclasses

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/globs2
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/globs2
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/globs2
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/globs2

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/mime.cache
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/mime.cache
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/mime.cache
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/mime.cache

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/icons
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/icons
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/icons
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/icons

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/XMLnamespaces
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/XMLnamespaces
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/XMLnamespaces
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/XMLnamespaces

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/aliases
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/aliases
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/aliases
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/aliases

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/globs
/gnu/store/pyfj2gb5c0mf5p1a56r9gwjylyl5kl7l-cambalache-0.12.1/share/mime/globs
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/globs
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/globs

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.wordprocessingml.document.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.openxmlformats-officedocument.wordprocessingml.document.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.wordprocessingml.document.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.spreadsheetml.template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.openxmlformats-officedocument.spreadsheetml.template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.spreadsheetml.template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text-web.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.text-web.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text-web.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/x-cbt.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/x-cbt.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/x-cbt.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-excel.template.macroenabled.12.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.ms-excel.template.macroenabled.12.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-excel.template.macroenabled.12.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-excel.sheet.macroenabled.12.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.ms-excel.sheet.macroenabled.12.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-excel.sheet.macroenabled.12.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-word.document.macroenabled.12.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.ms-word.document.macroenabled.12.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-word.document.macroenabled.12.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.presentation-template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.presentation-template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.presentation-template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text-master.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.text-master.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text-master.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.wordprocessingml.template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.openxmlformats-officedocument.wordprocessingml.template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.wordprocessingml.template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.graphics.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.graphics.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.graphics.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.spreadsheet-flat-xml.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.spreadsheet-flat-xml.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.spreadsheet-flat-xml.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.text.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text-template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.text-template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.text-template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.spreadsheet-template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.spreadsheet-template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.spreadsheet-template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openofficeorg.extension.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.openofficeorg.extension.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openofficeorg.extension.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.sun.xml.math.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.sun.xml.math.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.sun.xml.math.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.graphics-template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.graphics-template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.graphics-template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-powerpoint.presentation.macroenabled.12.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.ms-powerpoint.presentation.macroenabled.12.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-powerpoint.presentation.macroenabled.12.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-word.template.macroenabled.12.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.ms-word.template.macroenabled.12.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-word.template.macroenabled.12.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.presentationml.presentation.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.openxmlformats-officedocument.presentationml.presentation.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.openxmlformats-officedocument.presentationml.presentation.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-excel.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.ms-excel.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.ms-excel.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.spreadsheet.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.spreadsheet.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.spreadsheet.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.sun.xml.draw.template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.sun.xml.draw.template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.sun.xml.draw.template.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.presentation-flat-xml.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.oasis.opendocument.presentation-flat-xml.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.oasis.opendocument.presentation-flat-xml.xml

warning: collision encountered:
/gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.sun.xml.impress.template.xml
/gnu/store/7z9cgiq7dkajqfk991lniv77s1kbag28-shared-mime-info-1.15/share/mime/application/vnd.sun.xml.impress.template.xml
warning: choosing /gnu/store/rr3rr5p92zxskrx3vac4860gqpcy7wkf-xdg-mime-database/share/mime/application/vnd.sun.xml.impress.template.
This message was truncated. Download the full message here.
Attachment: publickey - luis.felipe.la@protonmail.com - 0x12DE1598.asc
Attachment: signature.asc
L
L
Ludovic Courtès wrote on 9 Sep 2023 13:02
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
87zg1vy5ho.fsf@gnu.org
Hi Luis,

Luis Felipe <luis.felipe.la@protonmail.com> skribis:

Toggle quote (16 lines)
>> Could you time just profile generation itself?
>>
>
>> To do that, you need to find the profile generation and then to rebuild
>> it, along these lines:
>>
>
>> DRV=$(guix gc --derivers $(readlink -f ~/.guix-profile))
>> time guix build --check $DRV
>
> The above results in
>
> real 1m28,841s
> user 0m2,169s
> sys 0m0,450s

Thanks.

It means that profile generation itself (and not just hooks) is slow.

The place to look at is (guix build union). Unfortunately, I suspect
there’s little room for optimization at this stage (see commit
12129998689648923b58c426362a1bc875da75f9 from… 2014).

Fundamentally, ‘union-build’ traverses every input directory, which is
expensive with low-end hard disks. It would still be worth
investigating (for example by strace’ing the ‘union-build’ process) in
case we missed optimizationm opportunities, though.

Thanks,
Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 44053@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 44053
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch