Poor profile generation performance on spinning disks

OpenSubmitted by Maxim Cournoyer.
Details
5 participants
  • Joshua Branson
  • Ludovic Courtès
  • Luis Felipe
  • Maxim Cournoyer
  • zimoun
Owner
unassigned
Severity
important
M
M
Maxim Cournoyer wrote on 18 Oct 05:35 +0200
Download
(name . bug-guix)(address . bug-guix@gnu.org)
87zh4k435a.fsf@gmail.com
Download
Hello!
I've noticed on multiple occasions that using Guix on traditionalspinning drives can be quite slow.
On my home machine, will is still relying on 2 x 1 TB spinning drives inRAID1, rebuilding my user profile, which contains 182 entries, takes onaverage about 20 minutes, even when there are no packages to be built:
Toggle snippet (22 lines)guix package -i perl --max-jobs=1 The following package will beupgraded: perl (dependencies or package changed)
The following derivation will be built: /gnu/store/lhywla1z2zcz16df4hbvvvngr9zmswr7-profile.drv
building CA certificate bundle...building fonts directory...generating GLib schema cache...creating GTK+ icon theme cache...building cache files for GTK+ input methods...building directory of Info manuals...building database for manual pages...building XDG desktop file cache...building XDG MIME database...building profile with 182 packages...
real 19m0.126suser 0m5.648ssys 0m0.333s
Most of the time remains spent after the message 'building profile with182 package...'. That part seems IO-bound, with the spinning disksgrinding heavily and the CPU mostly idling. The rest of the time (3minutes), was used by the profile hooks.
The same operation on a second, more modern machine equipped with M2SSDs does much better and takes about 1 minute to accomplish the same,so it seems the bad performance can be mostly attributed to the muchslower disk seek times of the spinning disks.
On the older machine, two profile hooks are also sticking out w.r.t. thetime they take (they take more than one minute opposed to a fewseconds):
Toggle snippet (26 lines)The following profile hook will be built: /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drvbuilding database for manual pages...Creating manual page database...[2139/2139] building list of man-db entries...175322 entries processed in 95.1 ssuccessfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drvsuccessfully built /gnu/store/08fanpydi7z4i3qnlqbr8iz23zdgsamw-manual-database.drv/gnu/store/wzp4mk2r7r4ysciw74gqbfkyai0zmrcc-manual-database
real 1m36.378suser 0m1.674ssys 0m0.108s
The following profile hook will be built: /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drvbuilding XDG MIME database...successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drvsuccessfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv/gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database
real 1m7.344suser 0m1.331ssys 0m0.053s
So we should profile what's going on while generating the profile (nopun intended) and try to improve this at first since this is where mostof the time is spent on spinning drives (17 minutes out of the 20 in theabove example).
After that we could look into the two above profile hooks.
Thanks,
Maxim
L
L
Luis Felipe wrote on 19 Oct 01:05 +0200
Download
(name . 44053@debbugs.gnu.org)(address . 44053@debbugs.gnu.org)
yuID-gVO5clS0T8k81R_AzM7g91lm_6kDlH9rJFE8OOv5XMxEsdFo6hCH_QGnJGMW4nyDfuduHS_cXzKOm0qov8BalGmi8NPGKwR_eInGFk=@protonmail.com
Download
I've never used SDDs, so I've been experiencing this since I installed Guix (~5 years ago). And yes, building the profile seems to be the more resource intensive operation. I usually can't do anything else while that happens because the computer becomes unresponsive.
Running the same command Maxim mentioned took the following time in my case:

$ time guix package -i perl --max-jobs=1Se instalará el siguiente paquete:   perl 5.30.2Se construirá la siguiente derivación:   /gnu/store/86g48nh8dhgdhzb8r9bxjk6mmszlssss-profile.drvconstruyendo empaquetado de certificados de CA...construyendo el directorio de tipografías...generando la caché de esquemas de GLib...creando la caché de temas de iconos de GTK+...construyendo los ficheros de caché para los métodos de entrada de GTK+...construyendo el directorio de manuales Info...construyendo la base de datos de páginas de manual...construyendo la caché de ficheros desktop XDG...construyendo la base de datos MIME XDG...construyendo perfil con 87 paquetes...

real    8m3,043suser    0m2,656ssys    0m0,239s

My computer:
Intel® Core™ i3-8100 CPU @ 3.60GHz × 4RAM: 4 GiB1 TB HDD
Z
Z
zimoun wrote on 19 Oct 10:18 +0200
Download
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 44053@debbugs.gnu.org)
CAJ3okZ1AMUteo3qpAHb0m6RS3ApawCrdfQwVNkDg6Xtc515LcA@mail.gmail.com
Download
Hi Maxim,
On Sun, 18 Oct 2020 at 05:37, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (4 lines)> On my home machine, will is still relying on 2 x 1 TB spinning drives in> RAID1, rebuilding my user profile, which contains 182 entries, takes on> average about 20 minutes, even when there are no packages to be built:
The database build time seems correlated to the number of packages inthe very profile. Well, I have not benchmarked to see if it islinear, quadratic; or worse or better.

Toggle quote (1 lines)> --8<---------------cut here---------------start------------->8---
[...]
Toggle quote (10 lines)> building XDG MIME database...> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv> /gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database>> real 1m7.344s> user 0m1.331s> sys 0m0.053s> --8<---------------cut here---------------end--------------->8---
Well, it is all clear to me, but I do not know if we can do betterbecause these 2 XDG updates seem relying on the external binaries"/bin/update-desktop-database" and "/bin/update-mime-database". Ihave not timed the function 'xdg-desktop-database' but all the timeshould be spent on these, I bet. :-)

All the best,simon
M
M
Maxim Cournoyer wrote on 19 Oct 20:18 +0200
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 44053@debbugs.gnu.org)
87d01e5bbe.fsf@gmail.com
Download
Hello Simon,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (12 lines)> Hi Maxim,>> On Sun, 18 Oct 2020 at 05:37, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:>>> On my home machine, will is still relying on 2 x 1 TB spinning drives in>> RAID1, rebuilding my user profile, which contains 182 entries, takes on>> average about 20 minutes, even when there are no packages to be built:>> The database build time seems correlated to the number of packages in> the very profile. Well, I have not benchmarked to see if it is> linear, quadratic; or worse or better.
It is. Small profiles operations are much faster than large profiles.Attached is a slightly trimmed version of the manifest I currently usefor my user profile, if you'd like to benchmark it on your system.
Download
(use-modules (gnu packages) (gnu packages emacs) (guix build-system emacs) (guix profiles)) (concatenate-manifests (list ;;; Emacs packages. (specifications->manifest '("emacs" "emacs-auctex" "emacs-bash-completion" "emacs-bbdb" "emacs-cmake-mode" "emacs-company" "emacs-company-quickhelp" "emacs-counsel" "emacs-csv-mode" "emacs-debbugs" "emacs-diff-hl" "emacs-el-mock" "emacs-elpy" "emacs-emms" "emacs-ggtags" "emacs-go-mode" "emacs-grep-a-lot" "emacs-guix" "emacs-htmlize" "emacs-ivy" "emacs-magit" "emacs-org" "emacs-org-reveal" "emacs-paredit" "emacs-pdf-tools" "emacs-qml-mode" "emacs-realgud" "emacs-string-inflection" "emacs-swiper" "emacs-w3m" "emacs-ws-butler" "emacs-yasnippet" "emacs-yasnippet-snippets")) ;; Other software. (specifications->manifest '("adb" "acpi" "alsa-utils" "anthy" "arc-icon-theme" "arc-theme" "aspell" "aspell-dict-en" "aspell-dict-fr" "autoconf" "automake" "autossh" "bash" "bc" "beep" "bind:utils" ;for 'dig' "bluez" "bridge-utils" "cheese" "compsize" "cqfd" "cryptsetup" "curl" "dbus" "dconf" "ddcutil" "diffoscope" "docker-cli" "dosfstools" "evince" "file" "font-adobe-source-han-sans" "font-dejavu" "font-google-roboto" "font-hack" "gcc-toolchain" "gdb" "geeqie" "ghostscript-with-x" "gimp" "git" "git:send-email" "glibc-locales" "global" "gnome-bluetooth" "gnome-boxes" "adwaita-icon-theme" "hicolor-icon-theme" "gnu-standards" "gnucash" "gnucash:doc" "gnupg" "graphviz" "gtk-engines" "guile" "guile-readline" "guile-sqlite3" "guile-ssh" "hackneyed-x11-cursors" "hicolor-icon-theme" "hunspell" "hunspell-dict-fr" "ibus" "ibus-anthy" "icecat" "imagemagick" "inetutils" "inkscape" "iotop" "jack" "jami" "keepassxc" "libjpeg" "libmtp" "libpcap" "libreoffice" "libssh" "libx11" "linphoneqt" "lm-sensors" "lsof" "ltrace" "lvm2" ;for dmsetup "make" "man-pages" "mesa-utils" "mpv" "mtr" "nmap" "openssh" "openvpn" "parted" "pavucontrol" "perl" "pinentry" "pkg-config" "poppler" "pulseaudio" "pv" "python" "python-wrapper" "qemu" "recutils" "rsync" "rtorrent" "screen" "setxkbmap" "shepherd" "sicp" "smartmontools" "spacefm" "stow" "strace" "sysstat" ;for iostat "tcpdump" "the-silver-searcher" ;ag "time" ;aliased to time+ "transmission" "transmission:gui" "tree" "unzip" "vinagre" "vorbis-tools" "weechat" "wget" "workrave" "wpa-supplicant" "xdpyinfo" "xdg-utils" "xev" "xmodmap" "xournal" "xrandr" "xrdb" "xsetroot" "yelp" "gxtuner" "shellcheck")) ;; Others. (specifications->manifest '("docker-compose" "emacs-adoc-mode" "emacs-clang-format" "emacs-clang-rename" "emacs-feature-mode" "picocom" "python-git-review" "sshpass" "ungoogled-chromium" "ddrescue"))))
Download
I'm betting that the part to optimize is:
Toggle snippet (6 lines) ;; Make the symlinks. (union-build output inputs #:symlink symlink #:log-port (%make-void-port "w"))
from the 'build-profile' procedure in (guix build profiles).
Toggle quote (19 lines)>>> --8<---------------cut here---------------start------------->8---> [...]>> building XDG MIME database...>> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv>> successfully built /gnu/store/cir84qj587i6is4akgqand7ahg9bj938-xdg-mime-database.drv>> /gnu/store/j0bznlj2ibnhirijhnwpkkxzz4qfk8wb-xdg-mime-database>>>> real 1m7.344s>> user 0m1.331s>> sys 0m0.053s>> --8<---------------cut here---------------end--------------->8--->> Well, it is all clear to me, but I do not know if we can do better> because these 2 XDG updates seem relying on the external binaries> "/bin/update-desktop-database" and "/bin/update-mime-database". I> have not timed the function 'xdg-desktop-database' but all the time> should be spent on these, I bet. :-)
They do, and this is indeed what takes time. But, this doesn't stopmotivated wizards from implementing our own database generator in Schemethat would do the same. Ludovic had done just that for the man-dbdatabase generator. The custom code in Guix can do it much faster thatit initially could back when it was relying on man-db to do that (that'salso probably the reason why we have issues such ashttps://issues.guix.gnu.org/38838;perhaps the generated database isslightly different -- but that's another topic :-)).
Thanks,
Maxim
J
J
Joshua Branson wrote on 28 Oct 00:14 +0100
Download
Re: ‘xdg-mime-database’ profile hook is slow
(name . Ludovic Courtès)(address . ludo@gnu.org)
87ft5zqn2a.fsf@dismail.de
Download
I am just posting my performance numbers. Hopefully you all find it helpful.I am running Guix System on a Lenovo Thinkpad T400 with 8GB on a hard drive.
#+BEGIN_SRC sh :results output :exports bothguix describe#+END_SRC
#+RESULTS:#+begin_exampleGeneration 49 Oct 20 2020 01:15:33 (current) my-personal-packages 547d900 repository URL: https://notabug.org/jbranso/guix-packages.git branch: master commit: 547d900c25d2bb9490946162e0fcb315d6bace68 guix 93c2518 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 93c251818dcd2bf5f60f42e595504293ee83371a
GUIX_PACKAGE_PATH="/home/joshua/prog/gnu/guix/guix-packages/"#+end_example
The "XDG MIME database" takes a while.
#+BEGIN_SRC sh :results output :exports bothtime guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)#+END_SRC
#+RESULTS:: The following profile hook will be built:: /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv: building XDG MIME database...: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv: /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database:: real 0m43.716s: user 0m3.626s: sys 0m0.258s
Cold cache:
#+BEGIN_SRC sh :results output :exports bothdrv="$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)"sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
time guix build --check "$drv"#+END_SRC
#+RESULTS:: The following profile hook will be built:: /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv: building XDG MIME database...: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv: successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv: /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database:: real 1m29.606s: user 0m1.401s: sys 0m0.238s
I have 64 packages in my profile.
#+BEGIN_SRC sh :results output :exports bothguix package -I | wc --lines#+END_SRC
#+RESULTS:: 64
If you want to see all packages:
#+BEGIN_SRC sh :results output :exports bothguix package -I#+END_SRC
#+RESULTS:#+begin_exampleguile-colorized 0.1 out /gnu/store/5h0jfssgyx49cq82wrmw57an7ngb7kkz-guile-colorized-0.1font-hack 3.003 out /gnu/store/52r8anazd4rnkq9m3vxk700jga5h0i74-font-hack-3.003font-dejavu 2.37 out /gnu/store/7y3lvk3xf4im8n44337mc6y0ccysvfia-font-dejavu-2.37font-adobe-source-code-pro 2.030R-ro-1.050R-it out /gnu/store/l5m74158njcan3p784gqhkir6zyk0bcp-font-adobe-source-code-pro-2.030R-ro-1.050R-itsicp 20170703-1.225c172 out /gnu/store/x2safq6fvcbhp2rljny89qiq74y1xd13-sicp-20170703-1.225c172unzip 6.0 out /gnu/store/1zx8knb7h79q6cr4kwhbz6z6v609309w-unzip-6.0font-google-noto 20171025 out /gnu/store/g2szydnbvs7qqy2nf7qylba0rapajmd8-font-google-noto-20171025font-awesome 4.7.0 out /gnu/store/gknn01i1nr80apmcx42w36cqcz3zzri2-font-awesome-4.7.0texinfo 6.7 out /gnu/store/w8k9hcigvhzrlrblv8lgqj77sm3833rs-texinfo-6.7gcc-toolchain 10.2.0 out /gnu/store/pknm43xsza6nlc7bn27djip8fis92akd-gcc-toolchain-10.2.0recutils 1.8 out /gnu/store/qlx179h9mfdv4zywx2ias2jmkygvs834-recutils-1.8guile-readline 3.0.2 out /gnu/store/va6l1ivclww22fi38w5h99pb4ndn99hg-guile-readline-3.0.2gtypist 2.9.5 out /gnu/store/03wd707lyz49f2r14cm9ybyzh7vfvi0j-gtypist-2.9.5scsh 0.0.0-1.1144324 out /gnu/store/57frd08q6gyd7ysq3aqwlxxmify3w53h-scsh-0.0.0-1.1144324make 4.3 out /gnu/store/4k33n2nhsnnaxk2ip75gj7xiqdjns5hq-make-4.3isync 1.3.3 out /gnu/store/745qmx9mmnwq0j3zrd50qcdd2bgv5clz-isync-1.3.3jmacs 27.1 out /gnu/store/0hqdddkvhi2acxjdssdi5as3wj6cgvnz-jmacs-27.1glibc-locales 2.31 out /gnu/store/x9c4dk4zc9mi5j9hm3lq1q539kkdhkln-glibc-locales-2.31guile 3.0.4 out /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4guile-json 4.3.2 out /gnu/store/12y1s0p3585ydf336gq85kjh5217pnzp-guile-json-4.3.2guile-syntax-highlight 0.1 out /gnu/store/dwjqscckpypqfq38hc97x5ipc9fplpp9-guile-syntax-highlight-0.1haunt 0.2.4 out /gnu/store/ccvqa9a6yclgkpi9izhpgf1vgjaghqvr-haunt-0.2.4gnutls 3.6.14 out /gnu/store/0f6agk6xk8rcz99907d39nd1vzmc4abq-gnutls-3.6.14wget 1.20.3 out /gnu/store/d7b16jqrgb1jz65hl7vnz261dbxmjh3s-wget-1.20.3gnupg 2.2.23 out /gnu/store/89cjqnar4aacg58wl6cm1h41rym9mv4l-gnupg-2.2.23sassc 3.6.1 out /gnu/store/6pqn2i0r12ypss13mdvsypskkcpaagji-sassc-3.6.1qterminal 0.15.0 out /gnu/store/y7qrfa78y785xfajb3gvvhwcm2d02l81-qterminal-0.15.0dmenu 5.0 out /gnu/store/393frjc5xjaab6nvp3wxpajb1rc6gynr-dmenu-5.0openssh 8.4p1 out /gnu/store/k8lnv9cqk6m15k18hhzg7w696r1vxx1v-openssh-8.4p1grim 1.3.1 out /gnu/store/9v5i9q7kwcmq1q31qri0icjkhh3csh9r-grim-1.3.1evince 3.36.5 out /gnu/store/arwvpbrzsch4w6afkpqln6625dfl0bz8-evince-3.36.5pulseaudio 13.0 out /gnu/store/19wcjfwdr3hzq1a4wcpld1zdrjichck0-pulseaudio-13.0libreoffice 6.4.6.2 out /gnu/store/7rbk52nzqmakd5whds5a6f07xrd2k870-libreoffice-6.4.6.2termite 15 out /gnu/store/8lqfy0mxqnq3xg24gwsxvzpxnvv6firh-termite-15mpv 0.32.0 out /gnu/store/2i2gzn7fa8qq0f1jkqhiyl0xw63fjr4w-mpv-0.32.0pinentry 1.1.0 out /gnu/store/bdsk9yq48lx1f7y7xr9sv4yrmm4ync9b-pinentry-1.1.0glib 2.62.6 doc /gnu/store/ql2226knbh4q04bbdiqcppvs5rjyl3ga-glib-2.62.6-docpidgin 2.13.0 out /gnu/store/fd06njmc5maid7b474v224iazy7bc20h-pidgin-2.13.0gimp 2.10.20 out /gnu/store/m397hvwhsd0l6v5d0vqxxkm4iyp0p9w3-gimp-2.10.20sway 1.4 out /gnu/store/27ggd92in0hpgmkgk79n2iy4angjxwvl-sway-1.4wf-recorder 0.2.1 out /gnu/store/glc6y75l4czpfyrgbrbgxg38ig004v1p-wf-recorder-0.2.1wofi 1.1.2 out /gnu/store/qaylx0k6wg3awsamn2rqr41q9c8iwypg-wofi-1.1.2linphoneqt 4.1.1 out /gnu/store/wlcviacpagxqnw1737x131if0fj7kzjv-linphoneqt-4.1.1qbittorrent 4.2.5 out /gnu/store/29cjbdmz809pdvmylbx307l2p7zv0ma9-qbittorrent-4.2.5pavucontrol 4.0 out /gnu/store/0ilm4lncr6gpg4h3zdvaxg1pibkm6934-pavucontrol-4.0git 2.28.0 send-email /gnu/store/g5x559qk5yxs19jcxqplqv8b9y32vak2-git-2.28.0-send-emailflatpak 1.8.2 out /gnu/store/ywfsnccvlr0j6afb5vnl3z4ismvcfbz6-flatpak-1.8.2xdg-desktop-portal 1.7.2 out /gnu/store/jkjx5dd7s3dp58p2hwsm8n2rm15z90ys-xdg-desktop-portal-1.7.2xdg-desktop-portal-gtk 1.7.1 out /gnu/store/zhssisyp66vggjnqi5y8nb1xnk55vnw1-xdg-desktop-portal-gtk-1.7.1xeyes 1.1.2 out /gnu/store/4jx8i19xd94d421n4acy2c1p0nn9h6d0-xeyes-1.1.2emacs-polymode 0.2 out /gnu/store/gmm1y1xahg44yw9ghsf0kwd40w86q7mf-emacs-polymode-0.2notmuch 0.31 out /gnu/store/0js5dwj8nxj0x0hxfllgzz4zxlsmyrz4-notmuch-0.31msmtp 1.8.12 out /gnu/store/s1g4nskvnsx9ww0032xg23548h7mnwvq-msmtp-1.8.12youtube-viewer 3.7.9 out /gnu/store/jm7q53lpda3a4rs2z7xv7cgig7adpncb-youtube-viewer-3.7.9guile-curl 0.6 out /gnu/store/9nnclxjxakwx9yc381arqrk1nhld21b5-guile-curl-0.6qemu 5.1.0 out /gnu/store/qzi36f60d2zlrdj9mg6b5y4i00w1fzfq-qemu-5.1.0icecat 78.3.1-guix0-preview1 out /gnu/store/71fkykdajhkshsvzwrnqm4lsc6qgycg2-icecat-78.3.1-guix0-preview1git 2.28.0 out /gnu/store/j66k3x1wz0bc6nh09dwa6i4jwahqcabi-git-2.28.0guix 1.1.0-29.4e3ed9b out /gnu/store/irirj169rncank4y7i6n8qyiiqbs76qr-guix-1.1.0-29.4e3ed9bffmpeg 4.3.1 out /gnu/store/sg4yq9i0v1lphvs1vx33dw9mn801adrd-ffmpeg-4.3.1pavucontrol-qt 0.15.0 out /gnu/store/9d0zp4wps5kpbqsl4yvjb96spb5xwz9b-pavucontrol-qt-0.15.0clementine 1.3.1-2.4619a4c out /gnu/store/5392fysgmmi2jmn98maz9ihzn9ahzhw9-clementine-1.3.1-2.4619a4cmako 1.4 out /gnu/store/k83v9wz6szf6k2p2sz1amvwqm687j3h2-mako-1.4ungoogled-chromium 86.0.4240.75-0.c34a56d out /gnu/store/m0s9mrbirbssis5nn9agx8k4sl2ck0y9-ungoogled-chromium-86.0.4240.75-0.c34a56d#+end_example

Thanks,
--Joshua BransonSent from Emacs and Gnushttps://gnucode.mehttps://video.hardlimit.com/accounts/joshua_branson/video-channels"You can have whatever you want, as long as you help enough other people get what they want." - Zig Ziglar
L
L
Luis Felipe wrote on 28 Oct 22:48 +0100
Download
(name . Joshua Branson)(address . jbranso@dismail.de)
teuOd2Q7Dbrx6oqUwBMU0LqyBVxb35-URBXKJ0OS0xjTrTxUYpoeKt5hTCxbfNWxDVq94Ez3Z2X0K-hPxzHJJczuP8PT8zL-SvDNYu5Bpu0=@protonmail.com
Download
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐On Tuesday, October 27, 2020 11:14 PM, Joshua Branson <jbranso@dismail.de> wrote:
[...]
Toggle quote (6 lines)>> #+BEGIN_SRC sh :results output :exports both> time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)> #+END_SRC

I haven't run this command because I don't know what it does (how will it affect my profile?), but if you think the results from this command would be more useful than what I posted to issue #44053, please let me know and I'll run it.
L
L
Ludovic Courtès wrote on 1 Nov 11:28 +0100
Download
control message for bug #44053
(address . control@debbugs.gnu.org)
87y2jlqsl6.fsf@gnu.org
Download
severity 44053 importantquit
L
L
Ludovic Courtès wrote on 2 Nov 12:12 +0100
Download
Re: bug#44053: ‘xdg-mime-database’ profile hook is slow
(name . zimoun)(address . zimon.toutoune@gmail.com)
87sg9sm2qf.fsf@gnu.org
Download
Hi,
Joshua Branson <jbranso@dismail.de> skribis:
Toggle quote (19 lines)> The "XDG MIME database" takes a while.>> #+BEGIN_SRC sh :results output :exports both> time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)> #+END_SRC>>> #+RESULTS:> : The following profile hook will be built:> : /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv> : building XDG MIME database...> : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv> : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv> : /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database> :> : real 0m43.716s> : user 0m3.626s> : sys 0m0.258s
I found that the MIME database is computed only over the subset of thepackages in your profile that provide ‘share/mime/packages’, plus‘shared-mime-info’.
In my profile, only LibreOffice provides that directory, so the union ismade over these two directories and that’s what ‘update-mime-database’works on.
Apart from LibreOffice, I found that ‘share/mime/packages’ is providedby at least: hugin, gcr, fontforge. Most GUI packages don’t have it.So in practice, we’re often rebuilding the exact same database.
Here’s the time taken by ‘update-mime-database’ alone:
Toggle snippet (44 lines)ludo@ribbon ~$ mkdir -p /tmp/mime/share/mime/packagesludo@ribbon ~$ cd /tmp/mime/share/mime/packagesludo@ribbon /tmp/mime/share/mime/packages$ for i in $(guix build libreoffice ^Cludo@ribbon /tmp/mime/share/mime/packages$ for i in ~/.guix-profile/share/mime/packages/* ; do ln -s $i ; doneludo@ribbon /tmp/mime/share/mime/packages$ ls -ltotalo 8lrwxrwxrwx 1 ludo users 64 Nov 2 11:46 freedesktop.org.xml -> /home/ludo/.guix-profile/share/mime/packages/freedesktop.org.xmllrwxrwxrwx 1 ludo users 60 Nov 2 11:46 libreoffice.xml -> /home/ludo/.guix-profile/share/mime/packages/libreoffice.xmlludo@ribbon /tmp/mime/share/mime/packages$ cd /tmp/mime/ludo@ribbon /tmp/mime$ time update-mime-database -V /tmp/mime/share/mimeUpdating MIME database in /tmp/mime/share/mime...
Parsing source file /tmp/mime/share/mime/packages/freedesktop.org.xml...Parsing source file /tmp/mime/share/mime/packages/libreoffice.xml...Wrote 1124 strings at 2c - 6310Wrote aliases at 6310 - 6be4Wrote parents at 6be4 - 8468Wrote literal globs at 8468 - 855cWrote suffix globs at 855c - 13514Wrote full globs at 13514 - 13554Wrote magic at 13554 - 2065cWrote namespace list at 2065c - 20798Wrote icons list at 20798 - 2079cWrote generic icons list at 2079c - 213b8Wrote types list at 213b8 - 22020
Note that '/tmp/mime/share' is not in the search pathset by the XDG_DATA_HOME and XDG_DATA_DIRSenvironment variables, so applications may notbe able to find it until you set them. Thedirectories currently searched are:
- /home/ludo/.local/share- /home/ludo/.guix-profile/share- /run/current-system/profile/share- /home/ludo/.guix-profile/share- /run/current-system/profile/share

real 0m2.166suser 0m0.278ssys 0m0.056s
To be compared with:
Toggle snippet (13 lines)$ drv="$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)"$ time guix build --check $drvThe following profile hook will be built: /gnu/store/jrmwsxpz3wmq37zx29lvb0r9nvcmdviz-xdg-mime-database.drvbuilding XDG MIME database...successfully built /gnu/store/jrmwsxpz3wmq37zx29lvb0r9nvcmdviz-xdg-mime-database.drv/gnu/store/npvj2sr9kxx48znh7zc8zmzwzs6brc90-xdg-mime-database
real 0m3.670suser 0m1.428ssys 0m0.029s
Thus, the build time itself is entirely taken by ‘update-mime-database’(the 1.4s above is “overhead” in ‘guix build’ it would seem).
Ludo’.
L
L
Ludovic Courtès wrote on 2 Nov 13:53 +0100
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)
87ft5rncn5.fsf@gnu.org
Download
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (4 lines)> Apart from LibreOffice, I found that ‘share/mime/packages’ is provided> by at least: hugin, gcr, fontforge. Most GUI packages don’t have it.> So in practice, we’re often rebuilding the exact same database.
On closer inspection, the time-consuming bit is processing‘share/mime/packages/freedesktop.org.xml’ (from ‘shared-mime-info’),which is quite large and leads to the creation of hundreds of file. Weend up re-processing it every time. This is particularly wastefulbecause the ‘shared-mime-info’ package already contains the result ofapplying ‘update-mime-database’ to itself.
Unfortunately, AIUI,
update-mime-database(X ∪ Y) ≠ update-mime-database(X) ∪ update-mime-database(Y)
(For example, the files‘share/mime/{globs,magic,XMLnamespaces,subclasses,aliases,types,generic-icons,icons,treemagic,mime.cache}’concatenate info from X and Y.)
So it would seem we cannot simply used the pre-built database from‘shared-mime-info’ and merge it with that of the other packages, atleast not without changing ‘update-mime-database’ or re-implementingparts of it on our side.
Ludo’.
L
L
Ludovic Courtès wrote on 2 Nov 15:04 +0100
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)
87blgflush.fsf@gnu.org
Download
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (13 lines)> Ludovic Courtès <ludo@gnu.org> skribis:>>> Apart from LibreOffice, I found that ‘share/mime/packages’ is provided>> by at least: hugin, gcr, fontforge. Most GUI packages don’t have it.>> So in practice, we’re often rebuilding the exact same database.>> On closer inspection, the time-consuming bit is processing> ‘share/mime/packages/freedesktop.org.xml’ (from ‘shared-mime-info’),> which is quite large and leads to the creation of hundreds of file. We> end up re-processing it every time. This is particularly wasteful> because the ‘shared-mime-info’ package already contains the result of> applying ‘update-mime-database’ to itself.
Based on these observations, I added a fast path to the‘xdg-mime-database’ hook:
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54
It’s down to 0s in the common case where the profile doesn’t contain anypackages providing ‘share/mime/packages’.
Ludo’.
Z
Z
zimoun wrote on 4 Nov 00:32 +0100
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)
86blgeggo8.fsf@gmail.com
Download
Hi,
Toggle quote (2 lines)> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54
Clever workaround! What are now the performances on previous examples(same profiles and same packages)?

Toggle quote (4 lines)> Unfortunately, AIUI,>> update-mime-database(X ∪ Y) ≠ update-mime-database(X) ∪ update-mime-database(Y)
Hum? Interesting… well weird! If it is not a “morphism” then it is notreally an updater. Anyway.
I am curious about the comparison:
time(update-mime-database(X ∪ Y))vs time(update-mime-database(X)) + time(update-mime-database(Y))vs time(update-mime-database(X ∩ Y))

Toggle quote (5 lines)> So it would seem we cannot simply used the pre-built database from> ‘shared-mime-info’ and merge it with that of the other packages, at> least not without changing ‘update-mime-database’ or re-implementing> parts of it on our side.
’shared-mime-info’ is simply a package, right? So what does it means:«Find a way to avoid reprocessing 'shared-mime-info'» in:
Toggle snippet (8 lines) ;; the database. TODO: Find a way to avoid reprocessing ;; 'shared-mime-info', which is the most expensive one.[...] (invoke #+(file-append shared-mime-info "/bin/update-mime-database") destdir)))))))
? Well, I miss why and how ’shared-mime-info’ could be the culprit hereor what could be improved with “shared-mime-info”? Maybe replace the“lazy reference” by a strict one if it makes sense.

All the best,simon
L
L
Luis Felipe wrote on 5 Nov 15:47 +0100
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)
wfCkfOh1kZkBBqR2DYrffbz9xGwrXpB5lNfNNbLywaXO2i620ONaFST7pkyF_vskC20eUnLuV36YyO6t8v6__7oyVbOzaCoba1muCHDP0S4=@protonmail.com
Download
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐On Tuesday, November 3, 2020 11:32 PM, zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (7 lines)> Hi,>> > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54>> Clever workaround! What are now the performances on previous examples> (same profiles and same packages)?
In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).

$ time guix package -i perl --max-jobs=1Se instalará el siguiente paquete: perl 5.30.2
Se construirá la siguiente derivación: /gnu/store/48imlmfi2hjmp4ksyidvy99c31ahj44z-profile.drv
Se descargarán 0,6 MBconstruyendo empaquetado de certificados de CA...descargando de https://ci.guix.gnu.org/nar/lzip/j3f5hx6yvhlj3sapiipigkwg05b49nwn-freetype-2.10.4... freetype-2.10.4 555KiB 707KiB/s 00:01 [##################] 100.0%
injertando 5 paquetes en /gnu/store/s4s7xzswar962p9w1nvf0hfcgmnxyfsh-at-spi2-core-2.34.0.drv ...injertando 1 paquete en /gnu/store/80n340f7bnba0ivp00z10a32rz8g8wsx-fontconfig-2.13.1.drv ...injertando 4 paquetes en /gnu/store/hm3a895l251mckzb9bhiglcwfb6xli5n-at-spi2-atk-2.34.1.drv ...injertando 6 paquetes en /gnu/store/9vwl8s2h7rbfhvimsarzc1q4gnclwpk2-cairo-1.16.0.drv ...injertando 6 paquetes en /gnu/store/k6gajkzfq4drpcrimbwxk1ls0h7z1nzw-cairo-1.16.0.drv ...injertando 4 paquetes en /gnu/store/klq5sp427s80d0mhz3b5x48xpmhf3ila-gdk-pixbuf-2.40.0.drv ...injertando 5 paquetes en /gnu/store/ijkpcwanfpj02i0rxk4ys0kfdpj6sz69-ghostscript-with-cups-9.52.drv ...generando la caché de esquemas de GLib...injertando 2 paquetes en /gnu/store/kg8ca3wz01ibhbm5wk707r44jjnrxdg1-harfbuzz-2.6.4.drv ...construyendo el directorio de manuales Info...injertando 4 paquetes en /gnu/store/97bgi57qs077pcqggvr7di0vhchlshk2-libxft-2.3.3.drv ...construyendo la base de datos de páginas de manual...injertando 1 paquete en /gnu/store/gb85hxzczf4m0crzwqpw2imxpr1d67nx-mkfontscale-1.2.1.drv ...injertando 9 paquetes en /gnu/store/ddiqkw9r0p1lhvqmdlcyrzf8v900pm4r-pango-1.42.4.drv ...injertando 1 paquete en /gnu/store/xd0ca96gvh3vn9bp3bcmczczcxm0bynd-mkfontdir-1.0.7.drv ...injertando 11 paquetes en /gnu/store/s53s5qx304b90l0dp734lqi42whwd0vd-librsvg-2.40.21.drv ...construyendo el directorio de tipografías...injertando 5 paquetes en /gnu/store/qg628jlnjmiyv964c0i4s7bbkwy32c6c-gdk-pixbuf+svg-2.40.0.drv ...injertando 8 paquetes en /gnu/store/1mj8cjf43nqd9r5151lqrf1sdd1sjg16-pango-1.44.7.drv ...injertando 8 paquetes en /gnu/store/d42vnhn9grcsyiibdgkiryqb7rxhb89j-poppler-0.86.1.drv ...construyendo la caché de ficheros desktop XDG...injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ...construyendo la base de datos MIME XDG...injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ...injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ...injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ...creando la caché de temas de iconos de GTK+...construyendo los ficheros de caché para los métodos de entrada de GTK+...construyendo perfil con 86 paquetes...

real 8m38,121suser 0m2,742ssys 0m0,338s
Z
Z
zimoun wrote on 5 Nov 17:10 +0100
Download
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
86imajg4xs.fsf@gmail.com
Download
Hi,
On Thu, 05 Nov 2020 at 14:47, Luis Felipe <luis.felipe.la@protonmail.com> wrote:
Toggle quote (8 lines)>> Clever workaround! What are now the performances on previous examples>> (same profiles and same packages)?>> In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).>>> $ time guix package -i perl --max-jobs=1
[...]
Toggle quote (4 lines)> real 8m38,121s> user 0m2,742s> sys 0m0,338s
Thanks. But it is hard to conclude because the previous timing ismissing. The right experiment should be:
Toggle snippet (13 lines)old=c73496f433044a76003b33c3855bb35ecd0df87fnew=5e7cf66fb35780f930ad0bc5fe21ac330df4411dguix time-machine --commit=$old -- helpguix time-machine --commit=$new -- help
drv=$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)
guix package -I |wc -ltime guix time-machine --commit=$old -- build --check $drvtime guix time-machine --commit=$new -- build --check $drv

And personally, I get:
18
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...The following profile hook will be built: /gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drvbuilding XDG MIME database...successfully built /gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drv/gnu/store/hrcj1h0hl9f6hm1akh682zj1mhpwbx4c-xdg-mime-database
real 0m8.063suser 0m6.226ssys 0m0.282s
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...The following profile hook will be built: /gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drvbuilding XDG MIME database...successfully built /gnu/store/2jlwri28aycsayql85rfphrks721ny1k-xdg-mime-database.drv/gnu/store/hrcj1h0hl9f6hm1akh682zj1mhpwbx4c-xdg-mime-database
real 0m8.016suser 0m6.223ssys 0m0.243s
Well, I have only 18 packages in my ~/.guix-profile. That’s why thedifference is not visible and the “time-machine” is totally dominatinghere.

All the best,simon
L
L
Luis Felipe wrote on 5 Nov 18:59 +0100
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)
mOI7DET2ltI7DDsuW_psxIqhFvJxKjWOWmGpwEJKTL5rN49dnI-dvMMxquvNl33w9N9gxqJT8KA8hanP03RENzWZTwUY4Y7cNvctDfucOik=@protonmail.com
Download
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐On Thursday, November 5, 2020 4:10 PM, zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (19 lines)> Hi,>> On Thu, 05 Nov 2020 at 14:47, Luis Felipe luis.felipe.la@protonmail.com wrote:>> > > Clever workaround! What are now the performances on previous examples> > > (same profiles and same packages)?> >> > In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).> > $ time guix package -i perl --max-jobs=1>> [...]>> > real 8m38,121s> > user 0m2,742s> > sys 0m0,338s>> Thanks. But it is hard to conclude because the previous timing is> missing.
I don't understand, I posted my original timing in https://issues.guix.gnu.org/44053#1.


Toggle quote (16 lines)> The right experiment should be:>> --8<---------------cut here---------------start------------->8---> old=c73496f433044a76003b33c3855bb35ecd0df87f> new=5e7cf66fb35780f930ad0bc5fe21ac330df4411d> guix time-machine --commit=$old -- help> guix time-machine --commit=$new -- help>> drv=$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)>> guix package -I |wc -l> time guix time-machine --commit=$old -- build --check $drv> time guix time-machine --commit=$new -- build --check $drv> --8<---------------cut here---------------end--------------->8---

I get this:
yo@gnu: ~❯ guix package -I |wc -l85
yo@gnu: ~❯ time guix time-machine --commit=$old -- build --check $drvActualizando el canal 'guix' del repositorio Git en 'https://git.savannah.gnu.org/git/guix.git'...Actualizando el canal 'luflac-x' del repositorio Git en 'git@gitlab.com:luis-felipe/guix-channel-x.git'...Se construirá la siguiente extensión del perfil: /gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drvconstruyendo la base de datos MIME XDG.../gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drv construido satisfactoriamente/gnu/store/i4qslq2517pwkwb5dyivrg8idsjd8n11-xdg-mime-database
real 0m56,245suser 0m4,324ssys 0m0,218s

yo@gnu: ~❯ time guix time-machine --commit=$new -- build --check $drvActualizando el canal 'guix' del repositorio Git en 'https://git.savannah.gnu.org/git/guix.git'...Actualizando el canal 'luflac-x' del repositorio Git en 'git@gitlab.com:luis-felipe/guix-channel-x.git'...Se construirá la siguiente extensión del perfil: /gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drvconstruyendo la base de datos MIME XDG.../gnu/store/8hswzn212w7wqlr7cgsvq5v7zjh62chc-xdg-mime-database.drv construido satisfactoriamente/gnu/store/i4qslq2517pwkwb5dyivrg8idsjd8n11-xdg-mime-database
real 0m43,272suser 0m4,214ssys 0m0,200s
L
L
Ludovic Courtès wrote on 6 Nov 10:10 +0100
Download
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
87blga3l7e.fsf@gnu.org
Download
Hi,
Luis Felipe <luis.felipe.la@protonmail.com> skribis:
Toggle quote (12 lines)> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐> On Tuesday, November 3, 2020 11:32 PM, zimoun <zimon.toutoune@gmail.com> wrote:>>> Hi,>>>> > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54>>>> Clever workaround! What are now the performances on previous examples>> (same profiles and same packages)?>> In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).
Please note that the change above addresses only one specific source ofslowness, the ‘xdg-mime-database’ hook, and only in specific cases.
It’s good to look at the overall timing of ‘guix install’, becausethat’s what matters in the end, but as we work on optimizing it, we haveto look at specific aspects of it.
Toggle quote (2 lines)> $ time guix package -i perl --max-jobs=1
[...]
Toggle quote (14 lines)> injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ...> construyendo la base de datos MIME XDG...> injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ...> injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ...> injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ...> creando la caché de temas de iconos de GTK+...> construyendo los ficheros de caché para los métodos de entrada de GTK+...> construyendo perfil con 86 paquetes...>>> real 8m38,121s> user 0m2,742s> sys 0m0,338s
Here it’s likely that grafting is what’s taking the most time on aspinning disk.
We should hack (guix status) so it can optionally prefix each event witha timestamp.
As far as ‘xdg-mime-database’ is concerned, it should be down to 0s,unless your profile contains one of the packages I cited (libreoffice,gcr, hugin, etc.).
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 6 Nov 10:12 +0100
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)
875z6i3l38.fsf@gnu.org
Download
Hello,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (15 lines)>> So it would seem we cannot simply used the pre-built database from>> ‘shared-mime-info’ and merge it with that of the other packages, at>> least not without changing ‘update-mime-database’ or re-implementing>> parts of it on our side.>> ’shared-mime-info’ is simply a package, right? So what does it means:> «Find a way to avoid reprocessing 'shared-mime-info'» in:>> ;; the database. TODO: Find a way to avoid reprocessing> ;; 'shared-mime-info', which is the most expensive one.> [...]> (invoke #+(file-append shared-mime-info> "/bin/update-mime-database")> destdir)))))))
‘shared-mime-info’ contains ‘share/mime/freedesktop.org.xml’, which isby far where ‘update-mime-database’ spends most of its time.
But it’s wasteful because ‘shared-mime-info’ already contains the resultof running ‘update-mime-database’ on itself.
HTH!
Ludo’.
Z
Z
zimoun wrote on 6 Nov 13:55 +0100
Download
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
864km2fxux.fsf@gmail.com
Download
Hi,
On Thu, 05 Nov 2020 at 17:59, Luis Felipe <luis.felipe.la@protonmail.com> wrote:
Therefore, it is nice improvement! :-)
Toggle quote (8 lines)> real 0m56,245s> user 0m4,324s> sys 0m0,218s
> real 0m43,272s> user 0m4,214s> sys 0m0,200s
Because, I bet that most of the time is spent in the “time-machine”part. I mean, compare with:
time guix time-machine --commit=$new -- help

All the best,simon
L
L
Luis Felipe wrote on 6 Nov 14:57 +0100
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)
bMZ76QeMfjltWvLabLrXiBZyPPfTxrDHumnzqmIyDYRiU4BpAXwS8Z0zo4fnvHlbFLP23GpP27GJBS6Q6YZYDJ7sXXRzMOFJGtvujtprDNo=@protonmail.com
Download
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐On Friday, November 6, 2020 9:10 AM, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (26 lines)> Hi,>> Luis Felipe luis.felipe.la@protonmail.com skribis:>> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐> > On Tuesday, November 3, 2020 11:32 PM, zimoun zimon.toutoune@gmail.com wrote:> >> > > Hi,> > >> > > > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54> > >> > > Clever workaround! What are now the performances on previous examples> > > (same profiles and same packages)?> >> > In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d).>> Please note that the change above addresses only one specific source of> slowness, the ‘xdg-mime-database’ hook, and only in specific cases.>> It’s good to look at the overall timing of ‘guix install’, because> that’s what matters in the end, but as we work on optimizing it, we have> to look at specific aspects of it.>> > $ time guix package -i perl --max-jobs=1

Yeah, sorry I was more focused on the general issue (#44053), but I understand.

Toggle quote (17 lines)> [...]>> > injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ...> > construyendo la base de datos MIME XDG...> > injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ...> > injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ...> > injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ...> > creando la caché de temas de iconos de GTK+...> > construyendo los ficheros de caché para los métodos de entrada de GTK+...> > construyendo perfil con 86 paquetes...> > real 8m38,121s> > user 0m2,742s> > sys 0m0,338s>> Here it’s likely that grafting is what’s taking the most time on a> spinning disk.
It does take some time, but since I can see the output change from grafting to grafting, I at least can tell guix is doing something, so I just let it be.
Compared to grafting, the last step "construyendo perfil con X paquetes..." ("building profile with X packages..."), just stays there without change for several minutes, so it actually seems slower to me. Initially, I thought that guix had frozen.
Also, even though, the "building profile" step has a throbber (| / - \) to indicate that something is being done, it frequently stops in one of the frames of the sequence and stays there until the end.

Toggle quote (7 lines)> We should hack (guix status) so it can optionally prefix each event with> a timestamp.>> As far as ‘xdg-mime-database’ is concerned, it should be down to 0s,> unless your profile contains one of the packages I cited (libreoffice,> gcr, hugin, etc.).
Yes, I have Libreoffice installed.
L
L
Ludovic Courtès wrote on 8 Nov 17:31 +0100
Download
(name . Luis Felipe)(address . luis.felipe.la@protonmail.com)
87pn4nvmh0.fsf@gnu.org
Download
Hi,
Luis Felipe <luis.felipe.la@protonmail.com> skribis:
Toggle quote (4 lines)> Compared to grafting, the last step "construyendo perfil con X paquetes..." ("building profile with X packages..."), just stays there without change for several minutes, so it actually seems slower to me. Initially, I thought that guix had frozen.>> Also, even though, the "building profile" step has a throbber (| / - \) to indicate that something is being done, it frequently stops in one of the frames of the sequence and stays there until the end.
Interesting, so we should profile that step and see what can be done. Isuspect it’s I/O-bound, but maybe we can at least improve feedback.
Thanks,Ludo’.
?
Your may also send email to 44053@debbugs.gnu.org to comment.