Hi! ludo@gnu.org (Ludovic Courtès) skribis: > ‘LocalStore::removeUnusedLinks’ traverses all the entries in > /gnu/store/.links and calls lstat(2) on each one of them and checks > ‘st_nlink’ to determine whether they can be deleted. > > There are two problems: lstat(2) can be slow on spinning disks as found > on hydra.gnu.org, and the algorithm is proportional in the number of > entries in /gnu/store/.links, which is a lot on hydra.gnu.org. Taking a step back, we could perhaps mitigate this with heuristics to reduce the number of entries in .links: 1. Do not deduplicate files with a size lower than some threshold; 2. Delete links with st_nlink <= 3 (instead of <= 2); that would prevent *further* deduplication of those files, but they’d already have two instances sharing the same inode; 3. Stop deduplicating once the number of entries in .links has reached a certain threshold. For #1, a key insight is that about 30% of the files actually deduplicated (in my store, where /gnu/store/.links has 2.2M entries) are smaller than 1 KiB: