Helm 3.6.5 on Emacs 27.1 leaks memory

  • Done
  • quality assurance status badge
Details
7 participants
  • Eli Zaretskii via web
  • Ludovic Courtès
  • Pierre Neidhardt
  • Mark H Weaver
  • Michael Rohleder
  • Ricardo Wurmus
  • zimoun
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
serious
L
L
Ludovic Courtès wrote on 14 Sep 2020 21:38
Emacs 27.1 memory consumption grows indefinitely
(address . bug-guix@gnu.org)
874ko0b12w.fsf@gnu.org
Hello,

Since I upgraded to Emacs 27.1, its memory consumption grows without
bounds, to the point that it gets OOM-killed after just a couple of
hours (meaning that it’s used most of the 16G of RAM of my laptop!).

Does that ring a bell to anyone? I have 50+ ^emacs- packages in my
profile; any clues on what could be the cause before diving further?

Toggle snippet (12 lines)
$ guix package -I ^emacs$
emacs 27.1 out /gnu/store/ipgb6s3phz3n740xlcdzaxnbgvncdh11-emacs-27.1
$ guix describe
Generacio 157 Sep 10 2020 08:49:21 (nuna)
guix 7090159
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 7090159c23d6345992ab976d71fefeb1583cfcdf
$ uname -rm
5.8.8-gnu x86_64

Ludo’.
M
M
Mark H Weaver wrote on 14 Sep 2020 22:40
87h7s0f5xg.fsf@netris.org
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (7 lines)
> Since I upgraded to Emacs 27.1, its memory consumption grows without
> bounds, to the point that it gets OOM-killed after just a couple of
> hours (meaning that it’s used most of the 16G of RAM of my laptop!).
>
> Does that ring a bell to anyone? I have 50+ ^emacs- packages in my
> profile; any clues on what could be the cause before diving further?

Here's another data point: although I haven't paid attention to the
memory consumption of Emacs 27.1, I've used it successfully for at least
10 hours at a time (probably more) on my X200 with only 4G of RAM. I
use 'emacs-no-x-toolkit' with only a few emacs packages:

emacs-no-x-toolkit
emacs-paredit
emacs-magit
emacs-geiser
emacs-org-caldav
emacs-nov-el
notmuch

I haven't actually used 'emacs-org-caldav' or 'emacs-nov-el' recently.
Paredit, Geiser, Magit, and Notmuch's Emacs mail client are the only
out-of-tree Emacs packages that I actively use.

Mark (boring Emacs user :)
R
R
Ricardo Wurmus wrote on 14 Sep 2020 23:56
(name . Ludovic Courtès)(address . ludo@gnu.org)
871rj4xbsu.fsf@elephly.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)
> Since I upgraded to Emacs 27.1, its memory consumption grows without
> bounds, to the point that it gets OOM-killed after just a couple of
> hours (meaning that it’s used most of the 16G of RAM of my laptop!).
[…]
Toggle quote (3 lines)
> $ guix package -I ^emacs$
> emacs 27.1 out /gnu/store/ipgb6s3phz3n740xlcdzaxnbgvncdh11-emacs-27.1

I have not observed this with /gnu/store/0v2r0d9gwbks7gmg5rygjb9mpb8i1bhl-emacs-27.1

(emacs-uptime) says "5 days, 11 hours, 38 minutes, 57 seconds"

--
Ricardo
Z
Z
zimoun wrote on 15 Sep 2020 00:11
(name . Ricardo Wurmus)(address . rekado@elephly.net)
CAJ3okZ0=FivgXo2gf5RceFGGfHMw5TizH3vc9Kzk51tdyGRYzw@mail.gmail.com
Toggle quote (2 lines)
> > emacs 27.1 out /gnu/store/ipgb6s3phz3n740xlcdzaxnbgvncdh11-emacs-27.1

With the exact store item, M-x emacs-uptime says "3 days, 3 hours, 4
minutes, 7 seconds".
My kernel is 4.19.0-10 from Debian.

I use these packages:
"diminish"
"yasnippet"
"smex"
"counsel"
"ivy"
"ivy-rich"
"magit"
"debbugs"
"ag"
"org"
"org-contrib"
"flycheck"
"auctex"
"auto-dictionary-mode"
"ox-pandoc"
"pandoc-mode"
"htmlize"
"org-re-reveal"
"typo"
"pdf-tools"
"paredit"
"geiser"
"guix"
;; "ess"
"julia-mode"
"pyvenv"
"haskell-mode"
"tuareg"
"rust-mode"
"lua-mode"
"markdown-mode"
"google-c-style"
"graphviz-dot-mode"
"skewer-mode"
"ws-butler"
"fill-column-indicator"
"page-break-lines"
"info-plus"
"keyfreq"
in addition to the ones in core, in case it helps.

Cheers,
simon
L
L
Ludovic Courtès wrote on 15 Sep 2020 16:19
(address . 43406@debbugs.gnu.org)
87k0wvf7g4.fsf@gnu.org
Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (16 lines)
> Since I upgraded to Emacs 27.1, its memory consumption grows without
> bounds, to the point that it gets OOM-killed after just a couple of
> hours (meaning that it’s used most of the 16G of RAM of my laptop!).
>
> Does that ring a bell to anyone? I have 50+ ^emacs- packages in my
> profile; any clues on what could be the cause before diving further?
>
> $ guix package -I ^emacs$
> emacs 27.1 out /gnu/store/ipgb6s3phz3n740xlcdzaxnbgvncdh11-emacs-27.1
> $ guix describe
> Generacio 157 Sep 10 2020 08:49:21 (nuna)
> guix 7090159
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 7090159c23d6345992ab976d71fefeb1583cfcdf

Emacs is about to crash :-), so let me share this finding from
M-x profiler-report (mem) on a sample of a few minutes:

Toggle snippet (27 lines)
- timer-event-handler 805,671,418 87%
- apply 804,987,874 87%
+ helm-ff--cache-mode-refresh 802,103,698 86%
+ emms-playing-time-display 1,349,461 0%
+ battery-update-handler 1,109,005 0%
+ #<compiled 0x1ff1153b5547> 221,094 0%
+ #<compiled 0x1ff11537c089> 49,632 0%
+ blink-cursor-start 30,368 0%
+ auto-revert-buffers 29,480 0%
+ highlight-symbol-temp-highlight 22,076 0%
+ mouse-avoidance-fancy 20,440 0%
+ display-time-event-handler 18,383 0%
+ show-paren-function 13,464 0%
+ erc-server-send-ping 7,500 0%
jit-lock-force-redisplay 4,136 0%
+ blink-cursor-timer-function 4,136 0%
+ nnimap-keepalive 2,400 0%
+ gnus-demon-run-callback 1,545 0%
+ isearch-lazy-highlight-start 1,056 0%
+ timer-activate 306,016 0%
+ timer-inc-time 191,264 0%
+ timer-activate-when-idle 92,928 0%
+ command-execute 100,936,674 10%
+ erc-server-filter-function 5,178,272 0%
+ redisplay_internal (C function) 4,988,306 0%

It’s Helm eating memory at each timer tick. That’s with:

Toggle snippet (11 lines)
emacs-helm-wikipedia 0.0.0-1.126f044 out /gnu/store/iy08rcx4hbravm9k3qlinc4wvklp3pqi-emacs-helm-wikipedia-0.0.0-1.126f044
emacs-helm-swoop 3.0.0 out /gnu/store/qnr6asnk0b360ilx8cnrp35kwrh4fsf7-emacs-helm-swoop-3.0.0
emacs-helm-shell-history 0.1-1.110d3c3 out /gnu/store/ljgfa72mw8jx6br9khl7idcmrw4i33hc-emacs-helm-shell-history-0.1-1.110d3c3
emacs-helm-make 0.1.0-1.feae8df out /gnu/store/lnizl47zxlj03waficpy7fvfrx1mdqn7-emacs-helm-make-0.1.0-1.feae8df
emacs-helm-ls-git 1.9.1-1.76654c7 out /gnu/store/897fkkh4bzpvbd06kf7vxr53hgkk0vfs-emacs-helm-ls-git-1.9.1-1.76654c7
emacs-helm-gtags 1.5.7 out /gnu/store/lhibi05hvvn3zglcx2ii44yw723pfyja-emacs-helm-gtags-1.5.7
emacs-helm 3.6.5 out /gnu/store/ka9lph0hpzaky0sa52zf09469apkhb68-emacs-helm-3.6.5
emacs-helm-system-packages 1.10.1-1.6572340 out /gnu/store/lvkan5dsbcdkvl3y1fjmsgb423nzmmqc-emacs-helm-system-packages-1.10.1-1.6572340
emacs-helm-emms 1.3-3.37e5aa0 out /gnu/store/g06lnxvyqnp9d2yqgfanp69350f7m2yp-emacs-helm-emms-1.3-3.37e5aa0

Cc’ing a couple of people who I think use Helm and hopefully have
valuable advice to share. :-)

Ludo’.
L
L
Ludovic Courtès wrote on 15 Sep 2020 16:21
(address . 43406@debbugs.gnu.org)
87ft7jf7du.fsf@gnu.org
More precisely:

Toggle snippet (10 lines)
- timer-event-handler 805,671,418 87%
- apply 804,987,874 87%
- helm-ff--cache-mode-refresh 802,103,698 86%
- helm-ff--cache-mode-refresh-1 801,321,290 86%
- helm-ff-refresh-cache 801,237,818 86%
- maphash 801,237,818 86%
+ #<compiled 0xec7961> 801,237,818 86%
helm-ff--cache-mode-max-idle-time 6,384 0%
+ run-with-idle-timer 651,464 0%
M
M
Michael Rohleder wrote on 15 Sep 2020 16:45
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 43406@debbugs.gnu.org)
87r1r3nlo4.fsf@rohleder.de
Hi Ludovic,

I also had trouble with that helm caching. I had the impression that it
was because of newer helm versions not emacs versions.

What helped me was disabling it with:
(setq helm-ff-keep-cached-candidates nil)

--
Alles fließt, nichts bleibt.
Heraklit
-----BEGIN PGP SIGNATURE-----

iQFFBAEBCAAvFiEEdV4t5dDVhcUueCgwfHr/vv7yyyUFAl9g04wRHG1pa2VAcm9o
bGVkZXIuZGUACgkQfHr/vv7yyyX43Qf+LXmdcgvDMb5I8XQAPD8fD5aen4XKuaQC
48O8CIlHnt+D0Xji1XJoXUbJ0p5JLksL3mdIJ8dKAS6Jv8XU32wBnlF+zpaThMEd
/ePKPrqHpYIAsgLl4hrTEjgaCbcKaM50+GreZkN2Nl1FMEUEK1gY2W0dTWyB9ahR
jIp+gu+hQfJwYGst3AKK2hnFhdlNlZy3QEXCpoaNUwxo9/7MTQAOq1A6ChWHQGLo
ndSzNsJmj9lpAdkirk+WOMXczkoqoatEeNRO15k5VGlbuLDS2v8KXRJqtntFvZvy
LgRuHwRKSDdMxR7vTAoujcExNboYVEzomMlJSE8duGFTy8/SpYuQxw==
=L+xO
-----END PGP SIGNATURE-----

P
P
Pierre Neidhardt wrote on 15 Sep 2020 16:47
(name . zimoun)(address . zimon.toutoune@gmail.com)
87bli75c6z.fsf@ambrevar.xyz
I believe believe helm-ff cache came with Helm 3.6.4.

I find it slow (and not so useful?) so I've disabled it with:

(setq helm-ff-keep-cached-candidates nil)

Hope that helps!

--
Pierre Neidhardt
-----BEGIN PGP SIGNATURE-----

iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl9g1AQSHG1haWxAYW1i
cmV2YXIueHl6AAoJEJvc9Jeku8x/NIgIAKHjkCRrdmsVy2W3Kg/Wy9qNEk25DVg2
0ayTpRI5feshPqDn9Px3R/BDBfqNEt4wHo5eYqfta39RzEKlRN18SE5QMHoMsji7
fXfcm8djWGRkUZaZm9H4qU+88VvDlc134ct3ukHim5VF5sfTpeDd6Bfl/Qp0EK+X
4YnA2FMHOqTXBnfwCTiPl/ev0ZasJZpEmo6SSY1MXtBoSfYz8MY+FsWp0rTsEtGu
57lns0PRrLVWTIRecuHFQGy7R7mSL/kVnWaHYpARK7+YWT0eBrMLQCCALs54L/Ob
sO6AsbDjim10LK5BK5Zfv6rh2epPW+Kvd2F6+iSuyOT2VRguvDf4QVY=
=5X1o
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 15 Sep 2020 17:05
control message for bug #43406
(address . control@debbugs.gnu.org)
875z8ft719.fsf@gnu.org
severity 43406 serious
quit
L
L
Ludovic Courtès wrote on 15 Sep 2020 18:48
Re: bug#43406: Emacs 27.1 memory consumption grows indefinitely
(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)
87lfhbrnol.fsf@gnu.org
Hi!

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

Toggle quote (6 lines)
> I believe believe helm-ff cache came with Helm 3.6.4.
>
> I find it slow (and not so useful?) so I've disabled it with:
>
> (setq helm-ff-keep-cached-candidates nil)

Michael Rohleder <mike@rohleder.de> skribis:

Toggle quote (3 lines)
> What helped me was disabling it with:
> (setq helm-ff-keep-cached-candidates nil)

I definitely helps, thanks!

I seem to have something else leaking still, but this time it’s
primarily Gnus and/or EIEIO:

Toggle snippet (39 lines)
- command-execute 1,117,368,135 88%
- call-interactively 1,117,276,295 88%
- funcall-interactively 1,116,838,121 88%
- gnus-group-save-newsrc 460,241,156 36%
- gnus-save-newsrc-file 460,241,156 36%
- gnus-run-hooks 429,283,350 34%
- apply 429,283,350 34%
- run-hooks 429,283,350 34%
- gnus-registry-save 429,283,350 34%
- eieio-persistent-save 427,938,700 33%
- apply 427,938,700 33%
+ #<compiled 0x6005bf1> 427,938,700 33%
+ gnus-registry--munge-group-names 1,340,064 0%
+ gnus-message 3,530 0%
+ gnus-gnus-to-quick-newsrc-format 20,126,054 1%
+ save-buffer 9,613,098 0%
+ gnus-agent-save-local 595,652 0%
+ gnus-gnus-to-newsrc-format 577,848 0%
+ gnus-group-set-mode-line 6,458 0%
+ gnus-message 3,736 0%
+ gnus-dribble-delete-file 3,352 0%
gnus-get-buffer-create 2,138 0%
+ gnus-prune-buffers 1,056 0%
- gnus-topic-read-group 208,616,775 16%
- gnus-group-read-group 208,616,775 16%
- gnus-summary-read-group 208,616,775 16%
- gnus-summary-read-group-1 208,616,775 16%
- gnus-run-hooks 163,821,469 13%
- apply 163,821,469 13%
- run-hooks 163,821,469 13%
- spam-find-spam 162,365,376 12%
- mapcar 162,328,432 12%
+ #<compiled 0x86338dd> 162,328,432 12%
+ gnus-parameter-spam-autodetect 35,888 0%
+ gnus-parameter-spam-autodetect-methods 1,056 0%
+ gnus-apply-kill-file 1,456,093 0%
+ gnus-select-newsgroup 31,456,212 2%

Any cache to turn off? :-)

Ludo’.
P
P
Pierre Neidhardt wrote on 15 Sep 2020 19:02
(name . Ludovic Courtès)(address . ludo@gnu.org)
87wo0v3rdi.fsf@ambrevar.xyz
I don't use Gnus, sorry.
Can you reproduce on a vanilla configuration (emacs -Q)?

--
Pierre Neidhardt
-----BEGIN PGP SIGNATURE-----

iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAl9g86kSHG1haWxAYW1i
cmV2YXIueHl6AAoJEJvc9Jeku8x/AisH/ighE33oi9fO9W244bWLK/LY5jOlKRzB
/yvLnQ0F7lM9Ns53iTwJJZw65vwTou5KiYpJDNS5v4zwlBJDhd5glcS4Qo4bAtqw
g+VlLPjrdqSdngZzdQBQYbMjTcdlUMMM8uD9dk/c3fcYHHgVMU5DQD0N0d9raRfL
SWg3uIoecOiS4qjld8x/b7xYkJdChFa0sBITWUdvnlnTzaerCAy/W/Zg7vCBzH0f
4EV6tkNRaqdxnPrWQuNr2dKuSAClPTg9+5YEgF8yk47VjnTerzQPnIT1L40PYy9j
HRyKnBS0KHOO3C26t5xKv9UvehAcapnL894knaEmWSbNIPXRWNuHqCA=
=8qIr
-----END PGP SIGNATURE-----

E
E
Eli Zaretskii via web wrote on 18 Sep 2020 08:53
Emacs 27.1 memory consumption grows indefinitely
(address . 43406@debbugs.gnu.org)
7f6cec640620.a886a4eb1532c11@guile.gnu.org
Please don't use "memory profile" as any indication of the Emacs memory usage: it is entirely unrelated to memory.

The Emacs profiler only profiles CPU usage. It samples the PC register using a special signal (SIGPROF), which is the usual method of sampling CPU in profilers. For those systems which don't have SIGPROF, it can fall back on sampling the CPU whenever Emacs calls one of the memory allocation functions -- this is called "mem" profiling.

So the "memory" profiles shown here don't show memory usage.
L
L
Ludovic Courtès wrote on 18 Sep 2020 10:22
control message for bug #43406
(address . control@debbugs.gnu.org)
87363f334u.fsf@gnu.org
retitle 43406 Helm 3.6.5 on Emacs 27.1 leaks memory
quit
L
L
Ludovic Courtès wrote on 18 Sep 2020 10:23
(name . Joshua Branson)(address . jbranso@dismail.de)(address . 43406-done@debbugs.gnu.org)
871riz334l.fsf@gnu.org
Hello!

Joshua Branson <jbranso@dismail.de> skribis:

Toggle quote (4 lines)
> I do think it has to do with helm. I can confirm that disabling
> helm-ff-cache with (setq helm-ff-keep-cached-candidates nil) really did
> the trick.

Same here. A couple of days later, I can confirm that my Emacs has
become sane again.

Closing!

Ludo’.
Closed
L
L
Ludovic Courtès wrote on 18 Sep 2020 13:47
Re: bug#43406: Emacs 27.1 memory consumption grows indefinitely
(name . Eli Zaretskii)(address . eliz@gnu.org)(address . 43406@debbugs.gnu.org)
87r1qzz4qe.fsf@gnu.org
Hi Eli,

Eli Zaretskii via web <issues.guix.gnu.org@elephly.net> skribis:

Toggle quote (6 lines)
> Please don't use "memory profile" as any indication of the Emacs memory usage: it is entirely unrelated to memory.
>
> The Emacs profiler only profiles CPU usage. It samples the PC register using a special signal (SIGPROF), which is the usual method of sampling CPU in profilers. For those systems which don't have SIGPROF, it can fall back on sampling the CPU whenever Emacs calls one of the memory allocation functions -- this is called "mem" profiling.
>
> So the "memory" profiles shown here don't show memory usage.

Understood, that’s the same thing as Guile’s ‘gcprof’. :-)

Thanks for your feedback,
Ludo’.
?
Your comment

This issue is archived.

To comment on this conversation send an email to 43406@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 43406
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch