Out of memory error when generating a docker-image.

  • Done
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • Leo Famulari
  • Roel Janssen
Owner
unassigned
Submitted by
Roel Janssen
Severity
normal
R
R
Roel Janssen wrote on 3 May 2021 00:09
(address . bug-guix@gnu.org)
2f30ed4cd1e7154fc49f800a2d8950df2ac6bc8f.camel@gnu.org
Dear Guix,

While generating a docker-image that contains only a sshd service, an
nginx service, and an iptables service, it seems the VM that is spawned
to generate a docker image was allocated too little memory. I attached
the log file.

Looking at 'guix/scripts/system.scm', it seems that we always pass 256M
of memory to the VM. After bumping that to 4096M, I was able to
produce a docker image.

I'd like to see what we can do here. Assigning too little memory leads
to problems generating the container, but assigning too much memory wil
l cause problems for computing machines that don't have much memory to
spare.

Would it be a good idea to make it configurable at run-time?

Kind regards,
Roel Janssen
Attachment: guix-docker-image.log
L
L
Leo Famulari wrote on 3 May 2021 18:39
(name . Roel Janssen)(address . roel@gnu.org)(address . 48178@debbugs.gnu.org)
YJAnKAe7HCWy/lpd@jasmine.lan
On Mon, May 03, 2021 at 12:09:36AM +0200, Roel Janssen wrote:
Toggle quote (4 lines)
> Looking at 'guix/scripts/system.scm', it seems that we always pass 256M
> of memory to the VM. After bumping that to 4096M, I was able to
> produce a docker image.

Can you test somes values that are in between? Like, 512M, 1024M, etc,
until we know how much is actually required? If 512M is enough, I don't
see a problem with increasing the hard-coded value to that.

Toggle quote (5 lines)
> I'd like to see what we can do here. Assigning too little memory leads
> to problems generating the container, but assigning too much memory wil
> l cause problems for computing machines that don't have much memory to
> spare.

There are some use cases for this code that we'd like to work on
low-resource machines (`guix system vm`), and other use cases (like
building Docker images) that shouldn't be expected to work on machines
with limited RAM.

Toggle quote (2 lines)
> Would it be a good idea to make it configurable at run-time?

Yeah, maybe.
R
R
Roel Janssen wrote on 3 May 2021 19:47
(name . Leo Famulari)(address . leo@famulari.name)(address . 48178@debbugs.gnu.org)
5578dcab6d15ae85a544b92f03b6a5d89223967c.camel@gnu.org
On Mon, 2021-05-03 at 12:39 -0400, Leo Famulari wrote:
Toggle quote (13 lines)
> On Mon, May 03, 2021 at 12:09:36AM +0200, Roel Janssen wrote:
> > Looking at 'guix/scripts/system.scm', it seems that we always pass
> > 256M
> > of memory to the VM.  After bumping that to 4096M, I was able to
> > produce a docker image.
>
> Can you test somes values that are in between? Like, 512M, 1024M,
> etc,
> until we know how much is actually required? If 512M is enough, I
> don't
> see a problem with increasing the hard-coded value to that.
>

I monitored the VM's memory usage and it peaked at 1.6G. But after
testing, it seems 1024 also works.

I tested with 2048 (worked), 1024 (worked), and 512 (didn't work).

Toggle quote (9 lines)
> > I'd like to see what we can do here.  Assigning too little memory
> > leads
> > to problems generating the container, but assigning too much memory
> > wil
> > l cause problems for computing machines that don't have much memory
> > to
> > spare.
>

In that case... The attached patch would only increase the size when
generating a Docker container image. Would that be acceptable?

Toggle quote (11 lines)
> There are some use cases for this code that we'd like to work on
> low-resource machines (`guix system vm`), and other use cases (like
> building Docker images) that shouldn't be expected to work on
> machines
> with limited RAM.
>
> > Would it be a good idea to make it configurable at run-time?
>
> Yeah, maybe.
>

I think it'd be better to have it somehow dynamically increase, but I
don't see how I could determine the VM size needed for a given system
configuration. So perhaps the attached patch is an acceptable
compromise.

Kind regards,
Roel Janssen
From 0dcaf5cfc529f0ce31af46d4f3039f4b7dbebb9b Mon Sep 17 00:00:00 2001
From: Roel Janssen <roel@gnu.org>
Date: Mon, 3 May 2021 18:54:08 +0200
Subject: [PATCH] guix: scripts system: Increase the size for generating Docker
container images.

* guix/scripts/system.scm: Override the default of 256.
---
guix/scripts/system.scm | 1 +
1 file changed, 1 insertion(+)

Toggle diff (14 lines)
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 0a051ee4e3..a25e0dac52 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -717,6 +717,7 @@ checking this by themselves in their 'check' procedure."
(lower-object (system-image image)))
((docker-image)
(system-docker-image os
+ #:memory-size 1024
#:shared-network? container-shared-network?)))))
(define (maybe-suggest-running-guix-pull)
--
2.31.1
E
E
Efraim Flashner wrote on 10 May 2021 09:29
(name . Roel Janssen)(address . roel@gnu.org)
YJjgyrRBM5xTkhGd@3900XT
On Mon, May 03, 2021 at 07:47:49PM +0200, Roel Janssen wrote:
Toggle quote (51 lines)
> On Mon, 2021-05-03 at 12:39 -0400, Leo Famulari wrote:
> > On Mon, May 03, 2021 at 12:09:36AM +0200, Roel Janssen wrote:
> > > Looking at 'guix/scripts/system.scm', it seems that we always pass
> > > 256M
> > > of memory to the VM.  After bumping that to 4096M, I was able to
> > > produce a docker image.
> >
> > Can you test somes values that are in between? Like, 512M, 1024M,
> > etc,
> > until we know how much is actually required? If 512M is enough, I
> > don't
> > see a problem with increasing the hard-coded value to that.
> >
>
> I monitored the VM's memory usage and it peaked at 1.6G. But after
> testing, it seems 1024 also works.
>
> I tested with 2048 (worked), 1024 (worked), and 512 (didn't work).
>
> > > I'd like to see what we can do here.  Assigning too little memory
> > > leads
> > > to problems generating the container, but assigning too much memory
> > > wil
> > > l cause problems for computing machines that don't have much memory
> > > to
> > > spare.
> >
>
> In that case... The attached patch would only increase the size when
> generating a Docker container image. Would that be acceptable?
>
> > There are some use cases for this code that we'd like to work on
> > low-resource machines (`guix system vm`), and other use cases (like
> > building Docker images) that shouldn't be expected to work on
> > machines
> > with limited RAM.
> >
> > > Would it be a good idea to make it configurable at run-time?
> >
> > Yeah, maybe.
> >
>
> I think it'd be better to have it somehow dynamically increase, but I
> don't see how I could determine the VM size needed for a given system
> configuration. So perhaps the attached patch is an acceptable
> compromise.
>
> Kind regards,
> Roel Janssen
>

Looks good to me!

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmCY4MoACgkQQarn3Mo9
g1FEow/+MY9PlN5OaK4wwuR6aFG+J3hLk5RxIrD8FBEwjsVTrbcWqkXDtimwBBZE
A3JNtOZK/Qh8W9gFuMtksmEQC5FjhJkZwUPve53ThB4TiAbCmGHdH6Hsu1//Mzb2
BnoLjzrt3YHepTmvr5P/UYQvA1ris+k3M1WfWGnBKaeXeN8BgdKgZR+rHLuiNUsn
7rSzZGYvCF038yk72P5MBJEK/4zqaJyE1eZNsX63WNjyrA3oWf2/l+D20xx1erbS
YK21Tok5Kx7XT8FfxwwyUZDAXfVkZ/dyYFD6qfomRPjBcENsjPlwQIGB8X1tui50
c5C+MVzDwWQkoMXcIZp+3d+JdXssh4LjwYfwSQ8rmI4FJhNRzmcEHVjTfryTsfAH
eqUeJLA5+VEeE/qeP42DgPsFOAscDtlXPOX6rRb1e778Z5Jv2EwQmQI3ljvzDX1p
MBDLGAJVAkx4XsvQ4ukX1Y8mCuuPMhk2fdePGBn8OhLB7g2VrrhWnBn+jH4yQuoK
tRVdTZzlRTrzsZgyO8fWKmx97qPpp5buHgiBgEzRt9Ld2KM0lFRqTG1P7z88zjwP
Hwmry1UqPkLhCvi5cfXfsOyjnCq0JQ0+Uaic1sKgR70OxPnxFMthXB6xRQV6mJvm
IHBOiP5Z30dJJpXv1eyjcYQDKHMGPDMr0Zhs0lFaVGUYJRnMGfg=
=XSpy
-----END PGP SIGNATURE-----


R
R
Roel Janssen wrote on 10 May 2021 09:46
(name . Efraim Flashner)(address . efraim@flashner.co.il)
1d6f2f78d43e79457f53d865ae1905385da30006.camel@gnu.org
On Mon, 2021-05-10 at 10:29 +0300, Efraim Flashner wrote:
Toggle quote (64 lines)
> On Mon, May 03, 2021 at 07:47:49PM +0200, Roel Janssen wrote:
> > On Mon, 2021-05-03 at 12:39 -0400, Leo Famulari wrote:
> > > On Mon, May 03, 2021 at 12:09:36AM +0200, Roel Janssen wrote:
> > > > Looking at 'guix/scripts/system.scm', it seems that we always
> > > > pass
> > > > 256M
> > > > of memory to the VM.  After bumping that to 4096M, I was able
> > > > to
> > > > produce a docker image.
> > >
> > > Can you test somes values that are in between? Like, 512M, 1024M,
> > > etc,
> > > until we know how much is actually required? If 512M is enough, I
> > > don't
> > > see a problem with increasing the hard-coded value to that.
> > >
> >
> > I monitored the VM's memory usage and it peaked at 1.6G. But after
> > testing, it seems 1024 also works.
> >
> > I tested with 2048 (worked), 1024 (worked), and 512 (didn't work).
> >
> > > > I'd like to see what we can do here.  Assigning too little
> > > > memory
> > > > leads
> > > > to problems generating the container, but assigning too much
> > > > memory
> > > > wil
> > > > l cause problems for computing machines that don't have much
> > > > memory
> > > > to
> > > > spare.
> > >
> >
> > In that case... The attached patch would only increase the size
> > when
> > generating a Docker container image.  Would that be acceptable?
> >
> > > There are some use cases for this code that we'd like to work on
> > > low-resource machines (`guix system vm`), and other use cases
> > > (like
> > > building Docker images) that shouldn't be expected to work on
> > > machines
> > > with limited RAM.
> > >
> > > > Would it be a good idea to make it configurable at run-time?
> > >
> > > Yeah, maybe.
> > >
> >
> > I think it'd be better to have it somehow dynamically increase, but
> > I
> > don't see how I could determine the VM size needed for a given
> > system
> > configuration.  So perhaps the attached patch is an acceptable
> > compromise.
> >
> > Kind regards,
> > Roel Janssen
> >
>
> Looks good to me!
>

Thank you for looking at it! I pushed the proposed patch in
ce3d05cc08c01351756ab5d5b7f25cfe0295c230.

Kind regards,
Roel Janssen
Closed
?