[PATCH 0/2] Add docker.

Danny Milosavljevic (2):

gnu: Add docker-engine.

gnu: Add docker-cli.

gnu/packages/docker.scm | 144 ++++++++++++++++++++++++++++++++++++++++

1 file changed, 144 insertions(+)

(docker-engine): New variable. Export it.

---

gnu/packages/docker.scm | 83 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 83 insertions(+)

---

gnu/packages/docker.scm | 61 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 61 insertions(+)

Danny Milosavljevic (3):

gnu: Add containerd.

gnu: Add docker-engine.

services: Add docker.

gnu/local.mk | 1 +

gnu/packages/docker.scm | 201 +++++++++++++++++++++++++++++++++++++++-

gnu/services/docker.scm | 90 ++++++++++++++++++

3 files changed, 291 insertions(+), 1 deletion(-)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 49 insertions(+)

(%docker-version): New variable.

---

gnu/packages/docker.scm | 152 +++++++++++++++++++++++++++++++++++++++-

1 file changed, 151 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/services/docker.scm | 90 +++++++++++++++++++++++++++++++++++++++++

2 files changed, 91 insertions(+)

create mode 100644 gnu/services/docker.scm

Includes docker-cli now.

Danny Milosavljevic (4):

gnu: Add containerd.

gnu: Add docker-engine.

services: Add docker.

gnu: Add docker-cli.

gnu/local.mk | 1 +

gnu/packages/docker.scm | 264 +++++++++++++++++++++++++++++++++++++++-

gnu/services/docker.scm | 90 ++++++++++++++

3 files changed, 354 insertions(+), 1 deletion(-)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 49 insertions(+)

(%docker-version): New variable.

---

gnu/packages/docker.scm | 152 +++++++++++++++++++++++++++++++++++++++-

1 file changed, 151 insertions(+), 1 deletion(-)

---

gnu/local.mk | 1 +

gnu/services/docker.scm | 90 +++++++++++++++++++++++++++++++++++++++++

2 files changed, 91 insertions(+)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 63 insertions(+)

Better with this additional patch:

Now with macro.

Danny Milosavljevic (4):

gnu: Add containerd.

gnu: Add docker-engine.

services: Add docker.

gnu: Add docker-cli.

doc/guix.texi | 10 ++

gnu/local.mk | 1 +

gnu/packages/docker.scm | 261 +++++++++++++++++++++++++++++++++++++++-

gnu/services/docker.scm | 93 ++++++++++++++

4 files changed, 364 insertions(+), 1 deletion(-)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 49 insertions(+)

(%docker-version): New variable.

---

gnu/packages/docker.scm | 149 +++++++++++++++++++++++++++++++++++++++-

1 file changed, 148 insertions(+), 1 deletion(-)

---

doc/guix.texi | 10 +++++

gnu/local.mk | 1 +

gnu/services/docker.scm | 93 +++++++++++++++++++++++++++++++++++++++++

3 files changed, 104 insertions(+)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 63 insertions(+)

Danny Milosavljevic (4):

gnu: Add containerd.

gnu: Add docker-engine.

services: Add docker.

gnu: Add docker-cli.

doc/guix.texi | 10 ++

gnu/local.mk | 1 +

gnu/packages/docker.scm | 299 +++++++++++++++++++++++++++++++++++++++-

gnu/services/docker.scm | 93 +++++++++++++

4 files changed, 402 insertions(+), 1 deletion(-)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 68 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 68 insertions(+)

(%docker-version): New variable.

---

gnu/packages/docker.scm | 168 +++++++++++++++++++++++++++++++++++++++-

1 file changed, 167 insertions(+), 1 deletion(-)

---

doc/guix.texi | 10 +++++

gnu/local.mk | 1 +

gnu/services/docker.scm | 93 +++++++++++++++++++++++++++++++++++++++++

3 files changed, 104 insertions(+)

create mode 100644 gnu/services/docker.scm

---

gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++

1 file changed, 63 insertions(+)

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

LGTM, thanks!

Ludo’.

Danny Milosavljevic <dannym@scratchpost.org> skribis:

[...]

No longer needed?

Could you add a comment saying what AUTO_GOPATH does?

It’s not clear to me what should be fixed; perhaps a leftover?

That’s potentially problematic. :-) Any idea how difficult it would be

to run these tests?

Comments can be removed?

Otherwise LGTM, thanks!

Ludo’.

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Nice!

“Docker” with a capital.

We’re missing “@end defvr” I guess.

I think we shouldn’t propagate the narrative that Docker = container.

So what about something like:

This is the type of the service that runs @url{…, Docker}, a daemon

that can execute application bundles (sometimes referred to as

``containers'') in isolated environments.

?

Also could you document ‘docker-configuration’ as well?

[...]

I suppose there could be a separate ‘containerd-service-type’ if it’s

useful; if it’s not, it’s OK to keep it this way.

As for the dependency, users would have to add both docker and

containerd to their service list, or docker-service-type could extend

containerd-service-type, which would ensure containerd-service-type is

automatically instantiated if it’s not already in the user’s service

list.

You can make the above (lambda (config) …) instead of (lambda (args) …).

Please add a ‘description’ field here, and please remove tabs from the

file. :-)

Could you consider adding a system test for docker/containerd? Perhaps

we could go as far as using ‘docker-image’ in (guix scripts pack) to

generate an image and make sure ‘docker load’ works, but maybe that’s

too much work.

Thank you,

Ludo’.

Danny Milosavljevic <dannym@scratchpost.org> skribis:

[...]

I suppose we cannot run the daemon in the build environment, can we?

Or is it possible to use some of the tests?

Shouldn’t libltdl be an input?

Otherwise LGTM, thanks!

Ludo’.

Hi Ludo,

On Sun, 06 Jan 2019 21:20:35 +0100

Ludovic Courtès <ludo@gnu.org> wrote:

It was meant as a detector in order to make compilation fail when, in future

versions, docker wants to invok new stuff that we didn't patch yet.

Should we do that?

Yes, I'll add one.

Yeah, I meant to check what hack/validate/default does and it seems to do

developer-specific tests (commit message formatted the right way etc), so

I guess we can just not invoke it.

Go has peculiar ideas of how the directory layout is supposed to be set up.

I could probably figure it out - but if someone with more Go knowledge could

step forward it would be much faster.

Yeah.

Thanks!

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

I see, it sounds like a good idea. Also add a comment explaining the

rationale.

OK.

I see Leo is Cc’d so we’ll see. :-)

Thank you,

Ludo’.

Hi Ludo,

Hi Leo,

On Tue, 08 Jan 2019 09:42:14 +0100

Ludovic Courtès <ludo@gnu.org> wrote:

Nevermind, I've fixed it and learned something in the process:

Linux doesn't actually know the current working directory as a string.

It only knows the inode, so if you call getcwd, what libc actually does is

it opendirs "..", then finds the entry with the same inode number as

the current directory, and then returns the name of that entry.

Now, gopath uses symlinks to set up their preferred directory hierarchy

in such a way:

ln -s ../../../.. .gopath/src/github.com/docker/docker

Now if you chdir into ".gopath/src/github.com/docker/docker" and then Go later

does getcwd, it will appear as if the chdir did not succeed (because it will

just use the old working directory because it has the same inode).

So Go was erroring out because the directory structure there was *still* wrong.

Solution: Set environment variable PWD to the correct name of the directory.

I've pushed this patchset to master.

I'll try to add a system test next - let's see.

Howdy!

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Are you sure? In the Linux port of glibc I see this:

And indeed, there’s a ‘getcwd’ syscall:

Great that you found a solution.

Thanks for taking the time to address this!

Ludo’.

Hi Ludo,

On Thu, 10 Jan 2019 09:50:49 +0100

Ludovic Courtès <ludo@gnu.org> wrote:

According to the POSIX standard ;)

Huh. I guess it boils down to whether the Linux "process" structure

has the cwd in it as a string or as an inode.

In Linux sources:

static inline void get_fs_pwd(struct fs_struct *fs, struct path *pwd)

{

spin_lock(&fs->lock);

*pwd = fs->pwd;

path_get(pwd);

spin_unlock(&fs->lock);

}

static void get_fs_root_and_pwd_rcu(struct fs_struct *fs, struct path *root,

struct path *pwd)

{

unsigned seq;

do {

seq = read_seqcount_begin(&fs->seq);

*root = fs->root;

*pwd = fs->pwd;

} while (read_seqcount_retry(&fs->seq, seq));

}

struct path {

struct vfsmount *mnt;

struct dentry *dentry;

} __randomize_layout;

SYSCALL_DEFINE2(getcwd, char __user *, buf, unsigned long, size)

{

int error;

struct path pwd, root;

char *page = __getname();

if (!page)

return -ENOMEM;

rcu_read_lock();

get_fs_root_and_pwd_rcu(current->fs, &root, &pwd);

error = -ENOENT;

if (!d_unlinked(pwd.dentry)) {

unsigned long len;

char *cwd = page + PATH_MAX;

int buflen = PATH_MAX;

prepend(&cwd, &buflen, "\0", 1);

error = prepend_path(&pwd, &root, &cwd, &buflen);

rcu_read_unlock();

if (error < 0)

goto out;

/* Unreachable from current root */

if (error > 0) {

error = prepend_unreachable(&cwd, &buflen);

if (error)

goto out;

}

error = -ERANGE;

len = PATH_MAX + page - cwd;

if (len <= size) {

error = len;

if (copy_to_user(buf, cwd, len))

error = -EFAULT;

}

} else {

rcu_read_unlock();

}

out:

__putname(page);

return error;

}

/*

*/

void set_fs_pwd(struct fs_struct *fs, const struct path *path)

{

struct path old_pwd;

path_get(path);

spin_lock(&fs->lock);

write_seqcount_begin(&fs->seq);

old_pwd = fs->pwd;

fs->pwd = *path; <----------------- !!!!

write_seqcount_end(&fs->seq);

spin_unlock(&fs->lock);

if (old_pwd.dentry)

path_put(&old_pwd);

}

int ksys_chdir(const char __user *filename)

{

struct path path;

int error;

unsigned int lookup_flags = LOOKUP_FOLLOW | LOOKUP_DIRECTORY;

retry:

error = user_path_at(AT_FDCWD, filename, lookup_flags, &path);

if (error)

goto out;

error = inode_permission(path.dentry->d_inode, MAY_EXEC | MAY_CHDIR);

if (error)

goto dput_and_out;

set_fs_pwd(current->fs, &path); <----------------- !!!

dput_and_out:

path_put(&path);

if (retry_estale(error, lookup_flags)) {

lookup_flags |= LOOKUP_REVAL;

goto retry;

}

out:

return error;

}

SYSCALL_DEFINE1(chdir, const char __user *, filename)

{

return ksys_chdir(filename);

}

SYSCALL_DEFINE1(fchdir, unsigned int, fd)

{

struct fd f = fdget_raw(fd);

int error;

error = -EBADF;

if (!f.file)

goto out;

error = -ENOTDIR;

if (!d_can_lookup(f.file->f_path.dentry))

goto out_putf;

error = inode_permission(file_inode(f.file), MAY_EXEC | MAY_CHDIR);

if (!error)

set_fs_pwd(current->fs, &f.file->f_path);

out_putf:

fdput(f);

out:

return error;

}

Interesting!

No problem :)

On Thu, Jan 10, 2019 at 03:22:10AM +0100, Danny Milosavljevic wrote:

Indeed, Go is very particular about this...

Okay, good :) Let me know if you have more Go questions.

Hi Danny,

docker-cli provides two identical commands in

"/gnu/store/*docker-cli*/bin/".

#+begin_SRC sh

~ ll /gnu/store/*docker-cli*/bin/*

-r-xr-xr-x 3 root root 64M Jan 1 1970 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker*

-r-xr-xr-x 3 root root 64M Jan 1 1970 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker-linux-amd64*

~ sha256sum /gnu/store/*docker-cli*/bin/*

62bc8199fd11f37129d6e8183865df698f495faf90a86bdbe5ee4891b201cbc8 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker

62bc8199fd11f37129d6e8183865df698f495faf90a86bdbe5ee4891b201cbc8 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker-linux-amd64

#+end_SRC

It wastes 64MB disk space. Can we remove "docker-linux-amd64"?

--

Meiyo Peng

Hi,

Done in commit f3705090965c2470a0ccc2c045edbc5f5fb7bb8d.

Thanks!