[PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

Done

Details

6 participants

Efraim Flashner
Jonathan Brielmaier
Ludovic Courtès
Pierre Neidhardt
Maxim Cournoyer
Marius Bakke

Owner: unassigned

Submitted by: Pierre Neidhardt

Severity: normal

Blocked by

37305

Pierre Neidhardt wrote on 26 Mar 2020 09:35

Recipients:(address . guix-patches@gnu.org)

Message-ID:20200326083524.20275-1-mail@ambrevar.xyz

* gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.

---

gnu/system/examples/desktop.tmpl | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)

Toggle diff (16 lines)diff --git a/gnu/system/examples/desktop.tmpl b/gnu/system/examples/desktop.tmpl
index 3931bad60d..e61e8064cd 100644
--- a/gnu/system/examples/desktop.tmpl
+++ b/gnu/system/examples/desktop.tmpl
@@ -34,7 +34,8 @@
                  (list (file-system
                          (device (file-system-label "my-root"))
                          (mount-point "/")
-                         (type "ext4")
+                         (type "btrfs")
+                         (options "subvol=rootfs,compress=zstd")
                          (dependencies mapped-devices))
                        (file-system
                          (device (uuid "1234-ABCD" 'fat))
-- 
2.25.1

Maxim Cournoyer wrote on 31 Mar 2020 03:26

control message for bug #40236

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87lfnhz544.fsf@gmail.com

block 40236 by 37305

quit

Maxim Cournoyer wrote on 31 Mar 2020 03:52

Re: [bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)(address . 40236@debbugs.gnu.org)

Message-ID:87h7y5z3x0.fsf@gmail.com

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (20 lines)> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
> ---
>  gnu/system/examples/desktop.tmpl | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/system/examples/desktop.tmpl b/gnu/system/examples/desktop.tmpl
> index 3931bad60d..e61e8064cd 100644
> --- a/gnu/system/examples/desktop.tmpl
> +++ b/gnu/system/examples/desktop.tmpl
> @@ -34,7 +34,8 @@
>                   (list (file-system
>                           (device (file-system-label "my-root"))
>                           (mount-point "/")
> -                         (type "ext4")
> +                         (type "btrfs")
> +                         (options "subvol=rootfs,compress=zstd")
>                           (dependencies mapped-devices))
>                         (file-system
>                           (device (uuid "1234-ABCD" 'fat))

I think Btrfs with compression is a fine, modern default of a file

system, but we shall get a good samples of opinions (I expect a variety

of them :-)) and common agreement before pushing this change.

For those wondering about the benefits of having the root file system on

a subvolume (named 'rootfs' in the proposed configuration), the

following page [0] explains it as:

[...] The above layout, which obviously serves as the system's

"main" filesystem, places data directly within the top-level

subvolume (namely everything for example /usr, that's not in a

child subvolume) This makes changing the structure (for example to

something more flat) more difficult, which is why it's generally

suggested to place the actual data in a subvolume (that is not the

top-level subvolume), in the above example, a better layout would

be the following:

toplevel (volume root directory, not mounted)

\-- root (subvolume root directory, to be mounted at /)

+-- home (subvolume root directory)

+-- var (subvolume root directory)

+-- www (subvolume root directory)

+-- lib (directory)

\-- postgresql (subvolume root directory)

In short, it allows exposing just the subvolumes which should be

visible, instead of everything.

I've marked this blocked by 37305, which includes GRUB support for

booting from a subvolume as well as documentation about Btrfs usage in

Guix.

[0] https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout

Pierre Neidhardt wrote on 31 Mar 2020 09:52

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 40236@debbugs.gnu.org)

Message-ID:878sjharlc.fsf@ambrevar.xyz

Thanks!

Pierre Neidhardt

https://ambrevar.xyz/

Jonathan Brielmaier wrote on 31 Mar 2020 14:09

Recipients:(address . guix-patches@gnu.org)

Message-ID:c59e2a29-00fe-373e-a941-0652a1a75f95@web.de

On 26.03.20 09:35, Pierre Neidhardt wrote:

Toggle quote (2 lines)

> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.

I would oppose this change as I had too much troubles with Btrfs on my
openSUSE machines. I used a simple partition layout spanning the whole
disk. AFAIK this is not really recommended with btrfs. My laptop was
from time to time unusable until I reinstalled it with ext4...

Further do we need all this rollback stuff from btrfs if we have it
already in Guix?

Pierre Neidhardt wrote on 31 Mar 2020 16:53

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 40236@debbugs.gnu.org)

Message-ID:87lfnga83c.fsf@ambrevar.xyz

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

Toggle quote (7 lines)

>> (list (file-system

>> (device (file-system-label "my-root"))

>> (mount-point "/")

>> - (type "ext4")

>> + (type "btrfs")

>> + (options "subvol=rootfs,compress=zstd")

By the way in your patch you did not document the syntax (options
"subvol=...").
Is it supported nonetheless in your patch?  Is it supported in current Guix?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

Maxim Cournoyer wrote on 1 Apr 2020 01:20

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)(address . 40236@debbugs.gnu.org)

Message-ID:87tv24w1ql.fsf@gmail.com

Hello Pierre!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (12 lines)> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>>>                   (list (file-system
>>>                           (device (file-system-label "my-root"))
>>>                           (mount-point "/")
>>> -                         (type "ext4")
>>> +                         (type "btrfs")
>>> +                         (options "subvol=rootfs,compress=zstd")
>
> By the way in your patch you did not document the syntax (options
> "subvol=...").

Are you sure you are looking at the right patch series

(https://issues.guix.info/issue/37305#16)?I documented usage examples

for a root file system using Btrfs subvolumes.

Toggle quote (2 lines)

> Is it supported nonetheless in your patch?

Yes.

Toggle quote (2 lines)

> Is it supported in current Guix?

Not currently; the patch set is in review :-). I have it running on 3

machines and there's a system test which shows it working as well, so

don't be afraid to try it!

Maxim

Pierre Neidhardt wrote on 1 Apr 2020 09:00

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 40236@debbugs.gnu.org)

Message-ID:87y2rf8zcx.fsf@ambrevar.xyz

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

Toggle quote (7 lines)

>> By the way in your patch you did not document the syntax (options

>> "subvol=...").

> Are you sure you are looking at the right patch series

> (https://issues.guix.info/issue/37305#16)? I documented usage examples

> for a root file system using Btrfs subvolumes.

I was looking at the wrong patch indeed, sorry about that! :)

Toggle quote (10 lines)>> Is it supported nonetheless in your patch?
>
> Yes.
>
>> Is it supported in current Guix?
>
> Not currently; the patch set is in review :-).  I have it running on 3
> machines and there's a system test which shows it working as well, so
> don't be afraid to try it!

Does this go on master or core-updates?
Does it rebuild the world?
I don't have much hardware to build anything at the moment, so I can only test
this if it's a small build ;)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

Ludovic Courtès wrote on 1 Apr 2020 23:28

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87lfnesxp6.fsf@gnu.org

Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (4 lines)

> I've marked this blocked by 37305, which includes GRUB support for

> booting from a subvolume as well as documentation about Btrfs usage in

> Guix.

Also, the installer defaults to ext4 and that’s probably a more tested

configuration currently.

Ludo’.

Pierre Neidhardt wrote on 2 Apr 2020 09:15

Recipients:(address . 40236@debbugs.gnu.org)

Message-ID:87pncqbbp5.fsf@ambrevar.xyz

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)

> Also, the installer defaults to ext4 and that’s probably a more tested

> configuration currently.

Could a subvolume-less Btrfs be more problematic than ext4?

If not, it might be a good idea to also switch to Btrfs in the installer.

I've tested Btrfs with Zstd:3 compression on a new GNOME + EXWM install:

I've achieved a stunning 30% compression like Maxim.

I must say that it's extremely useful, if not necessary, on a 60GiB SSD... :D

Pierre Neidhardt

https://ambrevar.xyz/

Ludovic Courtès wrote on 2 Apr 2020 10:04

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87zhbuqpns.fsf@gnu.org

Hi,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

Toggle quote (8 lines)

> Ludovic Courtès <ludo@gnu.org> writes:

>> Also, the installer defaults to ext4 and that’s probably a more tested

>> configuration currently.

> Could a subvolume-less Btrfs be more problematic than ext4?

> If not, it might be a good idea to also switch to Btrfs in the installer.

I think it’s more work than just switching: we have to do enough

testing, add new automated tests, etc.

Also, any change in this area should happen post-release. :-)

Thanks,

Ludo’.

Jonathan Brielmaier wrote on 2 Apr 2020 12:36

Recipients:(address . guix-patches@gnu.org)

Message-ID:17b2d82a-a18d-5a2b-f194-6e00890802be@web.de

On 02.04.20 10:04, Ludovic Courtès wrote:

Toggle quote (11 lines)>>> Also, the installer defaults to ext4 and that’s probably a more tested
>>> configuration currently.
>>
>> Could a subvolume-less Btrfs be more problematic than ext4?
>> If not, it might be a good idea to also switch to Btrfs in the installer.
>
> I think it’s more work than just switching: we have to do enough
> testing, add new automated tests, etc.
>
> Also, any change in this area should happen post-release.  :-)

Why not adding a gnu/system/examples/btrfs.tmpl in the mean time. There

you could showcase all the stuff btrfs can do for us :)

Maxim Cournoyer wrote on 4 Apr 2020 03:28

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87y2rc6nuk.fsf@gmail.com

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (15 lines)> Hi,
>
> Pierre Neidhardt <mail@ambrevar.xyz> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Also, the installer defaults to ext4 and that’s probably a more tested
>>> configuration currently.
>>
>> Could a subvolume-less Btrfs be more problematic than ext4?
>> If not, it might be a good idea to also switch to Btrfs in the installer.
>
> I think it’s more work than just switching: we have to do enough
> testing, add new automated tests, etc.

Fine points.

Toggle quote (2 lines)

> Also, any change in this area should happen post-release. :-)

Agreed!

Let's keep this issue open and revisit what extra tests and validation

should be done after the release to consider a switch viable.

Maxim

Pierre Neidhardt wrote on 6 Apr 2020 22:20

Recipients:(address . 40236@debbugs.gnu.org)

Message-ID:87r1x0welb.fsf@ambrevar.xyz

Hi Jonathan,

For some reason I did not receive your email:

Jonathan Brielmaier wrote on Tue Mar 31 14:09:04+0200 2020

Toggle quote (7 lines)

> > * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.

> I would oppose this change as I had too much troubles with Btrfs on my

> openSUSE machines. I used a simple partition layout spanning the whole

> disk. AFAIK this is not really recommended with btrfs. My laptop was

> from time to time unusable until I reinstalled it with ext4...

This is for Guix System only. Did you mean that it would be problematic

if you were running openSUSE in dual boot?

Toggle quote (3 lines)

> Further do we need all this rollback stuff from btrfs if we have it

> already in Guix?

Btrfs has many benefits over Ext3:

it offers compression to about 30% on average, it supports subvolumes,

snapshots and snapshot syncing, and much more.

For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)

Pierre Neidhardt

https://ambrevar.xyz/

Jonathan Brielmaier wrote on 6 Apr 2020 22:42

Recipients:(address . 40236@debbugs.gnu.org)

Message-ID:c9bde2d4-01ba-4827-f667-05b83d66d66e@web.de

On 06.04.20 22:20, Pierre Neidhardt wrote:> Jonathan Brielmaier wrote on

Tue Mar 31 14:09:04+0200 2020

Toggle quote (10 lines)>>> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
>>
>> I would oppose this change as I had too much troubles with Btrfs on my
>> openSUSE machines. I used a simple partition layout spanning the whole
>> disk. AFAIK this is not really recommended with btrfs. My laptop was
>> from time to time unusable until I reinstalled it with ext4...
>
> This is for Guix System only.  Did you mean that it would be problematic
> if you were running openSUSE in dual boot?

No, I did run openSUSE on btrfs, there was no Guix involved at all. But

btrfs seemed to be the root case of all my troubles (performance,

hang-ups etc).

Toggle quote (8 lines)

>> Further do we need all this rollback stuff from btrfs if we have it

>> already in Guix?

> Btrfs has many benefits over Ext3:

> it offers compression to about 30% on average, it supports subvolumes,

> snapshots and snapshot syncing, and much more.

Snapshots did fill up my disk. I had no use for them on my laptop. On a

Guix System even less, because you have rollbacks from the package

manager :)

Toggle quote (2 lines)

> For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)

So maybe create a config for the OSes of tomorrow: btrfs, wireguard,

rust etc :P

Pierre Neidhardt wrote on 7 Apr 2020 09:07

Recipients:(address . 40236@debbugs.gnu.org)

Message-ID:87o8s3wz7d.fsf@ambrevar.xyz

Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:

Toggle quote (4 lines)

> No, I did run openSUSE on btrfs, there was no Guix involved at all. But

> btrfs seemed to be the root case of all my troubles (performance,

> hang-ups etc).

When did you try this? Maybe some issues have been fixed in the

meantime.

I've used Btrfs on a few computers for many months now and it's been a

bliss. Of course experiences vary ;)

Toggle quote (10 lines)>>> Further do we need all this rollback stuff from btrfs if we have it
>>> already in Guix?
>>
>> Btrfs has many benefits over Ext3:
>>
>> it offers compression to about 30% on average, it supports subvolumes,
>> snapshots and snapshot syncing, and much more.
>
> Snapshots did fill up my disk.

What do you mean?

Toggle quote (4 lines)

> I had no use for them on my laptop. On a

> Guix System even less, because you have rollbacks from the package

> manager :)

Snapshots are not just for rollbacks: you can sync them across the

network to a remote machine. This allows you to deploy disk images very

fast.

Toggle quote (5 lines)

>> For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)

> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,

> rust etc :P

In the end, what I'm suggesting is this issue is merely a

recommendation.

Currently Guix is very annoying to use on small Ext4 partitions, e.g. a

64 GiB SSD. With compression on, you suddenly get 3x more space for

your /gnu/store :)

Pierre Neidhardt

https://ambrevar.xyz/

Maxim Cournoyer wrote on 8 Apr 2020 05:18

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87o8s2k6lr.fsf@gmail.com

Hello,

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (6 lines)

> Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:

>> No, I did run openSUSE on btrfs, there was no Guix involved at all. But

>> btrfs seemed to be the root case of all my troubles (performance,

>> hang-ups etc).

I have yet to encounter this kind of problem on the 3 Guix Systems I've

installed with root Btrfs file systems. It's been rock stable, even

under heavy use (I have a Guix machine configured as a Jenkins slave at

work that builds Yocto projects -- it churns through GiB of files

daily).

[...]

Toggle quote (2 lines)

>> Snapshots did fill up my disk.

Snapshots only fill up the disk when we use them (and leave them behind

for enough time that the content they refer to has been mostly

rewritten.

Toggle quote (10 lines)>> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
>> rust etc :P
>
> In the end, what I'm suggesting is this issue is merely a
> recommendation.
>
> Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
> 64 GiB SSD.  With compression on, you suddenly get 3x more space for
> your /gnu/store :)

I agree that compression is a nice feature.. It also speeds sequential
disk reads and writes. On an old laptop that has a 64 GiB SSD and uses
ext4, I have to 'guix gc' too often, and worry a lot about spaces
(there's literally not much else than Guix on the drive, but it manages
to fill it up quite easily :-).

Maxim

Efraim Flashner wrote on 9 Apr 2020 22:12

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:20200409201244.GR1518@E5400

On Tue, Apr 07, 2020 at 11:18:40PM -0400, Maxim Cournoyer wrote:

Toggle quote (41 lines)> Hello,
> 
> Pierre Neidhardt <mail@ambrevar.xyz> writes:
> 
> > Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:
> >
> >> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
> >> btrfs seemed to be the root case of all my troubles (performance,
> >> hang-ups etc).
> 
> I have yet to encounter this kind of problem on the 3 Guix Systems I've
> installed with root Btrfs file systems.  It's been rock stable, even
> under heavy use (I have a Guix machine configured as a Jenkins slave at
> work that builds Yocto projects -- it churns through GiB of files
> daily).
> 
> [...]
> 
> >> Snapshots did fill up my disk.
> 
> Snapshots only fill up the disk when we use them (and leave them behind
> for enough time that the content they refer to has been mostly
> rewritten.
> 
> >> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
> >> rust etc :P
> >
> > In the end, what I'm suggesting is this issue is merely a
> > recommendation.
> >
> > Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
> > 64 GiB SSD.  With compression on, you suddenly get 3x more space for
> > your /gnu/store :)
> 
> I agree that compression is a nice feature.. It also speeds sequential
> disk reads and writes. On an old laptop that has a 64 GiB SSD and uses
> ext4, I have to 'guix gc' too often, and worry a lot about spaces
> (there's literally not much else than Guix on the drive, but it manages
> to fill it up quite easily :-).
> 

(ins)efraim@E5400 ~$ sudo compsize -x /gnu/store/
Password:
Processed 3158140 files, 737675 regular extents (2467369 refs), 1335101 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       74%       51G          69G         180G
none       100%       32G          32G          86G
lzo         51%       19G          36G          93G

My understanding of this is that I have 36GB of files that are
compressed at 51% to 19GB, and overall due to the deduplication in the
store I have references to what would otherwise be 180GB total taking up
only 51G.

So compression saves me 26% ([69-51]/69), and deduplication saves me
62% ([180-69]/180).

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Pierre Neidhardt wrote on 10 Apr 2020 09:39

Recipients:

Message-ID:87o8rzvlg9.fsf@ambrevar.xyz

Toggle quote (3 lines)

> So compression saves me 26% ([69-51]/69), and deduplication saves me

> 62% ([180-69]/180).

Thanks for sharing!
zstd might give better results.  Any reason you chose lzo over zstd?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

Efraim Flashner wrote on 10 Apr 2020 10:24

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:20200410082425.GA1518@E5400

On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:

Toggle quote (7 lines)

> > So compression saves me 26% ([69-51]/69), and deduplication saves me

> > 62% ([180-69]/180).

> Thanks for sharing!

> zstd might give better results. Any reason you chose lzo over zstd?

My machine is about 10 years old so I was more concerned than normal
about the CPU usage. If lz4 was an option I would've gone with that, but
according to the Arch wiki or some other locations lzo was basically the
fastest option.

Since mail is mostly text I occasionally recompress that. I haven't
figured out how to use 'btrfs filesystem defragment' to choose the
compression level for zstd.

(ins)efraim@E5400 ~$ sudo compsize Maildir/
Password:
Processed 121378 files, 129245 regular extents (129245 refs), 19963 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       66%      3.3G         4.9G         4.9G
none       100%      282M         282M         282M
zlib        55%      137K         248K         248K
lzo         53%      125M         233M         233M
zstd        64%      2.9G         4.4G         4.4G


-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Pierre Neidhardt wrote on 10 Apr 2020 11:04

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:87eesvvhim.fsf@ambrevar.xyz

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (13 lines)> On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
>> > So compression saves me 26% ([69-51]/69), and deduplication saves me
>> > 62% ([180-69]/180).
>> 
>> Thanks for sharing!
>> zstd might give better results.  Any reason you chose lzo over zstd?
>> 
>
> My machine is about 10 years old so I was more concerned than normal
> about the CPU usage. If lz4 was an option I would've gone with that, but
> according to the Arch wiki or some other locations lzo was basically the
> fastest option.

I've tried zstd on an AMD Athlon II X4 635 (2010): it's perfectly

smooth, can't notice any performance drop. In fact, I wonder if it's

not even faster than before, but it's hard to measure.

Note that Arch Wiki tends to be on the conservative side when it comes

to performance. I would not use it as a reference for the general case:

it may guide users to sacrifice convenience and features over

unnoticeable performance gains.

Cheers!

Pierre Neidhardt

https://ambrevar.xyz/

Maxim Cournoyer wrote on 14 Apr 2020 04:20

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87h7xmiz96.fsf@gmail.com

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (19 lines)> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
>>> > So compression saves me 26% ([69-51]/69), and deduplication saves me
>>> > 62% ([180-69]/180).
>>> 
>>> Thanks for sharing!
>>> zstd might give better results.  Any reason you chose lzo over zstd?
>>> 
>>
>> My machine is about 10 years old so I was more concerned than normal
>> about the CPU usage. If lz4 was an option I would've gone with that, but
>> according to the Arch wiki or some other locations lzo was basically the
>> fastest option.
>
> I've tried zstd on an AMD Athlon II X4 635 (2010): it's perfectly
> smooth, can't notice any performance drop.  In fact, I wonder if it's
> not even faster than before, but it's hard to measure.

I've also tried zstd (default level, 3) on a Intel Q6700 desktop (2007).

I don't see any CPU spike caused by the compression. It's operating

quite smoothly actually, and gives me the following space savings:

Toggle snippet (8 lines)

sudo compsize -x /gnu/store

Processed 1613194 files, 402674 regular extents (1163093 refs), 665696 inline.

Type Perc Disk Usage Uncompressed Referenced

TOTAL 60% 15G 25G 63G

none 100% 10G 10G 28G

zstd 33% 5.1G 15G 34G

Recently on the #btrfs channel someone suggested to use the Btrfs option
compress-force=zstd rather than compress=zstd, the reason being that
zstd has its own algorithm to determine if it should compress a file or
not, and that this is faster than what Btrfs does on its own when trying
to test for compressibility.

Another suggestion was to use space_cache=v2 (it defaults to v1).  This
is supposedly more efficient at managing the free space pool on large
drives (TB and up).

Maxim

Pierre Neidhardt wrote on 14 Apr 2020 08:53

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87lfmya78b.fsf@ambrevar.xyz

Thanks for sharing those tips!

Pierre Neidhardt

https://ambrevar.xyz/

Pierre Neidhardt wrote on 2 May 2020 15:29

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 40236@debbugs.gnu.org)

Message-ID:87o8r6cvn1.fsf@ambrevar.xyz

Hi Maxim,

Out of curiosity, is #37305 (https://issues.guix.info/issue/37305#16)

going to make it on the next core-updates merge or is it too late?

Cheers!

Pierre Neidhardt

https://ambrevar.xyz/

Marius Bakke wrote on 2 May 2020 15:50

Recipients:(address . 40236@debbugs.gnu.org)

Message-ID:87zhaq30p8.fsf@devup.no

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (5 lines)

> Hi Maxim,

> Out of curiosity, is #37305 (https://issues.guix.info/issue/37305#16)

> going to make it on the next core-updates merge or is it too late?

Why would it have to go through 'core-updates'?

Pierre Neidhardt wrote on 2 May 2020 15:58

Recipients:(address . 40236@debbugs.gnu.org)

Message-ID:87ftcicubb.fsf@ambrevar.xyz

I assumed it rebuilt the world, maybe I remembered wrong.

Pierre Neidhardt

https://ambrevar.xyz/

Maxim Cournoyer wrote on 2 May 2020 21:03

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87pnbm40s5.fsf@gmail.com

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (2 lines)

> I assumed it rebuilt the world, maybe I remembered wrong.

No, it doesn't :-).  After it gets duly reviewed, it can go to the
master branch.

I've recently had the "chance" to experience a degraded RAID1 array,
which made me reconsider some of the simplifications that were made to
the original patch series (such as removing the option for users to pass
root options to the initrd -- this would be useful to input the
"degraded" mount option required to boot a degraded array, if your
system is no longer in a capacity to be reconfigured). So I may want to
reintroduce those changes, although this could also be done at a later
point as well.

Maxim

Pierre Neidhardt wrote on 3 May 2020 09:01

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87sgghbiyn.fsf@ambrevar.xyz

OK, thanks for the details.

What is "degraded RAID1"?

I'll see if I can test. the patch on my Btrfs system.

Anything else I can do to help?

Pierre Neidhardt

https://ambrevar.xyz/

Maxim Cournoyer wrote on 4 May 2020 17:22

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87pnbj3esx.fsf@gmail.com

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (4 lines)

> OK, thanks for the details.

> What is "degraded RAID1"?

Degraded RAID1 would be a RAID1 array (mirror disks) in which at least
one drive has failed.  As long as at least a single healthy disk
remains, the RAID1 array should remain functional but requires to be
mounted with the 'degraded' mount option (this is done to flag that
there is a serious problem that needs to be addressed, e.g., replace the
faulty drive).

Toggle quote (3 lines)

> I'll see if I can test. the patch on my Btrfs system.

> Anything else I can do to help?

Testing on a live system should provide extra confidence that the patch
work as advertised.  I have done so on my machines for a while (there's
also a system test), but I'd like to know if it works for someone else
:-).

Maxim

Pierre Neidhardt wrote on 31 May 2020 09:39

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87y2p83650.fsf@ambrevar.xyz

Now that 37305 is merged, we can revisit this issue.

1. This patch only includes a documentation update.

2. We could make Btrfs the default in the graphical installer.

3. We would probably need to update the tests, at least for the latter.

As mentioned above, I think it's safe to enable Btrfs without subvolume,

with Zstd compression.

For subvolumes, we would need to implement the corresponding tests.

Thoughts?

Pierre Neidhardt

https://ambrevar.xyz/

Efraim Flashner wrote on 31 May 2020 09:55

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:20200531075532.GD7397@E5400

On Sun, May 31, 2020 at 09:39:23AM +0200, Pierre Neidhardt wrote:

Toggle quote (13 lines)> Now that 37305 is merged, we can revisit this issue.
> 
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
> 3. We would probably need to update the tests, at least for the latter.
> 
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
> 
> For subvolumes, we would need to implement the corresponding tests.
> 
> Thoughts?

My main concern is the possibility of data loss. I don't know how much

is FUD and how much is legit so to me it seems safest to see what

OpenSuSE uses and to mimic that a bit (in terms of not using most of the

features available).

That said, I use btrfs with no subvolumes with compression turned on and

I'm pretty happy with that.

Two more things:

/var/guix/db should probably have CoW disabled, as should /tmp

would the deduplication of btrfs be "better" than the deduplication from

the daemon?

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

Ludovic Courtès wrote on 31 May 2020 23:07

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87ftbfreyr.fsf@gnu.org

Hi Pierre,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

Toggle quote (13 lines)> Now that 37305 is merged, we can revisit this issue.
>
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
> 3. We would probably need to update the tests, at least for the latter.
>
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
>
> For subvolumes, we would need to implement the corresponding tests.
>
> Thoughts?

Like I wrote earlier, ext4 is better tested configuration. For the

people who take care of the release, believe me, it’s reassuring to have

one less moving piece.

So the question is not whether Btrfs is “better”, it’s more about the

confidence and peace of mind of those responsible for the release and

subsequent maintenance.

Now, contributions that make Btrfs well-supported in the installer (is

anything missing there?) and in system tests would be welcome I guess.

In the manual, it’s also fine IMO to mention something like:

Some file systems, such as Btrfs, support compression, which is

reported to nicely complement file deduplication that the daemon

performs independently of the file system (@pxref{Invoking

guix-daemon}).

So a hint for savvy people rather than a recommendation.

Ludo’.

Maxim Cournoyer wrote on 1 Jun 2020 06:21

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:878sh75scf.fsf@gmail.com

Hello!

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (19 lines)> On Sun, May 31, 2020 at 09:39:23AM +0200, Pierre Neidhardt wrote:
>> Now that 37305 is merged, we can revisit this issue.
>>
>> 1. This patch only includes a documentation update.
>> 2. We could make Btrfs the default in the graphical installer.
>> 3. We would probably need to update the tests, at least for the latter.
>>
>> As mentioned above, I think it's safe to enable Btrfs without subvolume,
>> with Zstd compression.
>>
>> For subvolumes, we would need to implement the corresponding tests.
>>
>> Thoughts?
>
> My main concern is the possibility of data loss. I don't know how much
> is FUD and how much is legit so to me it seems safest to see what
> OpenSuSE uses and to mimic that a bit (in terms of not using most of the
> features available).

Except for Btrfs RAID 5 and 6, which are still known to have issues and
are considered experimental, I'd say mostly FUD, although there were
some bugs in Linux 5.1 and 5.2.  As you noted, (Open)SUSE defaults to
Btrfs, and companies such as Synology ship network storage products with
Btrfs on.  If it was that bad, nobody would buy those, and the companies
would stop proudly advertising their use of Btrfs.

Toggle quote (6 lines)

> That said, I use btrfs with no subvolumes with compression turned on and

> I'm pretty happy with that.

> Two more things:

> /var/guix/db should probably have CoW disabled, as should /tmp

I haven't bothered and my system seems to be doing OK.  When I asked in
#btrfs, people told me to keep CoW unless I was really sure it was a
problem (i.e., run benchmarks), as it implies loosing the checksum
validation and compression.  The command 'man 5 btrfs' also states that
"Updates in-place improve performance for workloads that do frequent
overwrites, at the cost of potential partial writes, in case the write
is interrupted (system crash, device failure).", which doesn't sound
safe to do for something as important as /var/guix/db.

Toggle quote (3 lines)

> would the deduplication of btrfs be "better" than the deduplication from

> the daemon?

On my system (with zstd compression), compsize -x /gnu/store suggests

a resounding yes:

Toggle snippet (8 lines)

sudo compsize -x /gnu/store

Processed 3479664 files, 954748 regular extents (3002677 refs), 1451082 inline.

Type Perc Disk Usage Uncompressed Referenced

TOTAL 57% 51G 88G 217G

none 100% 32G 32G 81G

zstd 33% 18G 56G 135G

The delta between the Uncompressed and Referenced column is attributed

to the deduplication done by Btrfs, and provides massive space savings

in my case (this is just for /gnu/store).

I'd need 217 GiB over a traditional fs such as EXT4 to hold my current

store, while an uncompressed Btrfs partition would use only 88 GiB.

With zstd compression, it's down to 51 GiB, or less that a quarter of

what would have been required using EXT4.

Maxim

Maxim Cournoyer wrote on 1 Jun 2020 06:49

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:871rmz5r1b.fsf@gmail.com

Hello Pierre,

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (5 lines)

> Now that 37305 is merged, we can revisit this issue.

> 1. This patch only includes a documentation update.

> 2. We could make Btrfs the default in the graphical installer.

Is it already possible to choose Btrfs in the graphical installer? I

guess so, but I haven't tried.

Toggle quote (5 lines)

> 3. We would probably need to update the tests, at least for the

> latter.

> As mentioned above, I think it's safe to enable Btrfs without subvolume,

> with Zstd compression.

My personal experience also tells me so, but it is true that it is still

newer and more exotic than EXT4.

Toggle quote (2 lines)

> For subvolumes, we would need to implement the corresponding tests.

What kind of tests are you referring to? There is a test in (gnu tests

install) that install root on a Btrfs subvolume and boot from it, but

I'm pretty sure the graphical installer doesn't have support for

configuring a Btrfs subvolumes layout.

Toggle quote (2 lines)

> Thoughts?

I'm in favor of encouraging the use of Btrfs with compression given that
Guix puts more stress on storage space compared to traditional distros,
and that I've had only positive experiences with Btrfs in the last year
and half or so that I've used it on my machines.

Maxim

Maxim Cournoyer wrote on 1 Jun 2020 07:03

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87wo4r4bu6.fsf@gmail.com

Hello Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (21 lines)> Hi Pierre,
>
> Pierre Neidhardt <mail@ambrevar.xyz> skribis:
>
>> Now that 37305 is merged, we can revisit this issue.
>>
>> 1. This patch only includes a documentation update.
>> 2. We could make Btrfs the default in the graphical installer.
>> 3. We would probably need to update the tests, at least for the latter.
>>
>> As mentioned above, I think it's safe to enable Btrfs without subvolume,
>> with Zstd compression.
>>
>> For subvolumes, we would need to implement the corresponding tests.
>>
>> Thoughts?
>
> Like I wrote earlier, ext4 is better tested configuration.  For the
> people who take care of the release, believe me, it’s reassuring to have
> one less moving piece.

I believe you! :-)

Toggle quote (7 lines)

> So the question is not whether Btrfs is “better”, it’s more about the

> confidence and peace of mind of those responsible for the release and

> subsequent maintenance.

> Now, contributions that make Btrfs well-supported in the installer (is

> anything missing there?) and in system tests would be welcome I guess.

I don't know about the installer (I guess there's no support for

configuring Btrfs subvolumes graphically), but for system tests we're

already covered (we have a general Btrfs installation test, as well as

one on a subvolume, even a Btrfs RAID one in patches).

Toggle quote (9 lines)

> In the manual, it’s also fine IMO to mention something like:

> Some file systems, such as Btrfs, support compression, which is

> reported to nicely complement file deduplication that the daemon

> performs independently of the file system (@pxref{Invoking

> guix-daemon}).

> So a hint for savvy people rather than a recommendation.

That's reasonable as a first step in democratizing Btrfs usage a bit,

without taking any risk, if that's better for your peace of mind :-).

Maxim

Efraim Flashner wrote on 1 Jun 2020 08:16

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:20200601061603.GF7397@E5400

Toggle quote (13 lines)> >
> > Two more things:
> > /var/guix/db should probably have CoW disabled, as should /tmp
> 
> I haven't bothered and my system seems to be doing OK.  When I asked in
> #btrfs, people told me to keep CoW unless I was really sure it was a
> problem (i.e., run benchmarks), as it implies loosing the checksum
> validation and compression.  The command 'man 5 btrfs' also states that
> "Updates in-place improve performance for workloads that do frequent
> overwrites, at the cost of potential partial writes, in case the write
> is interrupted (system crash, device failure).", which doesn't sound
> safe to do for something as important as /var/guix/db.

Fair enough. I had heard that the CoW stuff wasn't great for databases.

I thought Leo ran into some issues with CoW on /tmp with the syncthing

tests.

Toggle quote (24 lines)> > would the deduplication of btrfs be "better" than the deduplication from
> > the daemon?
> 
> On my system (with zstd compression), compsize -x /gnu/store suggests
> a resounding yes:
> 
> --8<---------------cut here---------------start------------->8---
> sudo compsize -x /gnu/store
> Processed 3479664 files, 954748 regular extents (3002677 refs), 1451082 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL       57%       51G          88G         217G
> none       100%       32G          32G          81G
> zstd        33%       18G          56G         135G
> --8<---------------cut here---------------end--------------->8---
> 
> The delta between the Uncompressed and Referenced column is attributed
> to the deduplication done by Btrfs, and provides massive space savings
> in my case (this is just for /gnu/store).
> 
> I'd need 217 GiB over a traditional fs such as EXT4 to hold my current
> store, while an uncompressed Btrfs partition would use only 88 GiB.
> With zstd compression, it's down to 51 GiB, or less that a quarter of
> what would have been required using EXT4.

I always understood that as with compression you're using 51G instead of
88G, and because of deduplication from the daemon it would only be 88G
instead of 217G. I took the numbers from 'none' to mean that the daemon
itself already did a lot of deduplication.

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Pierre Neidhardt wrote on 1 Jun 2020 09:48

Recipients:

Message-ID:87ftbfcjkn.fsf@ambrevar.xyz

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (5 lines)

> I always understood that as with compression you're using 51G instead of

> 88G, and because of deduplication from the daemon it would only be 88G

> instead of 217G. I took the numbers from 'none' to mean that the daemon

> itself already did a lot of deduplication.

That's what I understood as well.

Pierre Neidhardt

https://ambrevar.xyz/

Maxim Cournoyer wrote on 1 Jun 2020 20:29

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87h7vuod0x.fsf@gmail.com

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

Toggle quote (9 lines)

> Efraim Flashner <efraim@flashner.co.il> writes:

>> I always understood that as with compression you're using 51G instead of

>> 88G, and because of deduplication from the daemon it would only be 88G

>> instead of 217G. I took the numbers from 'none' to mean that the daemon

>> itself already did a lot of deduplication.

> That's what I understood as well.

tl;dr Your understanding is correct.

I was under the impression that hard links shouldn't inflate the numbers

reported under the "Referenced" column.

I just asked over #btrfs and while hard links shouldn't inflate those

numbers (as hard links aren't reflinks), they currently do. It's a bug.

Maxim

Pierre Neidhardt wrote on 2 Jun 2020 15:37

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87y2p5twqn.fsf@ambrevar.xyz

Toggle quote (5 lines)

> Some file systems, such as Btrfs, support compression, which is

> reported to nicely complement file deduplication that the daemon

> performs independently of the file system (@pxref{Invoking

> guix-daemon}).

Looks good to me! Do you want to proceed and add it then?

Pierre Neidhardt

https://ambrevar.xyz/

Ludovic Courtès wrote on 3 Jun 2020 22:00

Recipients:(name . Pierre Neidhardt)(address . mail@ambrevar.xyz)

Message-ID:87y2p3kjgl.fsf@gnu.org

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

Toggle quote (7 lines)

>> Some file systems, such as Btrfs, support compression, which is

>> reported to nicely complement file deduplication that the daemon

>> performs independently of the file system (@pxref{Invoking

>> guix-daemon}).

> Looks good to me! Do you want to proceed and add it then?

I pushed something in b6c445931e85c07286a79bc9c80fdc956d58b1e2.

I suggest we revisit the question of the default/recommended file system

in several months, a year or so.

Thanks,

Ludo’.

Closed

Pierre Neidhardt wrote on 4 Jun 2020 11:17

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87k10nb36t.fsf@ambrevar.xyz

Thanks!

Pierre Neidhardt

https://ambrevar.xyz/

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 40236@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date