From debbugs-submit-bounces@debbugs.gnu.org Fri May 01 23:04:34 2020 Received: (at submit) by debbugs.gnu.org; 2 May 2020 03:04:34 +0000 Received: from localhost ([127.0.0.1]:51041 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jUiS9-0007dV-Ex for submit@debbugs.gnu.org; Fri, 01 May 2020 23:04:34 -0400 Received: from lists.gnu.org ([209.51.188.17]:59110) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jUiS7-0007dN-TX for submit@debbugs.gnu.org; Fri, 01 May 2020 23:04:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47036) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jUiS6-0002G7-Hn for guix-patches@gnu.org; Fri, 01 May 2020 23:04:31 -0400 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jUiOa-0008L6-MX for guix-patches@gnu.org; Fri, 01 May 2020 23:04:29 -0400 Received: from mx1.riseup.net ([198.252.153.129]:58070) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jUiOZ-0007zX-PE for guix-patches@gnu.org; Fri, 01 May 2020 23:00:52 -0400 Received: from bell.riseup.net (unknown [10.0.1.178]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 49DYlX5vv4zFccb for ; Fri, 1 May 2020 20:00:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1588388441; bh=BeKmtfrjV2tFg7Dldp3amJZzJFiSs3+61IoXqawrisU=; h=Date:From:To:Subject:From; b=VOo3zNvSEeYhJYvtVqFVAvNUwC8gC6+vK3iSH7rhnnTg7memw5L8slOOp7VCHkQiG R8lEh1tmtpwtIWEJys5nfvWecZQ0AEiZFBOmysfeQtheNCzNiksSaEFbRzvD4ppr2H DqQzRjRq412/c6wNpnceuJWuCU+JXap+r/F44uVE= X-Riseup-User-ID: E19BC2A11B369688847C8513F3F431CCDF523C2BFE34ED334434A1071BFCE40F Received: from [127.0.0.1] (localhost [127.0.0.1]) by bell.riseup.net (Postfix) with ESMTPSA id 49DYlX0xLmzJqZf for ; Fri, 1 May 2020 20:00:39 -0700 (PDT) Date: Sat, 2 May 2020 02:48:00 +0200 From: raingloom To: guix-patches@gnu.org Subject: [PATCH] (incomplete) F2FS support Message-ID: <20200502024800.4b741809@riseup.net> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/HEw+PS8IGtrMD.IdzsFAr_h" Received-SPF: pass client-ip=198.252.153.129; envelope-from=raingloom@riseup.net; helo=mx1.riseup.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/01 23:00:41 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Received-From: 198.252.153.129 X-Spam-Score: -1.4 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.4 (--) --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline I've been working on this on and off for a while, but can't finish the last steps, so I'm sending it as is. What works: installing to and booting from a partition specified by UUID What seemed to work but then didn't: installing to a partition specified by its label. i tried to do the math, but for some reason it still can't find it. I'm pretty sure it worked at some point. What definitely doesn't work: file system check on boot. I tried and tried to create a statically linked fsck and I've concluded that it's impossible and that I hate linkers. Someone else can figure it out, because I've given up. This means that the init will look for it but won't find it. You will be thrown into a rescue shell, but if you exit it with `,q`, the init process continues without any issue. I used this to set Guix up on my desktop machine on an SSD and other than the issues I've mentioned, it seems to be working fine. I'm hoping someone will pick up the work where I left it. If not, I may revisit it once I've learned enough about how f2fs-tools is built. But I can't say that I'm eager to devote more of my life to that. About the patches: 0001-0005 are what you need for a working system, I included the rest to help anyone who wishes to continue the work. --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-file-system-Add-support-for-F2FS.patch From 383d8536b748786b2072f5f61e1e24d0c9b4585d Mon Sep 17 00:00:00 2001 From: raingloom Date: Fri, 3 Apr 2020 02:32:23 +0200 Subject: [PATCH 1/8] file-system: Add support for F2FS. * gnu/build/file-systems.scm (%f2fs-endianness): New syntax. (f2fs-superblock?, read-f2fs-superblock, f2fs-superblock-uuid) (f2fs-superblock-volume-name, check-f2fs-file-system): New procedures. (%partition-label-readers, %partition-uuid-readers, check-file-system): Register them. --- gnu/build/file-systems.scm | 61 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 2 deletions(-) diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm index 902563b219..3742252421 100644 --- a/gnu/build/file-systems.scm +++ b/gnu/build/file-systems.scm @@ -336,6 +336,58 @@ if DEVICE does not contain a JFS file system." (2 'reboot-required) (_ 'fatal-error))) + +;;; +;;; F2FS (Flash-Friendly File System) +;;; + +;;; https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git/tree/include/linux/f2fs_fs.h +;;; (but using xxd proved to be simpler) + +(define-syntax %f2fs-endianness + ;; Endianness of F2FS file systems + (identifier-syntax (endianness little))) + +;; F2FS actually stores two adjacent copies of the superblock. +;; should we read both? +(define (f2fs-superblock? sblock) + "Return #t when SBLOCK is an F2FS superblock." + (let ((magic (bytevector-u32-ref sblock 0 %f2fs-endianness))) + (= magic #xF2F52010))) + +(define (read-f2fs-superblock device) + "Return the raw contents of DEVICE's F2FS superblock as a bytevector, or #f +if DEVICE does not contain an F2FS file system." + (read-superblock device + ;; offset of magic in first copy + #x400 + ;; difference between magic of second + ;; and first copies + (- #x1400 #x400) + f2fs-superblock?)) + +(define (f2fs-superblock-uuid sblock) + "Return the UUID of F2FS superblock SBLOCK as a 16-byte bytevector." + (sub-bytevector sblock + (- (+ #x460 12) + ;; subtract superblock offset + #x400) + 16)) + +(define (f2fs-superblock-volume-name sblock) + "Return the volume name of SBLOCK as a string of at most 16 characters, or +#f if SBLOCK has no volume name." + (utf16->string (sub-bytevector sblock (- (+ #x470 12) #x400) 512) %f2fs-endianness)) + +(define (check-f2fs-file-system device) + "Return the health of a F2FS file system on DEVICE." + (match (status:exit-val + (system* "fsck.f2fs" "-p" device)) + ;; 0 and -1 are the only two possibilities + ;; (according to the manpage) + (0 'pass) + (_ 'fatal-error))) + ;;; ;;; LUKS encrypted devices. @@ -472,7 +524,9 @@ partition field reader that returned a value." (partition-field-reader read-fat16-superblock fat16-superblock-volume-name) (partition-field-reader read-jfs-superblock - jfs-superblock-volume-name))) + jfs-superblock-volume-name) + (partition-field-reader read-f2fs-superblock + f2fs-superblock-volume-name))) (define %partition-uuid-readers (list (partition-field-reader read-iso9660-superblock @@ -486,7 +540,9 @@ partition field reader that returned a value." (partition-field-reader read-fat16-superblock fat16-superblock-uuid) (partition-field-reader read-jfs-superblock - jfs-superblock-uuid))) + jfs-superblock-uuid) + (partition-field-reader read-f2fs-superblock + f2fs-superblock-uuid))) (define read-partition-label (cut read-partition-field <> %partition-label-readers)) @@ -582,6 +638,7 @@ were found." ((string-prefix? "btrfs" type) check-btrfs-file-system) ((string-suffix? "fat" type) check-fat-file-system) ((string-prefix? "jfs" type) check-jfs-file-system) + ((string-prefix? "f2fs" type) check-f2fs-file-system) ((string-prefix? "nfs" type) (const 'pass)) (else #f))) -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0002-gnu-Add-f2fs-tools-static.patch From 26208230e82cad199ec45707d156e9c23d43d04c Mon Sep 17 00:00:00 2001 From: raingloom Date: Fri, 3 Apr 2020 03:59:52 +0200 Subject: [PATCH 2/8] gnu: Add f2fs-tools-static * gnu/packages/file-systems.scm (f2fs-tools/static): New public variable. --- gnu/packages/linux.scm | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 701df764cd..fc8afbb90c 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -4469,6 +4469,15 @@ disks and SD cards. This package provides the userland utilities.") (inputs `(("libuuid" ,util-linux))))) +(define-public f2fs-tools/static + (static-package + (package + (inherit f2fs-tools) + (name "f2fs-tools-static") + (inputs + `(("libuuid:static" ,util-linux "static") + ,@(package-inputs f2fs-tools)))))) + (define-public freefall (package (name "freefall") -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0003-gnu-Add-f2fs-fsck-static.patch From 6fff1095c768b5863e6de45bb9b90aead77e121f Mon Sep 17 00:00:00 2001 From: raingloom Date: Fri, 3 Apr 2020 04:18:26 +0200 Subject: [PATCH 3/8] gnu: Add f2fs-fsck-static * gnu/packages/linux.scm (f2fs-fsck/static): New public variable. --- gnu/packages/linux.scm | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index fc8afbb90c..5b752eeef5 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -4478,6 +4478,38 @@ disks and SD cards. This package provides the userland utilities.") `(("libuuid:static" ,util-linux "static") ,@(package-inputs f2fs-tools)))))) +(define-public f2fs-fsck/static + (package + (name "f2fs-fsck-static") + (version (package-version f2fs-tools/static)) + (source #f) + (build-system trivial-build-system) + (arguments + `(#:modules ((guix build utils)) + #:builder + (begin + (use-modules (guix build utils) + (ice-9 ftw) + (srfi srfi-26)) + (let* ((f2fs-tools (assoc-ref %build-inputs "f2fs-tools-static")) + (fsck "fsck.f2fs") + (out (assoc-ref %outputs "out")) + (sbin (string-append out "/sbin"))) + (mkdir-p sbin) + (with-directory-excursion sbin + (install-file (string-append f2fs-tools "/sbin/" fsck) + ".") + (remove-store-references fsck) + (chmod fsck #o555)) + #t)))) + (inputs + `(("f2fs-tools-static" ,f2fs-tools/static))) + (home-page (package-home-page f2fs-tools/static)) + (synopsis "Statically-linked fsck.f2fs command from f2fs-tools") + (description "This package provides statically-linked fsck.f2fs command taken +from the f2fs-tools package. It is meant to be used in initrds.") + (license (package-license f2fs-tools/static)))) + (define-public freefall (package (name "freefall") -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0004-linux-initrd-Add-support-for-F2FS.patch From 0ea3a259b2f89b7a6a6962156ffaffec9d389c99 Mon Sep 17 00:00:00 2001 From: raingloom Date: Fri, 3 Apr 2020 07:00:41 +0200 Subject: [PATCH 4/8] =?UTF-8?q?linux-initrd:=20Add=20support=20for=20F2FS.?= =?UTF-8?q?=20*=20gnu/system/linux-initrd.scm=20(file-system-packages):=20?= =?UTF-8?q?Add=20f2fs-fsck/static.=20(file-system-type-modules):=20Add=20?= =?UTF-8?q?=E2=80=98f2fs=E2=80=99=20module.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- gnu/system/linux-initrd.scm | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm index dcc9b6b937..6a1840dbf6 100644 --- a/gnu/system/linux-initrd.scm +++ b/gnu/system/linux-initrd.scm @@ -245,6 +245,9 @@ FILE-SYSTEMS." '()) ,@(if (find (file-system-type-predicate "jfs") file-systems) (list jfs_fsck/static) + '()) + ,@(if (find (file-system-type-predicate "f2fs") file-systems) + (list f2fs-fsck/static) '()))) (define-syntax vhash ;TODO: factorize @@ -275,6 +278,7 @@ FILE-SYSTEMS." ("btrfs" => '("btrfs")) ("iso9660" => '("isofs")) ("jfs" => '("jfs")) + ("f2fs" => '("f2fs" "crc32_generic")) (else '()))) (define (file-system-modules file-systems) -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0005-disable-fsck.f2fs.patch From 4b8156682cae49df79ef992a55f951256163a6fa Mon Sep 17 00:00:00 2001 From: raingloom Date: Fri, 17 Apr 2020 00:51:28 +0200 Subject: [PATCH 5/8] disable fsck.f2fs --- gnu/system/linux-initrd.scm | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm index 6a1840dbf6..349c143298 100644 --- a/gnu/system/linux-initrd.scm +++ b/gnu/system/linux-initrd.scm @@ -246,9 +246,10 @@ FILE-SYSTEMS." ,@(if (find (file-system-type-predicate "jfs") file-systems) (list jfs_fsck/static) '()) - ,@(if (find (file-system-type-predicate "f2fs") file-systems) - (list f2fs-fsck/static) - '()))) + ;; ,@(if (find (file-system-type-predicate "f2fs") file-systems) + ;; (list f2fs-fsck/static) + ;; '()) + )) (define-syntax vhash ;TODO: factorize (syntax-rules (=>) -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0006-gnu-f2fs-tools-static-try-only-using-static-util-lin.patch From 3faa9923e70416d27eba822d423c5a5007c6df3d Mon Sep 17 00:00:00 2001 From: raingloom Date: Sun, 19 Apr 2020 12:30:55 +0200 Subject: [PATCH 6/8] gnu: f2fs-tools-static: try only using static util-linux --- gnu/packages/linux.scm | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index ab74380dd6..db7cf03b3e 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -4571,8 +4571,7 @@ disks and SD cards. This package provides the userland utilities.") (inherit f2fs-tools) (name "f2fs-tools-static") (inputs - `(("libuuid:static" ,util-linux "static") - ,@(package-inputs f2fs-tools)))))) + `(("libuuid:static" ,util-linux "static")))))) (define-public f2fs-fsck/static (package -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0007-add-temporary-debugger-breakpoint-in-f2fs-tools-stat.patch From 4e01a024dd8c9de3affb6a0f628fce7325ef07cc Mon Sep 17 00:00:00 2001 From: raingloom Date: Mon, 20 Apr 2020 14:57:33 +0200 Subject: [PATCH 7/8] add temporary debugger breakpoint in f2fs-tools-static --- gnu/packages/linux.scm | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 9b56e84477..c0fac7b6e5 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -4570,8 +4570,18 @@ disks and SD cards. This package provides the userland utilities.") (package (inherit f2fs-tools) (name "f2fs-tools-static") + (arguments + '(#:make-flags (list "CFLAGS=-v") + #:phases + (modify-phases %standard-phases + (add-before 'install 'fail + (lambda _ + (error "fail on purpose")))))) (inputs - `(("libuuid:static" ,util-linux "static")))))) + `(,@(package-inputs f2fs-tools) + ("libuuid:static" ,util-linux "static") + ("glibc:static" ,glibc "static") + ))))) (define-public f2fs-fsck/static (package -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0008-trying-the-original-approach-glibc-static-adding-a-b.patch From ab44403a7563fe6ca3edf4d1b7929a8d7d12429a Mon Sep 17 00:00:00 2001 From: raingloom Date: Thu, 23 Apr 2020 04:14:37 +0200 Subject: [PATCH 8/8] trying the original approach + glibc static + adding a "breakpoint" --- gnu/packages/linux.scm | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index c0fac7b6e5..801ffbdf88 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -4571,17 +4571,18 @@ disks and SD cards. This package provides the userland utilities.") (inherit f2fs-tools) (name "f2fs-tools-static") (arguments - '(#:make-flags (list "CFLAGS=-v") - #:phases - (modify-phases %standard-phases - (add-before 'install 'fail - (lambda _ - (error "fail on purpose")))))) + (substitute-keyword-arguments (package-arguments f2fs-tools) + ((#:make-flags flags) + `(cons* "CFLAGS=-v" ,flags)) + ((#:phases phases) + `(modify-phases ,phases + (add-before 'install 'fail + (lambda _ + (error "fail on purpose"))))))) (inputs - `(,@(package-inputs f2fs-tools) - ("libuuid:static" ,util-linux "static") + `(("libuuid:static" ,util-linux "static") ("glibc:static" ,glibc "static") - ))))) + ,@(package-inputs f2fs-tools)))))) (define-public f2fs-fsck/static (package -- 2.26.2 --MP_/HEw+PS8IGtrMD.IdzsFAr_h--