guix pull impossible after rebasing a local repository

DoneSubmitted by John Soo.
Details
4 participants
  • Arne Babenhauserheide
  • John Soo
  • Ludovic Courtès
  • zimoun
Owner
unassigned
Severity
normal
J
J
John Soo wrote on 29 May 18:38 +0200
(address . bug-guix@gnu.org)
87zh9qofw7.fsf@asu.edu
Hi Guix,
I use a local git repo with branch that I specify in channels.scm.My usual workflow is:
1. rebase on origin2. guix pull
This stopped working with the following error:
Updating channel 'guix' from Git repository at 'file:///home/john/projects/guix/.git'...guix pull: error: aborting update of channel 'guix' to commit 1444040933ac35b967720288dc30ed70e5481ed3, which is not a descendant of 57518fc7bf1efc899c0dabaa76685a319661f8e4hint: This could indicate that the channel has been tampered with and is trying toforce a roll-back, preventing you from getting the latest updates. If you thinkthis is not the case, explicitly allow non-forward updates.
After removing $HOME/.cache/guix I get the following error:guix pull: error: Git error: object not found - no match for id (57518fc7bf1efc899c0dabaa76685a319661f8e4)
Many thanks,
John
J
J
John Soo wrote on 29 May 18:41 +0200
Re: bug#41604: Acknowledgement (guix pull impossible after rebasing a local repository)
(address . 41604@debbugs.gnu.org)
87v9keofrb.fsf@asu.edu
One other note I forgot to mention. Using guix pull --allow-downgradesgives the same error.
Z
Z
zimoun wrote on 29 May 18:52 +0200
Re: bug#41604: guix pull impossible after rebasing a local repository
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604@debbugs.gnu.org)
CAJ3okZ34_5OLOmq514fqBCdOatk_yBrjyu6mcHTJwZome12E1w@mail.gmail.com
Hi John,
On Fri, 29 May 2020 at 18:39, John Soo <jsoo1@asu.edu> wrote:
Toggle quote (9 lines)> Updating channel 'guix' from Git repository at 'file:///home/john/projects/guix/.git'...> guix pull: error: aborting update of channel 'guix' to commit 1444040933ac35b967720288dc30ed70e5481ed3, which is not a descendant of 57518fc7bf1efc899c0dabaa76685a319661f8e4> hint: This could indicate that the channel has been tampered with and is trying to> force a roll-back, preventing you from getting the latest updates. If you think> this is not the case, explicitly allow non-forward updates.>> After removing $HOME/.cache/guix I get the following error:> guix pull: error: Git error: object not found - no match for id (57518fc7bf1efc899c0dabaa76685a319661f8e4)
Have you used '--allow-downgrades' before or after the big remove?And your current 'guix describe' is 57518fc7, right?

All the best,simon
J
J
John Soo wrote on 29 May 19:10 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 41604@debbugs.gnu.org)
87r1v2oefd.fsf@asu.edu
Hi there,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (7 lines)> Hi John,>> On Fri, 29 May 2020 at 18:39, John Soo <jsoo1@asu.edu> wrote:>> Have you used '--allow-downgrades' before or after the big remove?> And your current 'guix describe' is 57518fc7, right?
I used it after the cache removal unfortunately.
Guix describe has 57518fc7bf1efc899c0dabaa76685a319661f8e4as the current comment.
Kindly,
John
Z
Z
zimoun wrote on 29 May 19:44 +0200
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604@debbugs.gnu.org)
CAJ3okZ0mzxymvejAjkhtSso975cxivuj8OpYxgUc1S6xyWaUjw@mail.gmail.com
On Fri, 29 May 2020 at 19:10, John Soo <jsoo1@asu.edu> wrote:
Toggle quote (3 lines)> Guix describe has 57518fc7bf1efc899c0dabaa76685a319661f8e4> as the current comment.
Maybe, you can try:
guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4--allow-downgrades
if 57518fc7 is really in your channel at ~/john/projects/guix/.git --I mean, you have replaced 'https://' by a local clone where you arecommitting stuff, right?

Otherwise, you could try:
guix pull --rollback guix pull --commit=57518fc7 guix pull --allow-downgrades

Well, I do not know if it helps...

Cheers,simon
J
J
John Soo wrote on 29 May 20:22 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 41604@debbugs.gnu.org)
87k10uob2f.fsf@asu.edu
Hello,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (9 lines)> Maybe, you can try:>> guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4> --allow-downgrades
> if 57518fc7 is really in your channel at ~/john/projects/guix/.git --> I mean, you have replaced 'https://' by a local clone where you are> committing stuff, right?
Ah, I think that is the problem. I want to keep my same patches whichare rebased on top of origin/master. The patch contents are the same butthe commit hash has changed to something else.
$ git rev-parse HEAD1444040933ac35b967720288dc30ed70e5481ed3
Commit 57518fc7bf1efc899c0dabaa76685a319661f8e4 no longer exists in mybranch's history:
$ git log --oneline --grep='57518'(nothing)
I specify the branch in channels.scm:
(list (channel (name 'guix) (url "file:///home/john/projects/guix/.git") (branch "my-master")) (channel (name 'private) (url "file:///home/john/projects/guix-channel/.git") (branch "master")))
Toggle quote (6 lines)> Otherwise, you could try:>> guix pull --rollback> guix pull --commit=57518fc7> guix pull --allow-downgrades
All of the following give the same error as reported:
guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4guix pull --commit=1444040933ac35b967720288dc30ed70e5481ed3guix pull --allow-downgrades --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4guix pull --allow-downgrades --commit=1444040933ac35b967720288dc30ed70e5481ed3guix pull --allow-downgrades
Using --rollback does not seem like an option
$ guix pull --rollbackguix pull: error: rollback: unrecognized option
Toggle quote (2 lines)> Well, I do not know if it helps...
Definitely! I am getting more familiar with the pull options. Thank you!
- John
Z
Z
zimoun wrote on 29 May 20:39 +0200
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604@debbugs.gnu.org)
CAJ3okZ0kUQRPcxx9oexdCHFdJnAV19KQQGXZuiikut6wrKT2Cw@mail.gmail.com
On Fri, 29 May 2020 at 20:22, John Soo <jsoo1@asu.edu> wrote:
Toggle quote (4 lines)> Ah, I think that is the problem. I want to keep my same patches which> are rebased on top of origin/master. The patch contents are the same but> the commit hash has changed to something else.
Hehe! Dangerous zone. :-)
Personally, I keep clean ~/.config/guix/current by always pulling fromorigin/master.Then I have, as you, a local clone where I rebase, commit etc. But Ionly pull to another profile than the default one, to avoid similarsituations as you currently are. ;-)
/path/to/what-i-mean/bin/guix pull --url=/pah/local/clone--branch=kikoo -p /path/to/next
and I have some channels files under ~/.config/guix/ to simply someregular, e.g.,
guix pull -C ~/.config/guix/extra.scm -p /path/to/extra /tmp/extra/bin/guix install foo -p /tmp/test
And so "guix pull" always works.Anyway! :-)


Toggle quote (16 lines)> > Otherwise, you could try:> >> > guix pull --rollback> > guix pull --commit=57518fc7> > guix pull --allow-downgrades>> All of the following give the same error as reported:>> guix pull --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4> guix pull --commit=1444040933ac35b967720288dc30ed70e5481ed3> guix pull --allow-downgrades --commit=57518fc7bf1efc899c0dabaa76685a319661f8e4> guix pull --allow-downgrades --commit=1444040933ac35b967720288dc30ed70e5481ed3> guix pull --allow-downgrades>> Using --rollback does not seem like an option
What do you mean? An option for you or an option of "guix pull"?Ah my bad, it is "--roll-back". The double "--roll-back" and"--rollback" options is only allowed with "guix package". Thank youfor the notification. :-)

Is "guix pull --commit=1444040933 --allow-downgrades" not working for you?

Cheers,simon
J
J
John Soo wrote on 30 May 04:26 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 41604@debbugs.gnu.org)
87ftbinoo7.fsf@asu.edu
Hello,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (2 lines)> Is "guix pull --commit=1444040933 --allow-downgrades" not working for you?
Yeah guix pull --commit=1444040933 --allow-downgrades failed.
Toggle quote (5 lines)> What do you mean? An option for you or an option of "guix pull"?> Ah my bad, it is "--roll-back". The double "--roll-back" and> "--rollback" options is only allowed with "guix package". Thank you> for the notification. :-)
Ah, nice! roll-back helped me work around the problem. Very helpful,thanks!
Toggle quote (2 lines)> Hehe! Dangerous zone. :-)
I think I found that out, haha!

Toggle quote (18 lines)> Personally, I keep clean ~/.config/guix/current by always pulling from> origin/master.> Then I have, as you, a local clone where I rebase, commit etc. But I> only pull to another profile than the default one, to avoid similar> situations as you currently are. ;-)>> /path/to/what-i-mean/bin/guix pull --url=/pah/local/clone> --branch=kikoo -p /path/to/next>> and I have some channels files under ~/.config/guix/ to simply some> regular, e.g.,>> guix pull -C ~/.config/guix/extra.scm -p /path/to/extra> /tmp/extra/bin/guix install foo -p /tmp/test>> And so "guix pull" always works.> Anyway! :-)
Nice! I like the idea of having a "next" profile.
I guess that makes me wonder what the desired specification is.There is a lot of problem space to explore. As a user I would want tobe able to take my local patches as "the real truth".
Because guix has a linear git history, that means the user needs toalways rebase. Often I have patches open for months that I am currentlytesting and working on. It would be convenient for me to be able to guixpull into my default user profile.
On the other hand, as you point out, using a hash that disappears fromthe git history is dangerous. The git history no longer tracks the guixpull history and then there may exist pulls in history that may never berecoverable.
There seems to be some existing support for the rebasing into thedefault profile since branches are allowed as references in the channelsconfiguration. Has the rebase use case been discussed before? Oh, also,history items can be deleted in other places with --delete-generationsand friends. I am not sure what to classify this problem, bug orsomething else.
What do you think?
- John
Z
Z
zimoun wrote on 30 May 12:45 +0200
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604@debbugs.gnu.org)
CAJ3okZ3jSYqUJw0tETFtnpb8wC6oopPnrPBYKLCbkftn=WKiMQ@mail.gmail.com
Hi John,
On Sat, 30 May 2020 at 04:26, John Soo <jsoo1@asu.edu> wrote:
Toggle quote (2 lines)> Yeah guix pull --commit=1444040933 --allow-downgrades failed.
Hum? with this message:
Toggle snippet (4 lines)guix pull: error: Git error: object not found - no match for id(57518fc7bf1efc899c0dabaa76685a319661f8e4)
?

Toggle quote (4 lines)> > Hehe! Dangerous zone. :-)>> I think I found that out, haha!
To be honest, I have tried to use the checkout~/.cache/guix/checkout/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq as local clone in order to rebase, worktree and apply patches. Bad,really bad idea! The discipline to avoid breakage is too high. ;-)

Toggle quote (21 lines)> I guess that makes me wonder what the desired specification is.> There is a lot of problem space to explore. As a user I would want to> be able to take my local patches as "the real truth".>> Because guix has a linear git history, that means the user needs to> always rebase. Often I have patches open for months that I am currently> testing and working on. It would be convenient for me to be able to guix> pull into my default user profile.>> On the other hand, as you point out, using a hash that disappears from> the git history is dangerous. The git history no longer tracks the guix> pull history and then there may exist pulls in history that may never be> recoverable.>> There seems to be some existing support for the rebasing into the> default profile since branches are allowed as references in the channels> configuration. Has the rebase use case been discussed before? Oh, also,> history items can be deleted in other places with --delete-generations> and friends. I am not sure what to classify this problem, bug or> something else.
I am not sure to see what is the desired specification. Well, I donot know if my workflow is the good one but let me describe how it isorganised.
The local (master) checkout is at ~/src/guix/guix/ and then I onlycheckout the branches using 'worktree' (thanks to Ricardo to show methe tips during a "Skill Share" session on Dec. 2018 :-)). Well,Magit helps a lot here.Basically all my branches live under ~/src/guix/wk/. And when I testpatches, I always create a new branch. Well, I do not often rebaseenough so I never remember the 'onto' direction so I always read thedoc for that. :-)
(The price to pay with worktree is that I "compile" all Guix reallyoften because it does not reuse .go files; even if I tried tosynchronize the .go files to avoid such annoyance but I have failed.)
Just 'guix' refers to the one living at ~/.config/guix/current becauseI always want that "guix install foo" refers to upstream. Well, Inever pull the latest commit fromhttps://git.savannah.gnu.org/git/guix.gitbecause the substitutes arealmost never available so I pull 2 weeks behind. Basically, I do:"git -C ~/.cache/guix/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwqpull" then
guix pull --commit=$(git -C ~/.cache/guix/<long-hash> \ --before=$(date --date='2 weeksago' +%Y-%m-%d) \ --format="%h" | head -n1)
And when I want to install something from one of the branches, it depends:
a- inside "guix environment guix" and ~/src/guix/wk/<name> ./pre-inst-env guix install foo -p /tmp/profileand then I use this /tmp/profile as any other. Or I go with "guixenvironment". Well, if it is something I want to hold, then I writedown the manifest and channel files.
b- inside "guix environment guix" and ~/src/guix/wk/<name> ./pre-inst-env guix pull --branch=<name> --url=$PWD -p ~/tmp/<name> /tmp/<name>/bin/guix blablaand I almost never track "permanently" such other guix.
For custom packages, I am more often using '--load-path' than 'channel'.
Last, I use more and more "guix time-machine". Well actually morethan b-, which I only use to test patches about "guix pull". Roughlyspeaking, - "guix environment" creates temporary profile - "guix time-machine" create temporary pullAnd with this new "authentication" of "guix pull", I think now, itwould be better to do: guix time-machine --branch=<name> --url=~/src/wk/<name> -- blabla

Well, I do not know if all this is the correct way. But it is more orless what I am doing to avoid annoyance.

All the best,simon
A
A
Arne Babenhauserheide wrote on 30 May 18:18 +0200
(name . John Soo)(address . jsoo1@asu.edu)
87tuzxgzw2.fsf@web.de
John Soo <jsoo1@asu.edu> writes:
Toggle quote (23 lines)> I guess that makes me wonder what the desired specification is.> There is a lot of problem space to explore. As a user I would want to> be able to take my local patches as "the real truth".>> Because guix has a linear git history, that means the user needs to> always rebase. Often I have patches open for months that I am currently> testing and working on. It would be convenient for me to be able to guix> pull into my default user profile.>> On the other hand, as you point out, using a hash that disappears from> the git history is dangerous. The git history no longer tracks the guix> pull history and then there may exist pulls in history that may never be> recoverable.>> There seems to be some existing support for the rebasing into the> default profile since branches are allowed as references in the channels> configuration. Has the rebase use case been discussed before? Oh, also,> history items can be deleted in other places with --delete-generations> and friends. I am not sure what to classify this problem, bug or> something else.>> What do you think?
This sounds like a use-case for changeset evolution withhidden-but-retrievable commits:https://www.mercurial-scm.org/doc/evolution/index.html
With that you can rebase and have linear surface history, but therewritten commits still exist as hidden commits with references to thecommits that superseded them.
Problem: It not possible with Git. It would require switching toMercurial — which would also enable useful abilities like coordinatedrebasing in a group: https://blog.disy.net/hg-evolution/(and which is GPLv3-compatible)
Best wishes,Arne-- Unpolitisch seinheißt politisch seinohne es zu merken
J
J
John Soo wrote on 31 May 07:04 +0200
(name . Arne Babenhauserheide)(address . arne_bab@web.de)
1D37F750-2DA3-4DF1-91A5-92637E58E753@asu.edu
Hi Arne,
Interesting idea about mercurial.
My problem largely comes from the fact that I specified a branch in channels.scm for over a year with the same workflow. If a branch can be specified for a channel repo then the system probably should handle the current pull commit not being available in the history.
The commit history authentication is nice but seems to have regressed this particular use case.
- John
Z
Z
zimoun wrote on 1 Jun 18:35 +0200
(name . Arne Babenhauserheide)(address . arne_bab@web.de)
CAJ3okZ2GWdWneEFsMwP=tdcpa0ns0oD2HopjLs5Sdgy1tv56kA@mail.gmail.com
Dear Arne,
On Sat, 30 May 2020 at 18:18, Arne Babenhauserheide <arne_bab@web.de> wrote:
Toggle quote (8 lines)> This sounds like a use-case for changeset evolution with> hidden-but-retrievable commits:> https://www.mercurial-scm.org/doc/evolution/index.html>> With that you can rebase and have linear surface history, but the> rewritten commits still exist as hidden commits with references to the> commits that superseded them.
Thank you for the pointer. Yes it could be cool that "git rebase -i"still stores the rewritten commits as hidden -- which would be removedby "git gc".However, IMHO, the question seems more about how to the use the weaponthan the weapon itself. Other said, it seems more an issue aboutpractise than tool.Well, rewrite the history is always dangerous, whatever the tools; andone solution when one is not sure is to duplicate the branch withe.g., "git branch --copy".
In this use-case, there is 2 incompatible requirements: - content-addressed for reproducibility etc. well the core principles of Guix - rewrite these addresses on the flyWell, without care, for sure it would break one way or another,whatever the tool is, IMHO.

Toggle quote (5 lines)> Problem: It not possible with Git. It would require switching to> Mercurial — which would also enable useful abilities like coordinated> rebasing in a group: https://blog.disy.net/hg-evolution/> (and which is GPLv3-compatible)
I am sad too that 'hg' has lost against 'git'. All my PhD was undermercurial and I was really happy with it; tool easier to grab. Now, Iam happier with Magit. ;-)Speaking about DVSC, pijul and theory of patches looks veryinteresting. Whatever! :-)https://pijul.org/

All the best,simon
Z
Z
zimoun wrote on 1 Jun 18:48 +0200
(name . John Soo)(address . jsoo1@asu.edu)
CAJ3okZ3o5zKJtzTS4d9Q9=C+oV=Vc++KFU7iBbxZ8_KrG3wa0g@mail.gmail.com
Dear John,
On Sun, 31 May 2020 at 07:04, John Soo <jsoo1@asu.edu> wrote:
Toggle quote (2 lines)> My problem largely comes from the fact that I specified a branch in channels.scm for over a year with the same workflow. If a branch can be specified for a channel repo then the system probably should handle the current pull commit not being available in the history.
What do you mean?

Toggle quote (2 lines)> The commit history authentication is nice but seems to have regressed this particular use case.
I am not sure that the new history authentication is the issue here.IMHO, it is just the feature that shows up the flaw with yourworkflow. ;-)From my understanding, Guix is built around content-addressedprinciples (channel, store, etc.) so try to change on the fly theaddress of such content would lead to break one way or another, IMHO.

Well, is the issue fixed for you now?

All the best,simon
J
J
John Soo wrote on 1 Jun 19:28 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87wo4q1yru.fsf@asu.edu
Hello again,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (9 lines)> On Sun, 31 May 2020 at 07:04, John Soo <jsoo1@asu.edu> wrote:>>> My problem largely comes from the fact that I specified a branch in>> channels.scm for over a year with the same workflow. If a branch can>> be specified for a channel repo then the system probably should>> handle the current pull commit not being available in the history.>> What do you mean?
Well if a channel specification has a branch then the channelspecification explicitly allows the commit history to not always havethe commit from "guix describe". For example, say the channels.scm filehas the following:
(list (channel (name 'example-channel) (url "https://git.example.com/repo") (branch "example-branch")) ...)
the maintainer of example-channel may rebase example-branch on someother branch. Then guix-pull will indeed be impossible for anyone whohas example-channel specified with the branch field in channels.scm.
So what I am saying is that if unrelated histories are to be disallowedduring "guix pull" then branches should not be allowed in channelspecifications.
Toggle quote (7 lines)>> The commit history authentication is nice but seems to have>> regressed this particular use case.>> I am not sure that the new history authentication is the issue here.> IMHO, it is just the feature that shows up the flaw with your> workflow. ;-)
You are right, I should not assume what introduced my problem. What I doknow is that this workflow worked for the last year or more until lastweek. I should do a bisect to find the exact commit.
Toggle quote (7 lines)>>From my understanding, Guix is built around content-addressed> principles (channel, store, etc.) so try to change on the fly the> address of such content would lead to break one way or another, IMHO.>>> Well, is the issue fixed for you now?
The issue is not fixed and I think I found another problem with theworkaround. To reiterate, my workaround is to "guix pull --roll-back"until a generation that has a the commit that is in my history then"guix pull" again to get my new work. The problem is the next "guixpull" shows all news from the old commit until the newest commit. Inother words I get the news I saw from the previous time I pulled plusany new work. In this way the news continues to accumulate making thenews less and less useful and more and more noisy.
Btw, I am totally with you on pijul/darcs or some patch theory versioncontrol. Pijul does indeed look pretty promising. I packaged it in mychannel if you want to try it :).
Kindly,
John
L
L
Ludovic Courtès wrote on 3 Jun 11:28 +0200
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604@debbugs.gnu.org)
87a71kqyzw.fsf@gnu.org
Hi John,
John Soo <jsoo1@asu.edu> skribis:
Toggle quote (14 lines)> I use a local git repo with branch that I specify in channels.scm.> My usual workflow is:>> 1. rebase on origin> 2. guix pull>> This stopped working with the following error:>> Updating channel 'guix' from Git repository at 'file:///home/john/projects/guix/.git'...> guix pull: error: aborting update of channel 'guix' to commit 1444040933ac35b967720288dc30ed70e5481ed3, which is not a descendant of 57518fc7bf1efc899c0dabaa76685a319661f8e4> hint: This could indicate that the channel has been tampered with and is trying to> force a roll-back, preventing you from getting the latest updates. If you think> this is not the case, explicitly allow non-forward updates.
What happens is that ‘guix pull’ ensures that it only ever makes“fast-forward” updates by default, in Git parlance. The goal is todetect obvious “downgrade attacks”:
https://issues.guix.gnu.org/41425
(This can be overridden this by passing ‘--allow-downgrades’.)
The rebase workflow you describe unavoidably triggers the error becauseevery time you pull, you do a non-fast-forward pull (because the commityou were previously using, as shown in ‘guix describe’, has beenrewritten and no longer exists in the new commit graph.) So at least,it shows that the machinery works as advertised. :-)
What I would recommend is for your channel to be a regular “fork”: youcreate a branch containing your patches and regularly merge upstreammaster back into your branch. That way, you don’t need to rewritehistory and ‘guix pull’ is happy.
Alternately, if you like to have linear history (for example because youintend to eventually submit your patches upstream), you could useTopGit, which roughly allows you to version-control your rebases.
HTH!
From a pure Guix perspective, it’s “not-a-bug”. If one of the abovesuggestions works for you, I think we can close this issue.
Thanks for your feedback!
Ludo’.
J
J
John Soo wrote on 3 Jun 15:44 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41604@debbugs.gnu.org)
878sh41cx5.fsf@asu.edu
Hi Ludo!
I hope you are well.
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (6 lines)> What happens is that ‘guix pull’ ensures that it only ever makes> “fast-forward” updates by default, in Git parlance. The goal is to> detect obvious “downgrade attacks”:>> https://issues.guix.gnu.org/41425
Oh I see, I'm sorry I did not participate in that issue, now. Thatmakes sense and I'm glad to know this is part of the design.
Toggle quote (2 lines)> (This can be overridden this by passing ‘--allow-downgrades’.)
Does '--allow-downgrades' support unrelated git histories? I tried thatflag and it did not work.
Toggle quote (6 lines)> The rebase workflow you describe unavoidably triggers the error because> every time you pull, you do a non-fast-forward pull (because the commit> you were previously using, as shown in ‘guix describe’, has been> rewritten and no longer exists in the new commit graph.) So at least,> it shows that the machinery works as advertised. :-)
Totally. I think I understand the design goal now.
Toggle quote (5 lines)> What I would recommend is for your channel to be a regular “fork”: you> create a branch containing your patches and regularly merge upstream> master back into your branch. That way, you don’t need to rewrite> history and ‘guix pull’ is happy.
So far the need to avoid downgrade attacks makes sense. Here are thethings I think should be considered:
I have branches based on master in savannah that contain specific patchsets associated to patch requests upstream. I think I have 3 or 4 rightnow. My patches are also in the branch I have in channels.scm. I dothat for a few reasons:
1. To test the patches2. To workaround or use bugs/features/packages I want but are not upstream yet.
That means I tend to want to use my patches whether or not they areupstream yet. In fact I stopped working on my channel because it was soeasy to just make patches on upstream to contribute back.
It can take many months for patches to be merged. That is expectedsince we are all volunteers. Rebasing the patches is the easiest way tokeep them up to date so they can be applied cleanly.
There are two design and community goals I love about Guix: hackabilityand inclusivity. I feel that disallowing linear history makes theeasiest way to contribute to, hack on, and participate in Guix muchharder without proper support. For instance: instead of making patcheson top of upstream it is now easier just to work on my channel.
Certainly some tradeoffs should be made for security and I think yourrecent commit authentication work does that elegantly. Perhaps we caneasily have hackability and security with a flag like --allow-downgradescalled --allow-unrelated that allows the rebase workflow.
WDYT?

Toggle quote (4 lines)> Alternately, if you like to have linear history (for example because you> intend to eventually submit your patches upstream), you could use> TopGit, which roughly allows you to version-control your rebases.
Hmm. I am unaware of TopGit but I find rebasing to be the simplest andeasiest way to do my work. I'll look into it but I would rather not haveto use another tool for simplicity's sake.

Toggle quote (3 lines)> From a pure Guix perspective, it’s “not-a-bug”. If one of the above> suggestions works for you, I think we can close this issue.
I think I understand the reasoning now. I do hope a flag like--allow-unrelated might solve all the things.
Toggle quote (2 lines)> HTH!
Thanks as always. I really appreciate your communications.
- John
L
L
Ludovic Courtès wrote on 3 Jun 17:13 +0200
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604@debbugs.gnu.org)
87img8mbci.fsf@gnu.org
Hello,
John Soo <jsoo1@asu.edu> skribis:
Toggle quote (2 lines)> Ludovic Courtès <ludo@gnu.org> writes:
[...]
Toggle quote (5 lines)>> (This can be overridden this by passing ‘--allow-downgrades’.)>> Does '--allow-downgrades' support unrelated git histories? I tried that> flag and it did not work.
It supports unrelated Git histories. It could really be called‘--allow-anything’ but I thought it’d be less descriptive. :-)
If you hit a problem with that, please report it (perhaps I justoverlooked it in the other issue.)
Toggle quote (16 lines)> I have branches based on master in savannah that contain specific patch> sets associated to patch requests upstream. I think I have 3 or 4 right> now. My patches are also in the branch I have in channels.scm. I do> that for a few reasons:>> 1. To test the patches> 2. To workaround or use bugs/features/packages I want but are not upstream yet.>> That means I tend to want to use my patches whether or not they are> upstream yet. In fact I stopped working on my channel because it was so> easy to just make patches on upstream to contribute back.>> It can take many months for patches to be merged. That is expected> since we are all volunteers. Rebasing the patches is the easiest way to> keep them up to date so they can be applied cleanly.
Yes.
Toggle quote (11 lines)> There are two design and community goals I love about Guix: hackability> and inclusivity. I feel that disallowing linear history makes the> easiest way to contribute to, hack on, and participate in Guix much> harder without proper support. For instance: instead of making patches> on top of upstream it is now easier just to work on my channel.>> Certainly some tradeoffs should be made for security and I think your> recent commit authentication work does that elegantly. Perhaps we can> easily have hackability and security with a flag like --allow-downgrades> called --allow-unrelated that allows the rebase workflow.
Interesting, I hadn’t thought about how this mechanism would give anincentive to have a channel vs. contributing directly upstream.
Normally, ‘--allow-downgrades’ does exactly what you need, at leastthat’s the intent. I’d argue that it’s also reasonable to use it inthis case because obviously you know what you’re doing, and you’repulling from a local Git repository, so that’s fine.
Toggle quote (8 lines)>> Alternately, if you like to have linear history (for example because you>> intend to eventually submit your patches upstream), you could use>> TopGit, which roughly allows you to version-control your rebases.>> Hmm. I am unaware of TopGit but I find rebasing to be the simplest and> easiest way to do my work. I'll look into it but I would rather not have> to use another tool for simplicity's sake.
Yeah, that makes sense to me.
Thanks,Ludo’.
J
J
John Soo wrote on 4 Jun 16:11 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41604@debbugs.gnu.org)
874krq2a5k.fsf@asu.edu
Hi Ludo,
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (6 lines)> It supports unrelated Git histories. It could really be called> ‘--allow-anything’ but I thought it’d be less descriptive. :-)>> If you hit a problem with that, please report it (perhaps I just> overlooked it in the other issue.)
Oh nice, then this whole issue I think should be to report that--allow-downgrades does not allow everything. When I first reported theissue I tried --allow-downgrades thanks to some help on IRC. The firstattempt failed as I mentioned previously in the thread. However I didjust try --allow-downgrades again after a rebase and it seems to havesucceeded. With that, I think this issue can be closed. Thanks forthinking of this case!
Toggle quote (3 lines)> Interesting, I hadn’t thought about how this mechanism would give an> incentive to have a channel vs. contributing directly upstream.
I think I will add some notes about a rebase workflow and--allow-downgrades to the contributing documentation. Looking just nowI'm not sure using a local source tree as a channel is mentioned in thedocumentation either. I will open a different patch set to deal withthose issues.
Toggle quote (5 lines)> Normally, ‘--allow-downgrades’ does exactly what you need, at least> that’s the intent. I’d argue that it’s also reasonable to use it in> this case because obviously you know what you’re doing, and you’re> pulling from a local Git repository, so that’s fine.
100% it does what I need.
Thanks again, feel free to close.
- John
Z
Z
zimoun wrote on 5 Jun 03:23 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ3wEbUWYVdu5q5LUCJM9QKeomUiihHAiHH985r0xLdr+Q@mail.gmail.com
Hi Ludo,
On Wed, 3 Jun 2020 at 17:14, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (11 lines)> >> (This can be overridden this by passing ‘--allow-downgrades’.)> >> > Does '--allow-downgrades' support unrelated git histories? I tried that> > flag and it did not work.>> It supports unrelated Git histories. It could really be called> ‘--allow-anything’ but I thought it’d be less descriptive. :-)>> If you hit a problem with that, please report it (perhaps I just> overlooked it in the other issue.)
I think it is not a bug and it is a feature* :-) but the behaviour haschanged for the commits which do not belong to the repo anymore.That's why John has not seen the issue of his "rebase workflow"before.
*feature: at least, it seems expected from what I understand of the code. :-)

Let remind the commit history. Instead of create a channel file, Idirectly use the raw repo, but it is the same for any channel (Gitrepo).
Toggle snippet (15 lines)$ SRC=~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq$ git -C $SRC --no-pager log --oneline c873980d18^..8bd0b533b38bd0b533b3 gnu: libexif: Update to 0.6.22 [security fixes].e451612602 gnu: libgphoto2: Update to 2.5.25.9744cc7b46 pull: Protect against downgrade attacks.872898f768 channels: 'latest-channel-instances' guards againstnon-forward updates.8d1d56578a git: 'update-cached-checkout' returns the commit relation.9b049de84e channels: 'latest-channel-instances' doesn't leak internal state.c098c11be8 git: Add 'commit-relation'.86ac14b2f3 (HEAD -> master) gnu: protonvpn-cli: Tweak description.c873980d18 gnu: Add protonvpn-cli.

Here, a first session from a commit before the "downgrade attacks"commit. A commit is added, then pulled, then rebased where thisaddition is totally deleted of the Git repo, then another pulled. Noerror at all.
Toggle snippet (41 lines)$ guix describeGeneration 34 Jun 05 2020 02:16:22 (current) guix 86ac14b repository URL: https://git.savannah.gnu.org/git/guix.git commit: 86ac14b2f37efbb6f4a3ed1c3e183fbc9496b7a5
$ echo hello >> $SRC/README && git -C $SRC commit -am hello[master 20e984e931] hello 1 file changed, 1 insertion(+)
$ guix pull --commit=$(git -C $SRC rev-parse HEAD)Updating channel 'guix' from Git repository at'https://git.savannah.gnu.org/git/guix.git'...Building from this channel: guix https://git.savannah.gnu.org/git/guix.git 20e984eComputing Guix derivation for 'x86_64-linux'... /
$ guix describeGeneration 35 Jun 05 2020 02:32:43 (current) guix 20e984e repository URL: https://git.savannah.gnu.org/git/guix.git commit: 20e984e9311404295c9c82b54eac1c277709b0a0
$ git -C $SRC reset --hard HEAD^HEAD is now at 86ac14b2f3 gnu: protonvpn-cli: Tweak description.
$ git -C $SRC reflog expire --expire-unreachable=now --all$ git -C $SRC gc --prune=now --quiet
$ git -C $SRC show 20e984e9311404295c9c82b54eac1c277709b0a0fatal: bad object 20e984e9311404295c9c82b54eac1c277709b0a0
$ guix pull --commit=c873980d18Updating channel 'guix' from Git repository at'https://git.savannah.gnu.org/git/guix.git'...Building from this channel: guix https://git.savannah.gnu.org/git/guix.git c873980Computing Guix derivation for 'x86_64-linux'... /

Now the same session after the introduction of '--allow-downgrades'.
Toggle snippet (39 lines)$ guix describeGeneration 37 Jun 05 2020 01:28:52 (current) guix 8bd0b53 repository URL: https://git.savannah.gnu.org/git/guix.git commit: 8bd0b533b30d7ee5e03aee99a2eb96d5b0b1c836
$ echo hello >> $SRC/README && git -C $SRC commit -am hello[master 09f6e9b34c] hello 1 file changed, 1 insertion(+)
$ guix pull --commit=$(git -C $SRC rev-parse HEAD)Updating channel 'guix' from Git repository at'https://git.savannah.gnu.org/git/guix.git'...Building from this channel: guix https://git.savannah.gnu.org/git/guix.git 09f6e9bComputing Guix derivation for 'x86_64-linux'... /
$ guix describeGeneration 38 Jun 05 2020 02:57:13 (current) guix 09f6e9b repository URL: https://git.savannah.gnu.org/git/guix.git commit: 09f6e9b34c6239bcdd8ca9e030d698b5244507a6
$ git -C $SRC reset --hard HEAD^HEAD is now at 8bd0b533b3 gnu: libexif: Update to 0.6.22 [security fixes].
$ git -C $SRC reflog expire --expire-unreachable=now --all$ git -C $SRC gc --prune=now --quiet$ git -C $SRC show 09f6e9b34c6239bcdd8ca9e030d698b5244507a6fatal: bad object 09f6e9b34c6239bcdd8ca9e030d698b5244507a6
$ guix pull --commit=e451612602 --allow-downgradesUpdating channel 'guix' from Git repository at'https://git.savannah.gnu.org/git/guix.git'...guix pull: error: Git error: object not found - no match for id(09f6e9b34c6239bcdd8ca9e030d698b5244507a6)

Well, I admit it is an unexpected use-case. The solution here is to'--roll-back' but it can be tedious if we are talking about severalcommits which had been removed in the channel by the user. Therefore,the best is to use "--list-generations" to find a generation with anexisting commit and switch to it with "guix pull--switch-generation=NN". However, some '--news' will be lost. Nofree lunch. ;-)
I do not think it will be worth to allow this kind of workflow but onesolution could be to add a flag, i.e., '--allow-downgrade=dangerous',and bypass 'commit-relation' which is somehow the culprit here.
IMHO, it is better to document what to do when someone does a mistakeby removing the current commit where they is currently (describe).
WDYT?
All the best,simon
Z
Z
zimoun wrote on 5 Jun 03:44 +0200
(name . John Soo)(address . jsoo1@asu.edu)
CAJ3okZ2RuVqCRfUYUkQCwhx-6ZgX48fEgBsG3dO4XQ-iHjxL9A@mail.gmail.com
Hi John,
On Thu, 4 Jun 2020 at 16:12, John Soo <jsoo1@asu.edu> wrote:
Toggle quote (8 lines)> Oh nice, then this whole issue I think should be to report that> --allow-downgrades does not allow everything. When I first reported the> issue I tried --allow-downgrades thanks to some help on IRC. The first> attempt failed as I mentioned previously in the thread. However I did> just try --allow-downgrades again after a rebase and it seems to have> succeeded. With that, I think this issue can be closed. Thanks for> thinking of this case!
Well, there is 2 issues, I guess. One is fixed by allow downgrade.
The other one is not about '--allow-downgrades' but simply about pullfrom a removed commit (rebase). I do not think it could be consideredas a bug but the behaviour has changed and '--allow-downgrades' is notenough. Well, if I understand correctly what you described before inthis thread. :-)

Toggle quote (6 lines)> I think I will add some notes about a rebase workflow and> --allow-downgrades to the contributing documentation. Looking just now> I'm not sure using a local source tree as a channel is mentioned in the> documentation either. I will open a different patch set to deal with> those issues.
IMHO, the trick to be able to pull from a removed commit is to switchgeneration first. The price to pay is that history will be lost andso news will be screwed up.
Well, for sure describe your workflow could be an entry of the Cookbook. :-)

All the best,simon
L
L
Ludovic Courtès wrote on 5 Jun 18:13 +0200
(name . John Soo)(address . jsoo1@asu.edu)(address . 41604-done@debbugs.gnu.org)
87r1ut32yv.fsf@gnu.org
Hi,
John Soo <jsoo1@asu.edu> skribis:
Toggle quote (9 lines)>> Normally, ‘--allow-downgrades’ does exactly what you need, at least>> that’s the intent. I’d argue that it’s also reasonable to use it in>> this case because obviously you know what you’re doing, and you’re>> pulling from a local Git repository, so that’s fine.>> 100% it does what I need.>> Thanks again, feel free to close.
OK, closing. Let me know if you stumble upon issues in that area!
Ludo’.
Closed
L
L
Ludovic Courtès wrote on 5 Jun 18:17 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87h7vp32s0.fsf@gnu.org
Hi Simon,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (37 lines)> $ guix describe> Generation 37 Jun 05 2020 01:28:52 (current)> guix 8bd0b53> repository URL: https://git.savannah.gnu.org/git/guix.git> commit: 8bd0b533b30d7ee5e03aee99a2eb96d5b0b1c836>> $ echo hello >> $SRC/README && git -C $SRC commit -am hello> [master 09f6e9b34c] hello> 1 file changed, 1 insertion(+)>> $ guix pull --commit=$(git -C $SRC rev-parse HEAD)> Updating channel 'guix' from Git repository at> 'https://git.savannah.gnu.org/git/guix.git'...> Building from this channel:> guix https://git.savannah.gnu.org/git/guix.git 09f6e9b> Computing Guix derivation for 'x86_64-linux'... />> $ guix describe> Generation 38 Jun 05 2020 02:57:13 (current)> guix 09f6e9b> repository URL: https://git.savannah.gnu.org/git/guix.git> commit: 09f6e9b34c6239bcdd8ca9e030d698b5244507a6>> $ git -C $SRC reset --hard HEAD^> HEAD is now at 8bd0b533b3 gnu: libexif: Update to 0.6.22 [security fixes].>> $ git -C $SRC reflog expire --expire-unreachable=now --all> $ git -C $SRC gc --prune=now --quiet> $ git -C $SRC show 09f6e9b34c6239bcdd8ca9e030d698b5244507a6> fatal: bad object 09f6e9b34c6239bcdd8ca9e030d698b5244507a6>> $ guix pull --commit=e451612602 --allow-downgrades> Updating channel 'guix' from Git repository at> 'https://git.savannah.gnu.org/git/guix.git'...> guix pull: error: Git error: object not found - no match for id> (09f6e9b34c6239bcdd8ca9e030d698b5244507a6)
Ah I see, thanks for the reproducer.
Generally, rebasing does not necessarily implies ‘git gc’, so I thinkone has to be unlucky to have the former commit disappear right away,no?
But yeah, I don’t think there’s much we can do. Or perhaps we couldhave ‘commit-relation’ report 'unrelated when one of the commits doesnot exist, that’d be clearer and more useful than this error.
WDYT?
Ludo’.
Z
Z
zimoun wrote on 5 Jun 19:51 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ0247tNK66ak-upsE0jKvLrx0fyaCO5LUfJcKOn3YaG=A@mail.gmail.com
Hi Ludo,
On Fri, 5 Jun 2020 at 18:17, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (2 lines)> Generally, rebasing does not necessarily implies ‘git gc’, so I think
It depends on how "git gc" is configured. Well, I am not a Gitspecialist but from my understanding of the manual, by default, theseunreachable commits are kepts 30 days and then garbage collected.

Toggle quote (3 lines)> one has to be unlucky to have the former commit disappear right away,> no?
Unluck happens! as Forest Gump said. ;-)

Toggle quote (4 lines)> But yeah, I don’t think there’s much we can do. Or perhaps we could> have ‘commit-relation’ report 'unrelated when one of the commits does> not exist, that’d be clearer and more useful than this error.
And report a hint.For example, guix pull --switch-generation

Cheers,simon
L
L
Ludovic Courtès wrote on 7 Jun 23:16 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87sgf6r2z4.fsf@gnu.org
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (6 lines)>> But yeah, I don’t think there’s much we can do. Or perhaps we could>> have ‘commit-relation’ report 'unrelated when one of the commits does>> not exist, that’d be clearer and more useful than this error.>> And report a hint.
I did this:
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1fd7de45f218ce572a3fe87764ad15927e3dbdc4
So now, in a situation like John described, ‘guix pull’ will print theerror and hint for unrelated commits.
I think we can close now!
Ludo’.
Z
Z
zimoun wrote on 8 Jun 00:25 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ2tpaKRmSSzS-zUWjrDB9SCRd1zx8YeELgULBbfcs3KGw@mail.gmail.com
Hi Ludo,
On Sun, 7 Jun 2020 at 23:16, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (4 lines)> I did this:>> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1fd7de45f218ce572a3fe87764ad15927e3dbdc4
Cool! Thank you.
Toggle quote (3 lines)> So now, in a situation like John described, ‘guix pull’ will print the> error and hint for unrelated commits.
Yes! Perfect! :-)
From a removed commit: "guix pull" says:
Toggle snippet (6 lines)guix pull: error: aborting update of channel 'guix' to commite78275608065ef073775fabb9f1a757da65851f2, which is not a descendant ofb50fb18a610e2a495624bf4570bec60265332615hint: Use `--allow-downgrades' to force this downgrade.
then using the hint, it says:
Toggle snippet (8 lines)guix pull: warning: moving channel 'guix' fromb50fb18a610e2a495624bf4570bec60265332615 to unrelated commite78275608065ef073775fabb9f1a757da65851f2Building from this channel: guix https://git.savannah.gnu.org/git/guix.git e782756

Just to note that the hint
Toggle snippet (5 lines)hint: This could indicate that the channel has been tampered with and is tryingto force a roll-back, preventing you from getting the latest updates. Ifyou think this is not the case, explicitly allow non-forward updates.
does not report explicitly the option '--allow-downgrades'.

Toggle quote (2 lines)> I think we can close now!
Definitively! :-)
Cheers,simon
J
J
John Soo wrote on 8 Jun 01:52 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
54D87F21-F2FE-4949-8FD7-9BF8082436B3@asu.edu
Perfect! Thank you!
L
L
Ludovic Courtès wrote on 10 Jun 16:51 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87mu5bdleu.fsf@gnu.org
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (29 lines)> On Sun, 7 Jun 2020 at 23:16, Ludovic Courtès <ludo@gnu.org> wrote:>>> I did this:>>>> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=1fd7de45f218ce572a3fe87764ad15927e3dbdc4>> Cool! Thank you.>>> So now, in a situation like John described, ‘guix pull’ will print the>> error and hint for unrelated commits.>> Yes! Perfect! :-)>> From a removed commit: "guix pull" says:>> guix pull: error: aborting update of channel 'guix' to commit> e78275608065ef073775fabb9f1a757da65851f2, which is not a descendant of> b50fb18a610e2a495624bf4570bec60265332615> hint: Use `--allow-downgrades' to force this downgrade.>>> then using the hint, it says:>> guix pull: warning: moving channel 'guix' from> b50fb18a610e2a495624bf4570bec60265332615 to unrelated commit> e78275608065ef073775fabb9f1a757da65851f2> Building from this channel:> guix https://git.savannah.gnu.org/git/guix.git e782756
Thanks for testing, closing!
Toggle quote (8 lines)> Just to note that the hint>> hint: This could indicate that the channel has been tampered with and is trying> to force a roll-back, preventing you from getting the latest updates. If> you think this is not the case, explicitly allow non-forward updates.>> does not report explicitly the option '--allow-downgrades'.
Ah, right.
Ludo’.
Closed
?