Good morning Leo, > > Yes, it is true that there is something of a requirement of a strict > > behavior here, I suppose it is possible to use `git-fetch` instead of > > `url-fetch` for our external libraries. > > Yes you can use git-fetch, to make sure we are on the same page, are > you speaking of strict behavior requirements like for Bitcoin Core's > consensus code? Well we need to produce signatures and transactions that pass Bitcoin Core signature validation at least, so it is best to use a version of `libsecp256k1` (which produces signatures) that we know works, as well as `libwally-core` (which produces transactions). I would personally use the `libsecp256k1` version that `libwally-core` vendors in as well, since there may be specific interactions between `libwally-core` and `libsecp256k1` that may be different if we use the Bitcoin Core version of `libsecp256k1`. For `libsodium`, at least the hashing has to work correctly, but I think it is simple enough that strict behavior requirements are not so onerous. Indeed, we usually get this from the OS (but we need a later feature than that available on some old Ubuntu versions, which is why it got vendored in), so I should probably "just" add it as an input. > > > How do I generate `guix hash` for `git-fetch`ed `source`s? > > Actually what I do is put a wrong hash in and then copy the "actual > hash" from the error. I havent found another way but this definitely > feels subpar and prevents much verification before putting in the hash, > better suggestions welcome. Haha I shall do so as well. > > > However it also means that every release of C-Lightning I have to go > > double-check what git commit to use for each library (though `jsmn` > > and `libbacktrace` at least seem very stable). > > But it looks to me that unvendoring will require more extensive > > patching of the `Makefile` and an even larger maintenance burden on > > Guix side? > > Unvendoring is more or less a policy because we must be able to audit > each piece of software separately for freedom issues (licenses, > violations to the GNU FSDG), and it eases work for security-patching > also. I understand. This will require a largish amount of work I think. Would this technique be acceptable? * `add-before 'configure 'unvendor-externals` * `rm -rf` the vendored externals. * `ln -s` the needed `.h` and `.la`/`.a`/`.so` files from the `inputs` to the expected paths within the `external/` directory. ? > > Please do, I am not very familiar with any Python infrastructure and > > am primarily a C programmer here, I just barely hack together some > > kind of test in Python. > > If you can list the Python dependencies and their version I can look at > packaging them. We have a `requirements.txt` file which contains this, I duplicate below: ``` # Dependencies required to build and test c-lightning https://github.com/ElementsProject/libwally-core/releases/download/release_0.8.0/wallycore-0.8.0-cp36-cp36m-linux_x86_64.whl; 'linux' in sys_platform and python_version == '3.6' https://github.com/ElementsProject/libwally-core/releases/download/release_0.8.0/wallycore-0.8.0-cp37-cp37m-linux_x86_64.whl; 'linux' in sys_platform and python_version == '3.7' https://github.com/ElementsProject/libwally-core/releases/download/release_0.8.0/wallycore-0.8.0-cp37-cp37m-macosx_10_14_x86_64.whl; sys_platform == 'darwin' and python_version == '3.7' mrkd ~= 0.1.6 Mako ~= 1.1.3 # Dependencies from pyln-client Sphinx ~= 3.4.0 flake8==3.7.8 recommonmark>=0.7.* sphinx-rtd-theme==0.4.2 sphinxcontrib-websupport==1.1.0 tqdm==4.32.2 # Dependencies from pyln-testing Flask==1.1.* cheroot==8.5.* ephemeral-port-reserve==1.1.1 filelock==3.0.* flaky ~= 3.7.0 psutil==5.7.* psycopg2-binary==2.8.* pytest-rerunfailures==9.1.1 pytest-timeout ~= 1.4.2 pytest-xdist ~= 2.2.0 pytest==6.1.* python-bitcoinlib==0.11.* # Dependencies from pyln-proto base58 ~= 2.0.1 bitstring ~= 3.1.6 coincurve ~= 13.0.0 cryptography ~= 3.2 mypy ~= 0.790 pysocks ~= 1.7.1 # Dependencies from pyln-spec # None ``` Incidentally, we also install some Python modules. How do I "properly" export the Python modules within Guix? Regards, ZmnSCPxj