Building iwd on armhf triggers kernel panic

  • Done
  • quality assurance status badge
Details
3 participants
  • Maxim Cournoyer
  • Mark H Weaver
  • Vagrant Cascadian
Owner
unassigned
Submitted by
Vagrant Cascadian
Severity
normal
V
V
Vagrant Cascadian wrote on 1 Sep 2020 01:26
(address . bug-guix@gnu.org)
874kois8hg.fsf@ponder
I've been getting kernel panics when i try to build iwd, a dependency of
network-manager.

It crashes hard, so there's no recorded log, but I managed to scrape
this from an ssh session:

$ guix build iwd
...
PASS: unit/test-p2p
PASS: unit/test-client
./build-aux/test-driver: line 107: 6837 Aborted "$@" >
$log_file 2>&1
FAIL: unit/test-arc4
PASS: unit/test-sa


When the test-arc4 fails, soon after a kernel panic results and the
system freezes. Repeated this multiple times.

Going to remove iwd support for network manager locally and then try
again.


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCX02HDAAKCRDcUY/If5cW
qom9AQDHYK6hnWt/SkfYd1pGzPNmXj3NBpJt1Jlq9+P00mH1SAEA8UGY8Feuqs3E
Yiiahmad69mkBFmD3EDLdc2DvNNScgc=
=eafX
-----END PGP SIGNATURE-----

M
M
Mark H Weaver wrote on 1 Sep 2020 03:13
878sdufgdu.fsf@netris.org
Vagrant Cascadian <vagrant@debian.org> writes:
Toggle quote (3 lines)
> I've been getting kernel panics when i try to build iwd, a dependency of
> network-manager.

Which kernel package variant(s) and version(s) have you seen panic when
building iwd? What hardware?

Thanks,
Mark
V
V
Vagrant Cascadian wrote on 1 Sep 2020 07:40
87h7sirr62.fsf@ponder
On 2020-08-31, Mark H. Weaver wrote:
Toggle quote (7 lines)
> Vagrant Cascadian <vagrant@debian.org> writes:
>> I've been getting kernel panics when i try to build iwd, a dependency of
>> network-manager.
>
> Which kernel package variant(s) and version(s) have you seen panic when
> building iwd? What hardware?

The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
rockchip rk3288 CPU and 4GB of ram; I might be able to test on
additional hardware at some point (I've got a pretty old guix install on
a novena, but don't recall if it was in good working order last time it
booted).

linux-libre-arm-generic@5.4 5.4.57, and presumably older versions as
well, as I had commits in my local guix from mid June disabling iwd on
network-manager.

I've had issues booting linux-libre 5.6+ on this hardware, and struggled
to get the right initrd modules to use the regular "linux-libre"
variant, though in theory that should also be possible, but might
require additional kernel config options, and I'd like to postpone
exploring those rabbit holes for the moment...


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCX03etwAKCRDcUY/If5cW
qv9zAP92wYq7VrCz85HBYSS+nLZ45HfR7xhMwED4B1X4Hy/QLgD/ccJQalq5/pN2
GlJxwZdX4XZuYgL4P2T0Osas/oU4FAY=
=qoX5
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 2 Sep 2020 05:02
875z8wop8q.fsf@ponder
On 2020-08-31, Vagrant Cascadian wrote:
Toggle quote (14 lines)
> On 2020-08-31, Mark H. Weaver wrote:
>> Vagrant Cascadian <vagrant@debian.org> writes:
>>> I've been getting kernel panics when i try to build iwd, a dependency of
>>> network-manager.
>>
>> Which kernel package variant(s) and version(s) have you seen panic when
>> building iwd? What hardware?
>
> The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
> rockchip rk3288 CPU and 4GB of ram; I might be able to test on
> additional hardware at some point (I've got a pretty old guix install on
> a novena, but don't recall if it was in good working order last time it
> booted).

Ok, managed to build iwd just fine on a novena imx6 platform, running
armhf on "linux-libre" 5.4.25 (haven't yet finished upgrading the
system).

So it appears to potentially be hardware-specific? The imx6 vs. rk3288
are pretty different platforms; I'm not remembering off the top of my
head what the major differences are, but if the build is detecting some
optional cpu feature, and the test is assuming one way or the
other... though still, triggering a kernel panic is a bit exciting way
to fail!


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCX08LNgAKCRDcUY/If5cW
ql13AQD8XTpcbu1+tz5Z7OuKajxizMzEP3vIR1JnolPK+lngogEAu5giZHDOn2al
h4cw1WiR6GSTSdlMdJ2cIUDjcPhkHAE=
=5dpa
-----END PGP SIGNATURE-----

M
M
Mark H Weaver wrote on 2 Sep 2020 07:04
87k0xc22gu.fsf@netris.org
Hi Vagrant,

Vagrant Cascadian <vagrant@debian.org> writes:

Toggle quote (15 lines)
> On 2020-08-31, Vagrant Cascadian wrote:
>> On 2020-08-31, Mark H. Weaver wrote:
>>> Vagrant Cascadian <vagrant@debian.org> writes:
>>>> I've been getting kernel panics when i try to build iwd, a dependency of
>>>> network-manager.
>>>
>>> Which kernel package variant(s) and version(s) have you seen panic when
>>> building iwd? What hardware?
>>
>> The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
>> rockchip rk3288 CPU and 4GB of ram; I might be able to test on
>> additional hardware at some point (I've got a pretty old guix install on
>> a novena, but don't recall if it was in good working order last time it
>> booted).

That message went on to say the following, although you omitted it here:

Toggle quote (11 lines)
>> linux-libre-arm-generic@5.4 5.4.57, and presumably older versions as
>> well, as I had commits in my local guix from mid June disabling iwd on
>> network-manager.

> Ok, managed to build iwd just fine on a novena imx6 platform, running
> armhf on "linux-libre" 5.4.25 (haven't yet finished upgrading the
> system).
>
> So it appears to potentially be hardware-specific? The imx6 vs. rk3288
> are pretty different platforms;

Based on what you wrote, it sounds like it could either be a
hardware-specific problem, or else a kernel-configuration-specific
problem, since you used radically different kernel configurations in
these two cases.

If I understand correctly, 'linux-libre-arm-generic@5.4' is the variant
that panicked on an rk3288, whereas the 'linux-libre@5.4' variant with
our hand-crafted default configuration worked on an imx6.

I don't know whether it's possible to use precisely the same kernel
configuration on these two systems. Anyway, it would be helpful to
retry the 'iwd' build on these two devices with kernel versions and
configurations that are identical except for a minimal set of required
differences. That would help us narrow down the set of possible causes
for this kernel panic.

What do you think?

Thanks,
Mark
V
V
Vagrant Cascadian wrote on 21 Sep 2020 22:32
87a6xirhvb.fsf@ponder
On 2020-09-02, Mark H Weaver wrote:
Toggle quote (45 lines)
> Vagrant Cascadian <vagrant@debian.org> writes:
>> On 2020-08-31, Vagrant Cascadian wrote:
>>> On 2020-08-31, Mark H. Weaver wrote:
>>>> Vagrant Cascadian <vagrant@debian.org> writes:
>>>>> I've been getting kernel panics when i try to build iwd, a dependency of
>>>>> network-manager.
>>>>
>>>> Which kernel package variant(s) and version(s) have you seen panic when
>>>> building iwd? What hardware?
>>>
>>> The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
>>> rockchip rk3288 CPU and 4GB of ram; I might be able to test on
>>> additional hardware at some point (I've got a pretty old guix install on
>>> a novena, but don't recall if it was in good working order last time it
>>> booted).
>
> That message went on to say the following, although you omitted it here:
>
>>> linux-libre-arm-generic@5.4 5.4.57, and presumably older versions as
>>> well, as I had commits in my local guix from mid June disabling iwd on
>>> network-manager.
>
>> Ok, managed to build iwd just fine on a novena imx6 platform, running
>> armhf on "linux-libre" 5.4.25 (haven't yet finished upgrading the
>> system).
>>
>> So it appears to potentially be hardware-specific? The imx6 vs. rk3288
>> are pretty different platforms;
>
> Based on what you wrote, it sounds like it could either be a
> hardware-specific problem, or else a kernel-configuration-specific
> problem, since you used radically different kernel configurations in
> these two cases.
>
> If I understand correctly, 'linux-libre-arm-generic@5.4' is the variant
> that panicked on an rk3288, whereas the 'linux-libre@5.4' variant with
> our hand-crafted default configuration worked on an imx6.
>
> I don't know whether it's possible to use precisely the same kernel
> configuration on these two systems. Anyway, it would be helpful to
> retry the 'iwd' build on these two devices with kernel versions and
> configurations that are identical except for a minimal set of required
> differences. That would help us narrow down the set of possible causes
> for this kernel panic.

Sorry for the late follow-up...

Using a more recent version of linux-libre-arm-generic@5.4, it failed to
build on both the veyron-speedy (rk3288) and the novena (imx6). It
doesn't necessarily *always* trigger a kernel panic; I haven't
identified which conditions trigger the kernel panic, but so far I've
only seen them on the veyron-speedy. I haven't seen it successfully
build with any recent linux-libre-arm-generic kernel, though recent
linux-libre@5.4 and @5.8 worked.

So it seems something in iwd needs specific kernel features in order to
build at all...

Not sure when I'll get a chance to debug further, but figured I'd update
about what I've tried so far.

live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCX2kNyQAKCRDcUY/If5cW
qpaeAPwKpO0UWwy9mjEseFtttgRlw7TUIGvNyeZMIdFeQ79lxAD+O1BPjVdZq9UQ
cK2CJ0ymM4xGtdae4R837qAKI+3ORwM=
=VQ0F
-----END PGP SIGNATURE-----

M
M
Maxim Cournoyer wrote on 29 Sep 2022 02:26
(name . Vagrant Cascadian)(address . vagrant@debian.org)
875yh7qb3m.fsf@gmail.com
Hello,

[...]

Toggle quote (14 lines)
> Using a more recent version of linux-libre-arm-generic@5.4, it failed to
> build on both the veyron-speedy (rk3288) and the novena (imx6). It
> doesn't necessarily *always* trigger a kernel panic; I haven't
> identified which conditions trigger the kernel panic, but so far I've
> only seen them on the veyron-speedy. I haven't seen it successfully
> build with any recent linux-libre-arm-generic kernel, though recent
> linux-libre@5.4 and @5.8 worked.
>
> So it seems something in iwd needs specific kernel features in order to
> build at all...
>
> Not sure when I'll get a chance to debug further, but figured I'd update
> about what I've tried so far.

Bordeaux managed to build it, according to 'guix weather
--system=armhf-linux iwd', hence I'm closing this old forgotten issue.
Please reopen if you still experience it on a standard Guix System
kernel.

Thanks,

Maxim
Closed
?