Hmm...oh actually, I think I may know what happened here. We changed rpm-ostree to Requires: bootc in this release. bootc Requires: bootupd. And Anaconda detects whether bootupd is installed in the image and uses it to determine whether or not it should be used.
I think for f40 I can unwind that requirement...but ugh, messy. We really should have had a different way to tell Anaconda to use bootupd (probably by querying whether it has anything in the update payloads).
Trying to re-sever the rpm-ostree -> bootc requirement will be extremely messy right now. Not impossible, but really messy. So let's look at the bootc -> bootupd link.
Actually bootc today just Recommends: bootupd. So we could aim to explicitly exclude that package in the f40 workstation configs.
Except no wait, f40 workstation does include bootupd. What ostree ref or container image is this test case installing?
Note, I think what matters here is what's in the installer environment, not what's in the payload it's deploying. I'll look at the answers to the questions shortly, but my guess is this causes bootupd to be in the installer environment when it was not before. that shouldn't be hard to check once I've had lunch.
So in the 'success' case, bootc is not in the installer environment and bootupd is not in the ostree (it's a Silverblue ostree). In the 'failure' case, bootc is in the installer environment and bootupd is in the ostree.
anaconda's conditional seems to just depend on whether bootupd is in the installed system root (so, the ostree):
def have_bootupd(sysroot):
"""Is bootupd/bootupctl present in sysroot?"""
return os.path.exists(join_paths(sysroot, "/usr/bin/bootupctl"))
def run(self):
if have_bootupd(self._sysroot):
self._install_bootupd()
else:
self._move_grub_config()
self._set_kargs()
so, it's the presence of bootupd in the ostree that triggers the problem, I guess.
So in the 'success' case, bootc is not in the installer environment and bootupd is not in the ostree (it's a Silverblue ostree). In the 'failure' case, bootc is in the installer environment and bootupd is in the ostree.
anaconda's conditional seems to just depend on whether bootupd is in the installed system root (so, the ostree):
def have_bootupd(sysroot):
"""Is bootupd/bootupctl present in sysroot?"""
return os.path.exists(join_paths(sysroot, "/usr/bin/bootupctl"))
def run(self):
if have_bootupd(self._sysroot):
self._install_bootupd()
else:
self._move_grub_config()
self._set_kargs()
so, it's the presence of bootupd in the ostree that triggers the problem, I guess.
Ahhh man I was confused by finding it there, I didn't realize it was commented out.
Also of course I was looking at the container image quay.io/fedora/fedora-silverblue:40 but that already pulled this update, so of course it has bootupd.
If there's a way we can strengthen that to make it block bootupd despite the Recommends, that might be the best fix.
This update has been submitted for testing by walters.
This update's test gating status has been changed to 'ignored'.
This update has been pushed to testing.
walters edited this update.
Fixes https://gitlab.com/fedora/bootc/tracker/-/issues/54
This update has been submitted for stable by bodhi.
This update has been pushed to stable.
I think this broke ISO installs. We don't actually run openQA tests on rpm-ostree updates (I should fix that...), but since this went stable, the next F40 update tested failed: https://openqa.fedoraproject.org/tests/overview?distri=fedora&groupid=2&build=Update-FEDORA-2025-e38a38bec0&version=40 . The failure seems consistent - I've re-run the tests multiple times and they also failed the same on staging. In the install logs we see:
That failure is in bootupd, which is an independent project from rpm-ostree. It's not impossible that this change is related, but unlikely.
That test is for Silverblue it seems - there's an ongoing effort to enable bootupd there that might have gone wrong somehow? Although I don't see much at https://pagure.io/workstation-ostree-config/commits/f40
Hmm...oh actually, I think I may know what happened here. We changed rpm-ostree to Requires: bootc in this release. bootc Requires: bootupd. And Anaconda detects whether bootupd is installed in the image and uses it to determine whether or not it should be used.
I think for f40 I can unwind that requirement...but ugh, messy. We really should have had a different way to tell Anaconda to use bootupd (probably by querying whether it has anything in the update payloads).
Trying to re-sever the rpm-ostree -> bootc requirement will be extremely messy right now. Not impossible, but really messy. So let's look at the bootc -> bootupd link.
Actually bootc today just
Recommends: bootupd
. So we could aim to explicitly exclude that package in the f40 workstation configs.Except no wait, f40 workstation does include bootupd. What ostree ref or container image is this test case installing?
Also in a previous successful run, do the Anaconda logs show bootupd being used or not?
Note, I think what matters here is what's in the installer environment, not what's in the payload it's deploying. I'll look at the answers to the questions shortly, but my guess is this causes bootupd to be in the installer environment when it was not before. that shouldn't be hard to check once I've had lunch.
So, that is an interesting point because Anaconda required rpm-ostree, and it does now mean that bootc and bootupd will be pulled into the Anaconda environment. However...again in theory, having bootupd in the Anaconda environment itself should be irrelevant. See https://github.com/rhinstaller/anaconda/blob/91fe91ad6f03903114e6637d792b8d7caa679103/pyanaconda/modules/payloads/payload/rpm_ostree/util.py#L26 which looks at the target root, not the host environment.
The key thing should really be in the payload.
So in the 'success' case, bootc is not in the installer environment and bootupd is not in the ostree (it's a Silverblue ostree). In the 'failure' case, bootc is in the installer environment and bootupd is in the ostree.
anaconda's conditional seems to just depend on whether bootupd is in the installed system root (so, the ostree):
so, it's the presence of bootupd in the ostree that triggers the problem, I guess.
So in the 'success' case, bootc is not in the installer environment and bootupd is not in the ostree (it's a Silverblue ostree). In the 'failure' case, bootc is in the installer environment and bootupd is in the ostree.
anaconda's conditional seems to just depend on whether bootupd is in the installed system root (so, the ostree):
so, it's the presence of bootupd in the ostree that triggers the problem, I guess.
Also, it does make sense to me that bootupd was not in the ostree before, because it's explicitly commented out on the f40 branch of workstation-ostree-config: https://pagure.io/workstation-ostree-config/blob/f40/f/fedora-common-ostree.yaml#_12
If there's a way we can strengthen that to make it block bootupd despite the Recommends, that might be the best fix.
Ahhh man I was confused by finding it there, I didn't realize it was commented out.
Also of course I was looking at the container image
quay.io/fedora/fedora-silverblue:40
but that already pulled this update, so of course it has bootupd.Yes there is. I will submit a patch.
https://pagure.io/workstation-ostree-config/pull-request/600