Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot upgrade recovery parition to Kairos 3.2.4 because of a warning for Kairos 3.3.x #3092

Open
robarnold opened this issue Dec 19, 2024 · 7 comments
Open

Cannot upgrade recovery parition to Kairos 3.2.4 because of a warning for Kairos 3.3.x #3092

robarnold opened this issue Dec 19, 2024 · 7 comments
Labels
bug Something isn't working documentation Improvements or additions to documentation unconfirmed

Comments

@robarnold
Copy link
Contributor

Kairos version:

PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

KAIROS_ID="kairos"
KAIROS_FAMILY="ubuntu"
KAIROS_VARIANT="standard"
KAIROS_TARGETARCH="amd64"
KAIROS_SOFTWARE_VERSION_PREFIX="k3s"
KAIROS_VERSION="v3.2.4-v1.31.3-k3s1"
KAIROS_REGISTRY_AND_ORG="quay.io/kairos"
KAIROS_GITHUB_REPO="kairos-io/kairos"
KAIROS_ID_LIKE="kairos-standard-ubuntu-24.04"
KAIROS_VERSION_ID="v3.2.4-v1.31.3-k3s1"
KAIROS_IMAGE_REPO="quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1"
KAIROS_FLAVOR_RELEASE="24.04"
KAIROS_HOME_URL="https://github.com/kairos-io/kairos"
KAIROS_BUG_REPORT_URL="https://github.com/kairos-io/kairos/issues"
KAIROS_NAME="kairos-standard-ubuntu-24.04"
KAIROS_PRETTY_NAME="kairos-standard-ubuntu-24.04 v3.2.4-v1.31.3-k3s1"
KAIROS_IMAGE_LABEL="24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1"
KAIROS_ARTIFACT="kairos-ubuntu-24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3+k3s1"
KAIROS_FLAVOR="ubuntu"
KAIROS_MODEL="generic"
KAIROS_RELEASE="v3.2.4"
KAIROS_SOFTWARE_VERSION="v1.31.3+k3s1"

CPU architecture, OS, and Version:
AMD64, Ubuntu 24.04

Describe the bug
After upgrading from 3.1.2 to 3.2.4 and rebooting, I wanted to upgrade the recovery partition next. When I went to upgrade the recovery partition, I got this error:

kairos@node01:~$ sudo kairos-agent upgrade --recovery --source oci:quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1
warning: skipping /etc/kairos/branding/grubmenu.cfg (extension).
warning: skipping /etc/kairos/branding/install_text (extension).
warning: skipping /etc/kairos/branding/interactive_install_text (extension).
warning: skipping /etc/kairos/branding/recovery_text (extension).
warning: skipping /etc/kairos/branding/reset_text (extension).
warning: skipping /etc/kairos/versions.yaml because it has no valid header
warning: skipping /oem/.userdata_load (extension).
2024-12-19T21:54:08Z INF Kairos Agent version=v2.15.5
2024-12-19T21:54:08Z INF creating a runtime
2024-12-19T21:54:08Z INF detecting boot state
2024-12-19T21:54:08Z INF Boot Mode boot_mode=active_boot
2024-12-19T21:54:08Z INF Boot in uki mode result=false
No users found in any stage that are part of the 'admin' group.
In Kairos 3.3.x we no longer ship a default hardcoded user with the system configs and require users to provide their own user.Please provide at least 1 user that is part of the 'admin' group(for sudo) with your cloud configs.If you still want to continue without creating any users in the system, set 'install.nousers: true' to be in the config in order to allow a system with no users.

To Reproduce
On 3.2.4, run sudo kairos-agent upgrade --recovery --source oci:quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1

Expected behavior
Recovery partition is updated successfully using that command

Additional context
I don't know how to check the version of the recovery partition, in case that helps.
@robarnold robarnold added bug Something isn't working triage Add this label to issues that should be triaged and prioretized in the next planning call unconfirmed labels Dec 19, 2024
@Itxaka Itxaka moved this to Under review 🔍 in 🧙Issue tracking board Dec 23, 2024
@Itxaka Itxaka moved this from Under review 🔍 to Todo 🖊 in 🧙Issue tracking board Dec 23, 2024
@Itxaka
Copy link
Member

Itxaka commented Dec 23, 2024

umm, this seems to work on my test system:

/etc/kairos-release

root@localhost:~# cat /etc/kairos-release 
KAIROS_ID="kairos"
KAIROS_FAMILY="ubuntu"
KAIROS_VARIANT="standard"
KAIROS_TARGETARCH="amd64"
KAIROS_SOFTWARE_VERSION_PREFIX="k3s"
KAIROS_VERSION="v3.2.4-v1.31.3-k3s1"
KAIROS_REGISTRY_AND_ORG="quay.io/kairos"
KAIROS_GITHUB_REPO="kairos-io/kairos"
KAIROS_ID_LIKE="kairos-standard-ubuntu-24.04"
KAIROS_VERSION_ID="v3.2.4-v1.31.3-k3s1"
KAIROS_IMAGE_REPO="quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1"
KAIROS_FLAVOR_RELEASE="24.04"
KAIROS_HOME_URL="https://github.com/kairos-io/kairos"
KAIROS_BUG_REPORT_URL="https://github.com/kairos-io/kairos/issues"
KAIROS_NAME="kairos-standard-ubuntu-24.04"
KAIROS_PRETTY_NAME="kairos-standard-ubuntu-24.04 v3.2.4-v1.31.3-k3s1"
KAIROS_IMAGE_LABEL="24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1"
KAIROS_ARTIFACT="kairos-ubuntu-24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3+k3s1"
KAIROS_FLAVOR="ubuntu"
KAIROS_MODEL="generic"
KAIROS_RELEASE="v3.2.4"
KAIROS_SOFTWARE_VERSION="v1.31.3+k3s1"

/oem/90_custom.yaml

#cloud-config

# Sources:
# - /tmp/kairos-install-259104618/kairos-event-install-data.yaml
# - reader
# - cmdline

install:
    device: /dev/vda
name: Config generated by the installer
stages:
    initramfs:
        - users:
            kairos:
                groups:
                    - admin
                name: kairos
                passwd: kairos

Run kairos-agent upgrade --recovery --source oci:quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1

root@localhost:~# kairos-agent --debug upgrade --recovery --source oci:quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1
warning: skipping /etc/kairos/branding/grubmenu.cfg (extension).
warning: skipping /etc/kairos/branding/install_text (extension).
warning: skipping /etc/kairos/branding/interactive_install_text (extension).
warning: skipping /etc/kairos/branding/recovery_text (extension).
warning: skipping /etc/kairos/branding/reset_text (extension).
warning: skipping /etc/kairos/versions.yaml because it has no valid header
WARNING: jsonschema: '' does not validate with file:///root/schema.json#/required: missing properties: 'users'
2024-12-23T10:49:44Z INF Kairos Agent version=v2.15.5
2024-12-23T10:49:44Z DBG Kairos Agent version={"git_commit":"none","go_version":"go1.23.4","version":"v2.15.5"}
2024-12-23T10:49:44Z INF creating a runtime
2024-12-23T10:49:44Z INF detecting boot state
2024-12-23T10:49:44Z INF Boot Mode boot_mode=active_boot
2024-12-23T10:49:44Z INF Boot in uki mode result=false
2024-12-23T10:49:44Z DBG Loaded config: &config.Config{
  Install: &config.Install{
    Auto: false,
    Reboot: false,
    NoFormat: false,
    Device: "/dev/vda",
    Poweroff: false,
    GrubOptions: map[string]string(nil), // p0
    Bundles: nil,
    Encrypt: nil,
    SkipEncryptCopyPlugins: false,
    Env: nil,
    Source: "",
    EphemeralMounts: nil,
    BindMounts: nil,
    Partitions: v1.ElementalPartitions{
      BIOS: nil,
      EFI: nil,
      OEM: nil,
      Recovery: nil,
      State: nil,
      Persistent: nil,
    },
    Active: v1.Image{
      File: "",
      Label: "",
      Size: 0,
      FS: "",
      Source: nil,
      MountPoint: "",
      LoopDevice: "",
    },
    Recovery: v1.Image{
      File: "",
      Label: "",
      Size: 0,
      FS: "",
      Source: nil,
      MountPoint: "",
      LoopDevice: "",
    },
    Passive: v1.Image{
      File: "",
      Label: "",
      Size: 0,
      FS: "",
      Source: nil,
      MountPoint: "",
      LoopDevice: "",
    },
    GrubDefEntry: "",
    ExtraPartitions: nil,
    ExtraDirsRootfs: nil,
    Force: false,
    NoUsers: false,
  },
  Config: collector.Config{
    Sources: []string{
      "/oem/90_custom.yaml",
      "reader",
    },
    Values: collector.ConfigValues{
      "install": collector.ConfigValues{
        "device": "/dev/vda",
      },
      "name": "Config generated by the installer",
      "stages": collector.ConfigValues{
        "initramfs": []interface {}{
          collector.ConfigValues{
            "users": collector.ConfigValues{
              "kairos": collector.ConfigValues{
                "groups": []interface {}{
                  "admin",
                },
                "name": "kairos",
                "passwd": "kairos",
              },
            },
          },
        },
      },
      "upgrade": collector.ConfigValues{
        "entry": "recovery",
        "recovery-system": collector.ConfigValues{
          "uri": "oci:quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1",
        },
        "system": collector.ConfigValues{
          "uri": "oci:quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1",
        },
      },
    },
  },
  ConfigURL: "",
  Options: p0,
  FailOnBundleErrors: false,
  Bundles: nil,
  GrubOptions: p0,
  Env: nil,
  Debug: false,
  Strict: false,
  CloudInitPaths: nil,
  EjectCD: false,
  Fs: &vfs.osfs{}, // p1
  Mounter: &mount.Mounter{
    mounterPath: "/usr/bin/mount",
    withSystemd: nil,
    trySystemd: true,
    withSafeNotMountedBehavior: true,
  },
  Runner: &v1.RealRunner{}, // p2
  Syscall: &v1.RealSyscall{},
  CloudInitRunner: &cloudinit.YipCloudInitRunner{
    exec: &executor.DefaultExecutor{
      plugins: []executor.Plugin{
        plugins.DNS,
        plugins.Download,
        plugins.Git,
        plugins.Entities,
        plugins.EnsureDirectories,
        plugins.EnsureFiles,
        plugins.Commands,
        plugins.DeleteEntities,
        plugins.Hostname,
        plugins.Sysctl,
        plugins.User,
        plugins.SSH,
        plugins.LoadModules,
        plugins.Timesyncd,
        plugins.Systemctl,
        plugins.Environment,
        plugins.SystemdFirstboot,
        plugins.DataSources,
        plugins.Layout,
      },
      conditionals: []executor.Plugin{
        plugins.NodeConditional,
        plugins.IfConditional,
      },
      modifier: ,
    },
    fs: p1,
    console: &cloudinit.cloudInitConsole{
      runner: p2,
    },
  },
  ImageExtractor: v1.OCIImageExtractor{},
  Platform: &v1.Platform{
    OS: "linux",
    Arch: "x86_64",
    GolangArch: "amd64",
  },
  Cosign: false,
  Verify: false,
  CosignPubKey: "",
  Arch: "x86_64",
  SquashFsCompressionConfig: []string{},
  SquashFsNoCompression: true,
  UkiMaxEntries: 3,
}
2024-12-23T10:49:44Z DBG Scanning for disks path=/sys/block/
2024-12-23T10:49:44Z DBG Reading file file=loop0
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop0/size
2024-12-23T10:49:44Z DBG Reading disk file file=/sys/block/loop0
2024-12-23T10:49:44Z DBG Reading file file=loop1
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop1/size
2024-12-23T10:49:44Z DBG Reading file file=loop2
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop2/size
2024-12-23T10:49:44Z DBG Reading file file=loop3
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop3/size
2024-12-23T10:49:44Z DBG Reading file file=loop4
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop4/size
2024-12-23T10:49:44Z DBG Reading file file=loop5
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop5/size
2024-12-23T10:49:44Z DBG Reading file file=loop6
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop6/size
2024-12-23T10:49:44Z DBG Reading file file=loop7
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/loop7/size
2024-12-23T10:49:44Z DBG Reading file file=sr0
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/sr0/size
2024-12-23T10:49:44Z DBG Reading disk file file=/sys/block/sr0
2024-12-23T10:49:44Z DBG Reading file file=vda
2024-12-23T10:49:44Z DBG Reading disk size path=/sys/block/vda/size
2024-12-23T10:49:44Z DBG Reading disk file file=/sys/block/vda
2024-12-23T10:49:44Z DBG Reading partition file file=vda1
2024-12-23T10:49:44Z DBG Reading size file file=/sys/block/vda/vda1/size
2024-12-23T10:49:44Z DBG Reading mounts file file=/proc/mounts
2024-12-23T10:49:44Z DBG Reading partition file file=vda2
2024-12-23T10:49:44Z DBG Reading size file file=/sys/block/vda/vda2/size
2024-12-23T10:49:44Z DBG Reading mounts file file=/proc/mounts
2024-12-23T10:49:44Z DBG Reading partition file file=vda3
2024-12-23T10:49:44Z DBG Reading size file file=/sys/block/vda/vda3/size
2024-12-23T10:49:44Z DBG Reading mounts file file=/proc/mounts
2024-12-23T10:49:44Z DBG Reading partition file file=vda4
2024-12-23T10:49:44Z DBG Reading size file file=/sys/block/vda/vda4/size
2024-12-23T10:49:44Z DBG Reading mounts file file=/proc/mounts
2024-12-23T10:49:44Z DBG Reading partition file file=vda5
2024-12-23T10:49:44Z DBG Reading size file file=/sys/block/vda/vda5/size
2024-12-23T10:49:44Z DBG Reading mounts file file=/proc/mounts
2024-12-23T10:49:45Z DBG Calculated size (3897MB) is higher than specified/default size (3072MB)
2024-12-23T10:49:45Z INF Setting image size to 3897MB
2024-12-23T10:49:45Z DBG Loaded upgrade spec: &v1.UpgradeSpec{
  Entry: "recovery",
  Active: v1.Image{
    File: "/run/initramfs/cos-state/cOS/transition.img",
    Label: "COS_ACTIVE",
    Size: 3072,
    FS: "ext2",
    Source: &v1.ImageSource{
      source: "quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1",
      srcType: "oci",
    },
    MountPoint: "/run/cos/transition",
    LoopDevice: "",
  },
  Recovery: v1.Image{
    File: "/run/cos/recovery/cOS/transition.img",
    Label: "COS_SYSTEM",
    Size: 3897,
    FS: "ext2",
    Source: &v1.ImageSource{
      source: "quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1",
      srcType: "oci",
    },
    MountPoint: "/run/cos/transition",
    LoopDevice: "",
  },
  GrubDefEntry: "",
  Reboot: false,
  PowerOff: false,
  ExtraDirsRootfs: nil,
  Passive: v1.Image{
    File: "/run/initramfs/cos-state/cOS/passive.img",
    Label: "COS_PASSIVE",
    Size: 3072,
    FS: "ext2",
    Source: &v1.ImageSource{
      source: "/run/initramfs/cos-state/cOS/transition.img",
      srcType: "file",
    },
    MountPoint: "",
    LoopDevice: "",
  },
  Partitions: v1.ElementalPartitions{
    BIOS: nil,
    EFI: nil,
    OEM: &types.Partition{
      Name: "vda2",
      FilesystemLabel: "COS_OEM",
      Size: 64,
      FS: "ext4",
      Flags: nil,
      UUID: "8bfa06f9-ca4f-56dc-90c9-49cf20f4f45e",
      MountPoint: "/oem",
      Path: "/dev/vda2",
      Disk: "/dev/vda",
    },
    Recovery: &types.Partition{
      Name: "vda3",
      FilesystemLabel: "COS_RECOVERY",
      Size: 3238,
      FS: "ext4",
      Flags: nil,
      UUID: "768fbd69-b2a9-5eab-902c-db8745b4767c",
      MountPoint: "/run/cos/recovery",
      Path: "/dev/vda3",
      Disk: "/dev/vda",
    },
    State: &types.Partition{
      Name: "vda4",
      FilesystemLabel: "COS_STATE",
      Size: 5557,
      FS: "ext4",
      Flags: nil,
      UUID: "843bfe4e-83b2-5b42-ba6d-a47e3ba1262e",
      MountPoint: "/run/initramfs/cos-state",
      Path: "/dev/vda4",
      Disk: "/dev/vda",
    },
    Persistent: &types.Partition{
      Name: "vda5",
      FilesystemLabel: "COS_PERSISTENT",
      Size: 32098,
      FS: "ext4",
      Flags: nil,
      UUID: "85c39d0f-4867-5227-8334-f5eec606d9eb",
      MountPoint: "/run/cos/persistent",
      Path: "/dev/vda5",
      Disk: "/dev/vda",
    },
  },
  State: &v1.InstallState{
    Date: "2024-12-23T10:45:15Z",
    Partitions: map[string]*v1.PartitionState{
      "oem": &v1.PartitionState{
        FSLabel: "COS_OEM",
        Images: map[string]*v1.ImageState(nil), // p0
      },
      "persistent": &v1.PartitionState{
        FSLabel: "COS_PERSISTENT",
        Images: p0,
      },
      "recovery": &v1.PartitionState{
        FSLabel: "COS_RECOVERY",
        Images: map[string]*v1.ImageState{
          "recovery": &v1.ImageState{
            Source: &v1.ImageSource{
              source: "quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1",
              srcType: "oci",
            },
            SourceMetadata: nil,
            Label: "COS_SYSTEM",
            FS: "ext2",
          },
        },
      },
      "state": &v1.PartitionState{
        FSLabel: "COS_STATE",
        Images: map[string]*v1.ImageState{
          "active": &v1.ImageState{
            Source: &v1.ImageSource{
              source: "quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1",
              srcType: "oci",
            },
            SourceMetadata: nil,
            Label: "COS_ACTIVE",
            FS: "ext2",
          },
          "passive": &v1.ImageState{
            Source: &v1.ImageSource{
              source: "/run/rootfsbase",
              srcType: "dir",
            },
            SourceMetadata: nil,
            Label: "COS_ACTIVE",
            FS: "ext2",
          },
        },
      },
    },
  },
}
2024-12-23T10:49:45Z DBG Mounting partition COS_STATE
2024-12-23T10:49:45Z DBG Mounting partition COS_RECOVERY
2024-12-23T10:49:45Z DBG mounting persistent partition
2024-12-23T10:49:45Z DBG Mounting partition COS_PERSISTENT
2024-12-23T10:49:45Z INF Applying 'before-upgrade' hook
2024-12-23T10:49:45Z INF Running before-upgrade hook
2024-12-23T10:49:45Z DBG Cloud-init paths set to [/system/oem /oem/ /usr/local/cloud-config/]
2024-12-23T10:49:45Z INF Running stage: before-upgrade.before

2024-12-23T10:49:45Z INF Done executing stage 'before-upgrade.before'

2024-12-23T10:49:45Z INF Running stage: before-upgrade

2024-12-23T10:49:45Z INF Done executing stage 'before-upgrade'

2024-12-23T10:49:45Z INF Running stage: before-upgrade.after

2024-12-23T10:49:45Z INF Done executing stage 'before-upgrade.after'

2024-12-23T10:49:45Z INF Running stage: before-upgrade.before

2024-12-23T10:49:45Z INF Done executing stage 'before-upgrade.before'

2024-12-23T10:49:45Z INF Running stage: before-upgrade

2024-12-23T10:49:45Z INF Done executing stage 'before-upgrade'

2024-12-23T10:49:45Z INF Running stage: before-upgrade.after

2024-12-23T10:49:45Z INF Done executing stage 'before-upgrade.after'

2024-12-23T10:49:45Z INF deploying image quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1 to /run/cos/recovery/cOS/transition.img
2024-12-23T10:49:45Z INF Creating file system image /run/cos/recovery/cOS/transition.img with size 3897Mb
2024-12-23T10:49:45Z DBG Running cmd: 'mkfs.ext2 -L COS_SYSTEM /run/cos/recovery/cOS/transition.img'
2024-12-23T10:49:45Z DBG Mounting image COS_SYSTEM
2024-12-23T10:49:45Z DBG Opening loop control device
2024-12-23T10:49:45Z DBG Getting free loop device
2024-12-23T10:49:45Z DBG Opening loop device device=/dev/loop1
2024-12-23T10:49:45Z DBG Opening img file image=/run/cos/recovery/cOS/transition.img
2024-12-23T10:49:45Z DBG Setting loop device
2024-12-23T10:49:45Z DBG Setting loop flags
2024-12-23T10:49:45Z INF Copying quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1 source to /run/cos/transition
2024-12-23T10:50:34Z INF Finished copying quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1 into /run/cos/transition
2024-12-23T10:50:34Z DBG Extra mounts: map[/oem:/oem /run/cos/persistent:/usr/local]
2024-12-23T10:50:34Z DBG Mounting /dev to chroot
2024-12-23T10:50:34Z DBG Mounted /dev to /run/cos/transition/dev
2024-12-23T10:50:34Z DBG Mounting /dev/pts to chroot
2024-12-23T10:50:34Z DBG Mounted /dev/pts to /run/cos/transition/dev/pts
2024-12-23T10:50:34Z DBG Mounting /proc to chroot
2024-12-23T10:50:34Z DBG Mounted /proc to /run/cos/transition/proc
2024-12-23T10:50:34Z DBG Mounting /sys to chroot
2024-12-23T10:50:34Z DBG Mounted /sys to /run/cos/transition/sys
2024-12-23T10:50:34Z DBG Mounting /oem to chroot
2024-12-23T10:50:34Z DBG Mounted /oem to /run/cos/transition/oem
2024-12-23T10:50:34Z DBG Mounting /run/cos/persistent to chroot
2024-12-23T10:50:34Z DBG Mounted /run/cos/persistent to /run/cos/transition/usr/local
2024-12-23T10:50:34Z DBG No files relabelling as SELinux utilities are not found
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/usr/local from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/oem from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/sys from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/proc from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/dev/pts from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/dev from chroot
2024-12-23T10:50:34Z INF Applying 'after-upgrade-chroot' hook
2024-12-23T10:50:34Z DBG Extra mounts: map[/oem:/oem /run/cos/persistent:/usr/local]
2024-12-23T10:50:34Z DBG Mounting /dev to chroot
2024-12-23T10:50:34Z DBG Mounted /dev to /run/cos/transition/dev
2024-12-23T10:50:34Z DBG Mounting /dev/pts to chroot
2024-12-23T10:50:34Z DBG Mounted /dev/pts to /run/cos/transition/dev/pts
2024-12-23T10:50:34Z DBG Mounting /proc to chroot
2024-12-23T10:50:34Z DBG Mounted /proc to /run/cos/transition/proc
2024-12-23T10:50:34Z DBG Mounting /sys to chroot
2024-12-23T10:50:34Z DBG Mounted /sys to /run/cos/transition/sys
2024-12-23T10:50:34Z DBG Mounting /oem to chroot
2024-12-23T10:50:34Z DBG Mounted /oem to /run/cos/transition/oem
2024-12-23T10:50:34Z DBG Mounting /run/cos/persistent to chroot
2024-12-23T10:50:34Z DBG Mounted /run/cos/persistent to /run/cos/transition/usr/local
2024-12-23T10:50:34Z INF Running after-upgrade-chroot hook
2024-12-23T10:50:34Z DBG Cloud-init paths set to [/system/oem /oem/ /usr/local/cloud-config/]
2024-12-23T10:50:34Z INF Running stage: after-upgrade-chroot.before

2024-12-23T10:50:34Z INF Done executing stage 'after-upgrade-chroot.before'

2024-12-23T10:50:34Z INF Running stage: after-upgrade-chroot

2024-12-23T10:50:34Z INF Done executing stage 'after-upgrade-chroot'

2024-12-23T10:50:34Z INF Running stage: after-upgrade-chroot.after

2024-12-23T10:50:34Z INF Done executing stage 'after-upgrade-chroot.after'

2024-12-23T10:50:34Z INF Running stage: after-upgrade-chroot.before

2024-12-23T10:50:34Z INF Done executing stage 'after-upgrade-chroot.before'

2024-12-23T10:50:34Z INF Running stage: after-upgrade-chroot

2024-12-23T10:50:34Z INF Done executing stage 'after-upgrade-chroot'

2024-12-23T10:50:34Z INF Running stage: after-upgrade-chroot.after

2024-12-23T10:50:34Z INF Done executing stage 'after-upgrade-chroot.after'

2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/usr/local from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/oem from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/sys from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/proc from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/dev/pts from chroot
2024-12-23T10:50:34Z DBG Unmounting /run/cos/transition/dev from chroot
2024-12-23T10:50:34Z DBG Unmounting image COS_SYSTEM
2024-12-23T10:50:35Z DBG Opening loop device device=/dev/loop1
2024-12-23T10:50:35Z DBG Clearing loop device
2024-12-23T10:50:35Z INF Moving /run/cos/recovery/cOS/transition.img to /run/cos/recovery/cOS/recovery.img
2024-12-23T10:50:35Z INF Finished moving /run/cos/recovery/cOS/transition.img to /run/cos/recovery/cOS/recovery.img
2024-12-23T10:50:35Z INF Applying 'after-upgrade' hook
2024-12-23T10:50:35Z INF Running after-upgrade hook
2024-12-23T10:50:35Z DBG Cloud-init paths set to [/system/oem /oem/ /usr/local/cloud-config/]
2024-12-23T10:50:35Z INF Running stage: after-upgrade.before

2024-12-23T10:50:35Z INF Done executing stage 'after-upgrade.before'

2024-12-23T10:50:35Z INF Running stage: after-upgrade

2024-12-23T10:50:35Z DBG Generating op for stage 'Boot assessment and branding.Mount state'
2024-12-23T10:50:35Z DBG Generating op for stage 'Boot assessment and branding.Set upgrade sentinel'
2024-12-23T10:50:35Z DBG Generating op for stage 'Boot assessment and branding.Hook boot assessment grub configuration'
2024-12-23T10:50:35Z DBG Generating op for stage 'Boot assessment and branding.Grub branding'
2024-12-23T10:50:35Z DBG Generating op for stage 'Boot assessment and branding.Add boot assessment grub configuration'
2024-12-23T10:50:35Z DBG Generating op for stage 'Boot assessment and branding.umount state'
2024-12-23T10:50:35Z DBG Generating op for stage 'Kcrypt.Update plugins'
2024-12-23T10:50:35Z DBG Reading '/system/oem/08_grub.yaml'
2024-12-23T10:50:35Z DBG Executing stage 'Boot assessment and branding.Mount state'
2024-12-23T10:50:35Z DBG Reading '/system/oem/21_kcrypt.yaml'
2024-12-23T10:50:35Z DBG Executing stage 'Kcrypt.Update plugins'
2024-12-23T10:50:35Z DBG running command `[ $(kairos-agent state get oem.found) == 'true' ]`
2024-12-23T10:50:35Z DBG running command `grep -vq "rd.immucore.uki" /proc/cmdline`
2024-12-23T10:50:35Z DBG If statement result 
2024-12-23T10:50:35Z INF Processing stage step 'Boot assessment and branding.Mount state'. ( commands: 1, files: 0, ... )
2024-12-23T10:50:35Z DBG Stage: schema.Stage{
  Commands: []string{
    "STATEDIR=/tmp/mnt/STATE\nSTATE=$(blkid -L COS_STATE || true)\nmkdir -p $STATEDIR || true\nmount ${STATE} $STATEDIR\n",
  },
  If: "grep -vq \"rd.immucore.uki\" /proc/cmdline",
  Name: "Mount state",
}
2024-12-23T10:50:35Z DBG running command `STATEDIR=/tmp/mnt/STATE
STATE=$(blkid -L COS_STATE || true)
mkdir -p $STATEDIR || true
mount ${STATE} $STATEDIR
`
2024-12-23T10:50:35Z DBG Empty command output
2024-12-23T10:50:35Z DBG Device field empty, skipping layout plugin
2024-12-23T10:50:36Z WRN (conditional) Skip 'Skipping stage (if statement error: failed to run [ $(kairos-agent state get oem.found) == 'true' ]: exit status 2)' stage name: Kcrypt.Update plugins
2024-12-23T10:50:36Z DBG Reading '/system/oem/08_grub.yaml'
2024-12-23T10:50:36Z DBG Executing stage 'Boot assessment and branding.Set upgrade sentinel'
2024-12-23T10:50:36Z DBG running command `grep -vq "rd.immucore.uki" /proc/cmdline`
2024-12-23T10:50:36Z DBG If statement result 
2024-12-23T10:50:36Z INF Processing stage step 'Boot assessment and branding.Set upgrade sentinel'. ( commands: 1, files: 0, ... )
2024-12-23T10:50:36Z DBG Stage: schema.Stage{
  Commands: []string{
    "grub2-editenv /tmp/mnt/STATE/boot_assessment set enable_boot_assessment=yes\n",
  },
  If: "grep -vq \"rd.immucore.uki\" /proc/cmdline",
  Name: "Set upgrade sentinel",
}
2024-12-23T10:50:36Z DBG running command `grub2-editenv /tmp/mnt/STATE/boot_assessment set enable_boot_assessment=yes
`
2024-12-23T10:50:36Z DBG Empty command output
2024-12-23T10:50:36Z DBG Device field empty, skipping layout plugin
2024-12-23T10:50:36Z DBG Reading '/system/oem/08_grub.yaml'
2024-12-23T10:50:36Z DBG Executing stage 'Boot assessment and branding.Hook boot assessment grub configuration'
2024-12-23T10:50:36Z DBG running command `! grep -q "grub_boot_assessment" /tmp/mnt/STATE/grubcustom && grep -vq "rd.immucore.uki" /proc/cmdline
`
2024-12-23T10:50:36Z WRN (conditional) Skip 'Skipping stage (if statement error: failed to run ! grep -q "grub_boot_assessment" /tmp/mnt/STATE/grubcustom && grep -vq "rd.immucore.uki" /proc/cmdline
: exit status 1)' stage name: Boot assessment and branding.Hook boot assessment grub configuration
2024-12-23T10:50:36Z DBG Reading '/system/oem/08_grub.yaml'
2024-12-23T10:50:36Z DBG Executing stage 'Boot assessment and branding.Grub branding'
2024-12-23T10:50:36Z DBG running command `[ -e "/etc/kairos/branding/grubmenu.cfg" ] && grep -vq "rd.immucore.uki" /proc/cmdline`
2024-12-23T10:50:36Z DBG If statement result 
2024-12-23T10:50:36Z INF Processing stage step 'Boot assessment and branding.Grub branding'. ( commands: 1, files: 0, ... )
2024-12-23T10:50:36Z DBG Stage: schema.Stage{
  Commands: []string{
    "cp -rfv /etc/kairos/branding/grubmenu.cfg /tmp/mnt/STATE/grubmenu",
  },
  If: "[ -e \"/etc/kairos/branding/grubmenu.cfg\" ] && grep -vq \"rd.immucore.uki\" /proc/cmdline",
  Name: "Grub branding",
}
2024-12-23T10:50:36Z DBG running command `cp -rfv /etc/kairos/branding/grubmenu.cfg /tmp/mnt/STATE/grubmenu`
2024-12-23T10:50:36Z DBG Command output: '/etc/kairos/branding/grubmenu.cfg' -> '/tmp/mnt/STATE/grubmenu'

2024-12-23T10:50:36Z DBG Device field empty, skipping layout plugin
2024-12-23T10:50:36Z DBG Reading '/system/oem/08_grub.yaml'
2024-12-23T10:50:36Z DBG Executing stage 'Boot assessment and branding.Add boot assessment grub configuration'
2024-12-23T10:50:36Z DBG running command `grep -vq "rd.immucore.uki" /proc/cmdline`
2024-12-23T10:50:36Z DBG If statement result 
2024-12-23T10:50:36Z INF Processing stage step 'Boot assessment and branding.Add boot assessment grub configuration'. ( commands: 0, files: 1, ... )
2024-12-23T10:50:36Z DBG Stage: schema.Stage{
  Files: []schema.File{
    schema.File{
      Path: "/tmp/mnt/STATE/grub_boot_assessment",
      Content: "set boot_assessment=\"/boot_assessment\"\nsearch --no-floppy --file --set=boot_assessment_blk \"${boot_assessment}\"\nif [ \"${boot_assessment_blk}\" ]; then\n  load_env -f \"(${boot_assessment_blk})${boot_assessment}\"\nfi\nif [ \"${enable_boot_assessment}\" = \"yes\" -o \"${enable_boot_assessment_always}\" = \"yes\" ]; then\n  if [ -z \"${selected_entry}\" ]; then\n    if [ \"${boot_assessment_tentative}\" = \"yes\" ]; then\n      set default=\"fallback\"\n      set extra_passive_cmdline=\"upgrade_failure\"\n    else\n      set boot_assessment_tentative=\"yes\"\n      save_env -f \"(${boot_assessment_blk})${boot_assessment}\" boot_assessment_tentative\n    fi\n  fi\nfi\n",
    },
  },
  If: "grep -vq \"rd.immucore.uki\" /proc/cmdline",
  Name: "Add boot assessment grub configuration",
}
2024-12-23T10:50:36Z DBG Creating file /tmp/mnt/STATE/grub_boot_assessment
2024-12-23T10:50:36Z DBG Device field empty, skipping layout plugin
2024-12-23T10:50:36Z DBG Reading '/system/oem/08_grub.yaml'
2024-12-23T10:50:36Z DBG Executing stage 'Boot assessment and branding.umount state'
2024-12-23T10:50:36Z DBG running command `grep -vq "rd.immucore.uki" /proc/cmdline`
2024-12-23T10:50:36Z DBG If statement result 
2024-12-23T10:50:36Z INF Processing stage step 'Boot assessment and branding.umount state'. ( commands: 1, files: 0, ... )
2024-12-23T10:50:36Z DBG Stage: schema.Stage{
  Commands: []string{
    "umount /tmp/mnt/STATE",
  },
  If: "grep -vq \"rd.immucore.uki\" /proc/cmdline",
  Name: "umount state",
}
2024-12-23T10:50:36Z DBG running command `umount /tmp/mnt/STATE`
2024-12-23T10:50:36Z DBG Empty command output
2024-12-23T10:50:36Z DBG Device field empty, skipping layout plugin
2024-12-23T10:50:36Z INF Done executing stage 'after-upgrade'

2024-12-23T10:50:36Z INF Running stage: after-upgrade.after

2024-12-23T10:50:36Z INF Done executing stage 'after-upgrade.after'

2024-12-23T10:50:36Z INF Running stage: after-upgrade.before

2024-12-23T10:50:36Z INF Done executing stage 'after-upgrade.before'

2024-12-23T10:50:36Z INF Running stage: after-upgrade

2024-12-23T10:50:36Z INF Done executing stage 'after-upgrade'

2024-12-23T10:50:36Z INF Running stage: after-upgrade.after

2024-12-23T10:50:36Z INF Done executing stage 'after-upgrade.after'

2024-12-23T10:50:36Z INF Upgrade completed
2024-12-23T10:50:36Z DBG Not unmounting image, /run/cos/transition doesn't look like mountpoint
2024-12-23T10:50:36Z DBG Unmounting partition COS_RECOVERY
2024-12-23T10:50:36Z DBG Mounting partition COS_STATE
2024-12-23T10:50:36Z DBG Running Lifecycle hook
2024-12-23T10:50:36Z DBG Finish Lifecycle hook

@Itxaka
Copy link
Member

Itxaka commented Dec 23, 2024

Usually that checks means that the installer has gone over the cloud configs in the system and has found no users with the admin group.

We now require at least 1 user in the system that its part of the admin group as that ensures that after reboot, the system will have at least 1 user that can sudo. By default this is the kairos user but it could be any other user created during install, as long as its part of the admin group it should be ok.

Can you post your config files under /oem ? Maybe we missed something there :D

@Itxaka
Copy link
Member

Itxaka commented Dec 23, 2024

If this is a blocker, you migth eb able to workaround it by setting the install.nousers: true in your /oem/90_custom.yaml as a workaround, but if the check is correct you might end up with a recovery which can be logged in.

@robarnold
Copy link
Contributor Author

I managed to work around it by adding the lock_passwd and groups fields to my cloud config:

users:
- name: kairos
  passwd: {{ kairos_password }}
  lock_passwd: true
  groups:
    - admin

I got them from an example in the documentation because I didn't see anything about Kairos 3.3 changes on the blog or releases. The kairos user already existed and had admin from when it was installed on the Node, so I don't think I need to change the initramfs portion to match, but I'm still a little confused at there being two ways to configure users.

@jimmykarily
Copy link
Contributor

The warning comes from this change which is part of kairos-agent v2.15.5 but our cloud init files still create that default user with the admin group (here).

I can see it in the image used for upgrade:

~ $ docker run --rm -it quay.io/kairos/ubuntu:24.04-standard-amd64-generic-v3.2.4-k3sv1.31.3-k3s1
root@0544e9dee5f9:/# cat /system/oem/10_accounting.yaml
name: "Default user, permissions and serial login"
stages:
  initramfs:
    - name: "Setup groups"
      ensure_entities:
      - entity: |
           kind: "group"
           group_name: "admin"
           password: "x"
           gid: 900
    - name: "Setup users"
      users:
        kairos:
          passwd: "!"
          shell: /bin/bash
          homedir: "/home/kairos"
          groups:
            - "admin"

... (more things)

So, I don't understand why it can't find the kairos user. Unless there was some other config (in /oem ?) that overwrote the user with no admin group or something.

@robarnold what do you mean by two ways to configure users ? cloud init files and user config?

@robarnold
Copy link
Contributor Author

Yeah, that you can specify users in both like this:

#cloud-config

users:
- name: "kairos"                       # The username for the user.
  passwd: "kairos"

and like this:

#cloud-config

stages:
   initramfs:
     - name: "Set user and password"
       users:
        testuser:
          groups:
          - "admin"
          passwd: "mypassword"
          shell: /bin/bash
          homedir: "/home/testuser"

Both are documented on https://kairos.io/docs/reference/configuration/ and it's not clear from the docs when to use one vs the other. So I was using the first style of users: block, but I don't know if that only affects installs or if it'll automatically update groups post-install if I update the config in /oem.

@jimmykarily
Copy link
Contributor

Top level users: and the users: key inside stages: are treated the same (applied on every boot).
We simply construct a Stage in code out of the top level key and we treat it as a "boot" stage.

You are right, it's worth documenting it.

@jimmykarily jimmykarily added documentation Improvements or additions to documentation and removed triage Add this label to issues that should be triaged and prioretized in the next planning call labels Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working documentation Improvements or additions to documentation unconfirmed
Projects
🧙Issue tracking board
Status: Todo 🖊
Milestone
No milestone
Development

No branches or pull requests
3 participants
@robarnold @Itxaka @jimmykarily