Commit Graph

104 Commits

Author SHA1 Message Date
M.Schmidt
6607728e24 improve(lvm,cryptroot): export private key with a structurally equal naming scheme; cleanup 2024-12-13 10:01:19 +01:00
M.Schmidt
590b75fd6f improve(lvm+cryptroot extensions): enable possibility for LVM on LUKS 2024-12-13 10:01:19 +01:00
M.Schmidt
3ec24d40e8 fix(lvm,cryptroot extensions): introduce another variable to keep track of the physical partition/uuid and rootdevice/uuid seperately 2024-12-13 10:01:19 +01:00
user
502e139598 New option: CRYPTROOT_MAPPER. Make ROOT_MAPPER name for CRYPTROOT changable. 2024-11-05 16:35:02 +01:00
user
817519210b Fix partitioning root UUID bug in /etc/crypttab. 2024-11-05 16:35:02 +01:00
rafael
dcff6cf71d fixes #7210 for extensions that modify rootdevice 2024-09-14 21:44:52 +02:00
Igor Pecovnik
8ee40367d4 Revert "Store list of preinstalled extensions EXTENSIONS and add VENDORPRETTYNAME to image fingerprint"
This reverts commit 9af94d5977.
2024-09-08 16:22:02 +02:00
Igor Pecovnik
9af94d5977 Store list of preinstalled extensions EXTENSIONS and add VENDORPRETTYNAME to image fingerprint 2024-09-03 22:03:04 +02:00
ColorfulRhino
b8dbe607e5 partitioning: Declare rootpart as local variable
The others like `uefipart` and `bootpart` are also defined as local var. Establish consistency.
2024-08-05 20:39:57 +02:00
ColorfulRhino
1092d60d3e partitioning: Set correct partition type UUID for root filesystem
Previously, only the type "Generic Linux filesystem" was used.
Use the correct type "Linux root ($ARCHITECTURE)" for the root filesystem.
2024-08-05 20:39:57 +02:00
ColorfulRhino
a027c4be37 partitioning: Add check to use legacy BIOS boot partition with GPT only, not MBR
Exit with an error message if trying to add this partition to an MBR disk.
2024-08-05 20:39:57 +02:00
ColorfulRhino
01d8716bf9 partitioning: Output the partitioning options to the debug log 2024-08-05 20:39:57 +02:00
ColorfulRhino
0b6d61f4c4 partitioning: Add some comments to improve readbility 2024-08-05 20:39:57 +02:00
ColorfulRhino
d638276fcc partitioning: Reduce commit mount option for btrfs and ext4 to 120
Mount option `commit=600` (=10 minutes) seems pretty high.
Reduce it to 120 for ext4 (default for ext4 is `commit=5`)
and 120 for btrfs (default for btrfs is `commit=30`) to reduce
risk of losing data on power loss.

Sources:
- https://www.kernel.org/doc/html/latest/admin-guide/ext4.html
- https://wiki.archlinux.org/title/Btrfs#Commit_interval
- https://btrfs.readthedocs.io/en/latest/Administration.html
2024-07-09 11:31:51 +02:00
Ricardo Pardini
1ff783d867 fix: partitioning: disable orphan_file (FEATURE_C12) for ext4 filesystems on 1.47+ e2fsprogs host
- otherwise:
  ```
  fsck.ext4 -a -C0 /dev/mmcblk0p1
  /dev/mmcblk0p1 has unsupported feature(s): FEATURE_C12
  e2fsck: Get a newer version of e2fsck!
  ```
- `orphan_file` can only be disabled on recent-enough e2fsprogs, thus check version and compare
2024-06-27 14:25:20 +02:00
Igor Pecovnik
75ce67f329 Run shellfmt to complete code 2024-06-22 19:22:24 +02:00
Alex Shumsky
bbd5699e13 partitioning: fix race condition on loop device allocation 2024-06-21 22:14:01 -04:00
JohnTheCoolingFan
7da42c8a42 Revert "Added a call to losetup -f"
This reverts commit b3d9a17a57.
2024-05-20 15:58:51 +02:00
JohnTheCoolingFan
b3d9a17a57 Added a call to losetup -f
See https://github.com/armbian/build/issues/6568
2024-05-20 08:51:39 +02:00
Ricardo Pardini
40371934d2 lib & config: give shellcheck directions (to /dev/null for dynamic, to repo-relative path for static) for all sourced references
- in preparation for tightening the shellcheck severity level
  - it needs to be able to follow all sources; dynamic ones are ignored, static ones need root-relative prefix
2024-05-19 19:41:35 +02:00
Igor
33906869c0 Improve loop devices management by allocating random free device instead of next one (#6345) 2024-03-06 01:09:54 +01:00
BlitzKrieg
85eced8d64 Adds --partscan parameter to losetup command for forcing the kernel to scan partition table. #6333 2024-03-02 21:00:55 +01:00
Rafael
94d4c007d1 Feature lvm support (#6204)
* using the configured volume group name
* added LVM support
* ensuring /boot never on LVM volume, created hook to setup root device
* preparing root device via extension, not assuming any particular partition for root
* using tab spacing
* using global parameter to require a boot partition
* using boot require, moving cryptroot code to extension
* adds crypt image suffix
---------

Co-authored-by: rafael <rvalle@privaz.io>
2024-02-01 20:01:58 +01:00
Alex Shumsky
eee0d33bbc partitioning: allow to disable boot partition if bootloader known to support rootfs 2024-01-22 21:18:15 +01:00
Alex Shumsky
86a743ad83 partitioning: do not force image size estimation for (potentially) compressed btrfs rootfs 2024-01-22 21:18:15 +01:00
Igor Pecovnik
565a33d62d Add SWITCH to define XZ compression as a parameter and leave defaults as is 2024-01-17 13:08:13 +01:00
Ricardo Pardini
c434c3613c core: semantic change, KERNELSOURCE must be explicitly 'none' to avoid kernel compilation (not unset) 2024-01-07 18:58:36 +01:00
Gunjan Gupta
1c3b7ec36c Raspberry Pi: Added rpi5 support and removed flash-kernel dependency (#6039)
* Disable flash kernel and rearrage functions inorder they are executed
* Add board config for raspberry pi 5
* Change config.txt and cmdline.txt to match official rpi version
* Install Rpi related packages from Armbian repository
* Remove ddk branch aka RASPI (Ubuntu) kernel
* We still have unsupported releases, but for different reason
* Lower kernel output at booting to meet our other builds. Default is too mouch for end user
* Add more info where we import packages
* Add testing targets
---------
Co-authored-by: Igor <igor@armbian.com>
Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com>
2023-12-17 16:48:22 +01:00
Gunjan Gupta
c45a63e63b Add option to include home directory in the final image 2023-11-21 20:29:37 +05:30
Igor
6bcf74f2e7 Anonymize default build engine settings (#5865)
Improve naming confusion by changing build framework defaults. Set framework defaults VENDOR to Armbian-unofficial for self build images. Unsupported boards and unsupported distribution have framework defaults, except VENDOR and VENDORURL. We build images with predefined values.
2023-11-15 22:00:21 +01:00
Ricardo Pardini
28b87b0641 initrd: cache: add /etc/modprobe.d to cache key (eg for MODULES_BLACKLIST) 2023-11-15 00:32:16 +01:00
Ricardo Pardini
46728e94f2 rootfs-to-image: create_image_from_sdcard_rootfs(): ignore /home during rsync; fixes #5864 2023-11-06 16:17:21 +01:00
Ricardo Pardini
1f436bddba initrd: if debugging, don't filter update-initramfs output (errors might be lurking in there) 2023-10-31 08:08:06 +01:00
Ricardo Pardini
bba856bb57 partitioning: efi: introduce UEFI_MOUNT_POINT_SKIP_FSTAB, if set to yes, skips /etc/fstab entry for the ESP
- sometimes we just don't want the ESP listed in the fstab, otherwise works the same
2023-10-14 00:03:12 +02:00
Ricardo Pardini
11df81403d config/lib/exts: run shellfmt
- `bash lib/tools/shellfmt.sh`
- mostly spacing changes that accumulated over time
2023-08-22 14:28:14 +02:00
Ricardo Pardini
7008b14cd5 RFC: partitioning: stop disabling 64bit and metadata_csum for mkfs.ext4
- this has been a contention point for many years
2023-08-22 08:16:24 +02:00
Ricardo Pardini
2142f73b97 hashed-OCI-revisioned-debs: introduce "reversioning" of .deb packages
> tl-dr:
> - maximize OCI cache hit ratio across nightlies/releases/PRs/etc;
> - publish simple `Version:`'s that don't include a crazy hash in repo and images
> - introduce `output/packages-hashed` directory
> - radically change the `output/debs` directory structure

- simplify artifact's `prepare_version()` method for `deb` and `deb-tar` artifacts:
  - `artifact_base_dir` and `artifact_final_file` will now be auto-calculated; thus removed from each artifact (except `rootfs`)
  - `artifact_deb_repo` ("global", "jammy", "bookworm") is now required; "global" means common across all RELEASES
  - `artifact_deb_arch` is now required, "all" is arch-independent, otherwise use `${ARCH}`
  - `artifact_map_debs` is now auto-calculated based on the above, and shouldn't be specified manually
  - `artifact_final_version_reversioned` is optional, and can force the final version of the artifact (specific for the `base-files` case)
  - artifacts that need special handling for reversioning can add function names to `artifact_debs_reversion_functions` array (`base-files` and `bsp-cli` cases)
  - artifacts `prepare_version()` should set `artifact_version`, but _never_ include it in other variables; `artifact_version` is now changed by framework after `prepare_version()` returns
- no longer use/refer/mention `${REVISION}` when building packages. All packages should be `${REVISION}`-agnostic.
- `${REVISION}` (actually, `artifact_final_version_reversioned`) will be automatically swapped in the `control` file during reversioning
- `fakeroot_dpkg_deb_build()` now takes exactly two arguments: the directory to pack, and the deb ID (key of `artifact_map_packages` dict); add this change in all the artifact's code for this
- `obtain_complete_artifact()`:
  - automatically adds `-Rxxxx` "revisioning-hash" to `artifact_version`, by hashing the revisioning functions and any `artifact_debs_reversion_functions` set
  - calculates more complex subdirectory paths for both the `output/packages-hashed` and `output/debs`/`output/debs-beta` directories
    - with the new subdirectories we can be sure a re-version is already done correctly and can skip it (eg, for partial `download-debs` re-runs)
    - in the future we can automatically clean/remove old versions that are no longer relevant based on the dir structure
    - exports a lot more information to JSON, including the new subdirectory paths
  - comment-out code that implemented `skip_unpack_if_found_in_caches`, I'm very unsure why we had this in the first place
- `obtain_artifact_from_remote_cache()`
  - for `deb` type artifacts, OCI won't preserve the subdirectory structure, so move downloaded files to the correct subdirectory manually
  - this is not needed for `deb-tar`, since that can preserve the dir structure itself
- introduce `artifacts-reversion.sh` and its main function `artifact_reversion_for_deployment()`
  - this has the logic for reversioning .deb's, by `ar`-unpacking them, changing `control.tar` (and possibly `data.tar`), handling `.xz` compression, etc.
  - also handles hashing those functions, for consistency. Any changes in reversioning code actually change the artifact itself so we're not caught by surprise
  - by default, it changes `control` file only:
    - replace `Version:` (which is the hash-version originally) with `artifact_final_version_reversioned` (which is mostly just `${REVISION}`)
    - add a custom field `Armbian-Original-Hash:` with the original hash-version
  - `artifact_reversion_for_deployment()` is called by
    - new CLI wrapper `cli_obtain_complete_artifact()`, used for CLI building of specific artifact, but also for `download-artifact`
    - `build_artifact_for_image()` used during image build
- `armbian-bsp-cli-deb.sh`: move `${REVISION}` related stuff from the main package build to new reversioning functions.
- `artifact-armbian-base-files.sh`: move `${REVISION}` related stuff from the main package build to new reversioning functions.
- `kernel`:
  - add some custom fields to `DEBIAN/control`:
    - `Armbian-Kernel-Version:` / `Armbian-Kernel-Version-Family:` (for future use: cleanup of usage of `Source: ` field which should be removed)
  - declutter the `Description:` field, moving long description out of the first line
  - obtain `IMAGE_INSTALLED_KERNEL_VERSION` from the reversioned deb (this is still a hack and has not been fixed)
- `uboot`:
  - declutter the `Description:` field, moving long description out of the first line
  - use the reversioned .deb when deploying u-boot to the image
- `main_default_build_packages()` now stores reversioned values and complete paths to reversioned .deb's
- `list_installed_packages()` now compares custom field `Armbian-Original-Hash: `, and not the `Version:` to make sure debs in the image are the ones we want
- `install_artifact_deb_chroot()` is a new wrapper around `install_deb_chroot()` for easy handling of reversioned debs
  - use it everywhere `install_deb_chroot()` was used in `distro-agnostic.sh` and `distro-specific.sh`
2023-08-12 09:58:32 +02:00
Ricardo Pardini
ed02ac2447 initrd: fix undeclared variable update_initramfs_cmd 2023-08-09 11:20:28 +02:00
Gunjan Gupta
154fdb83f9 Fix cryptroot setup with flash-kernel and ssh unlock on newer dropbear versions 2023-08-02 08:30:27 +02:00
Gunjan Gupta
37d79f1ee2 Fix cache miss detection for initrd
We were not monitoring /usr/share/initramfs-tools before where most
of extra hooks gets installed. While testing builds with CRYPTROOT
I created build with dropbear ssh key unlock support first and then
went for password only, but it still used initrd image with dropbear
files and older keys. Including /usr/share/initramfs-tools fixes the
same. Also as dropbear keys were autogenerated, they needed to be
monitored as well.
2023-07-22 02:18:49 +05:30
Gunjan Gupta
2bf0ddc034 Fix intermitent loop device not available error when CRYPTROOT_ENABLE is enabled 2023-07-22 02:18:49 +05:30
Ricardo Pardini
a4c703dfb0 partitioning: declare global readonly ROOT_PART_UUID for later hooks
- re-use local `root_part_uuid` instead of running blkid multiple times
2023-07-18 14:21:01 +02:00
Ricardo Pardini
c5fe25aa17 partitioning: move hook call image_specific_armbian_env_ready() after the end of related manipulations
- depending on rootpart/extlinux/boot.ini it might not have run or had any effect
2023-07-04 03:09:07 +05:30
Ricardo Pardini
0a5740040e rootfs-to-image: find, warn, and remove dangling symlinks in FAT32 /boot 2023-05-24 19:23:37 +02:00
Ricardo Pardini
2e570b9473 initrd: when cache hit, don't preserve perms during deploy, otherwise FAT32 /boot gags 2023-05-21 13:59:26 +02:00
Igor Velkov
9b68c4e42a Add support for nilfs2 fs based image
Signed-off-by: Igor Velkov <iav@iav.lv>
2023-05-16 08:51:05 +02:00
Ricardo Pardini
b18cd8aaf4 logging: curb some warnings into info's 2023-05-09 10:04:57 +02:00
Ricardo Pardini
c692676da7 kernel-debs: postinst/etc: fix: unify is_boot_dev_vfat() function; make work during image build too via env var
- determinining if `/boot` is vfat is more complex than it looks;
  - use a common function to unify across all scripts
- during image build, the kernel install is always done with a non-fat, non-mounted /boot
  - use a special variable passed through the env so unified function can know ahead of time
  - introduce `extra_apt_envs` param to `chroot_sdcard_apt_get()`; array with extra environment vars
- if vfat /boot:
  - `linux-update-symlinks` is not called (Debian would just choke on this)
  - don't even try to symlink Armbian stuff, move instead
- more/better logging board-side (unfortunately mixed in all the set -x debugs)
2023-05-01 22:46:43 +02:00
Werner
945bb6dff9 Use native xz 2023-04-18 09:29:17 +02:00
Ricardo Pardini
7ea8e040d0 cli: flash: introduce flash CLI command; introduce hook post_build_image_write 2023-04-15 22:43:06 +02:00