If the array exists in this location, initializing an empty array
will destroy all the data that was stored in it.
Do not initialize an empty KERNEL_DRIVERS_SKIP array if it exists.
This allows us to create an array anywhere in the script,
including the user configuration, in a secure way.
- Replace `armbian.list` with `armbian.sources`. This holds the same
information in a newer format, deb822.
- Replace HTTP with HTTPS for Armbian repositories.
* Refactor armbian-config into install as external package
- https://github.armbian.com/configng repository is always present
- package can still be removed from the install list
Since this tool will have deeper integration with Armbian, we should probably drop possibility to remove the tool from the list.
* Enable armbian-config as extension
- `post_repo_customize_image`: runs after repos have been enabled
- `post_armbian_repo_customize_image`: same, but only if Armbian repo is enabled
- both run after apt update, so packages can be directly installed from repos
- refactor `prepare_host_binfmt_qemu()` out of `prepare_host_noninteractive()` and into `rootfs/qemu-static.sh`
- further split into more functions and return early to avoid deep nesting
- implement force import and load of qemu-arm for non-armhf capable arm64 hosts (incl magic numbers)
- enhance `deploy_qemu_binary_to_chroot()` & `undeploy_qemu_binary_from_chroot()`;
- add 2nd param "caller" for better logging/tracking
- does sanity-check and preserve existing binary if it exists
- explicitly deploy/undeploy for the 3 cases:
- image: moved undeploy from `post_debootstrap_tweaks()` into image build proper for consistency
- rootfs: was leaving trash behind (since post_debootstrap_tweaks never ran for rootfs), now properly undeploys
- initrd: was already fine, just added caller info
- added `arch-test` host dependency
- ensure `arch-test ${ARCH}` works during prepare-host
- > tl,dr: "can build 32-bit armv7 armhf using Apple silicon; can use rootfs cache cross-arch reliably"
- https://tracker.debian.org/news/1579223/software-properties-removed-from-testing/
- that provides add-apt-repository, but we don't use that on Debian, hopefully
- move also cpufrequtils hack (which requires release) to specific function after userspace checks and when DISTRIBUTION is set
- `software-properties-gtk` if of course also gone, so remove that for desktop's sake
- useful for `collabora` and other experimental kernels, we don't want to have to add it to each individual board's KERNEL_TARGES one by one
- but we don't want to allow typos in BRANCH to emit very strange unrelated errors
* main-config: fix: avoid errors when BRANCH contains a dash; convert to underscore
* rockchip64_common: shellfmt, no changes
* rockchip64_common: move SERIALCON defaulting logic to a (verbose) hook for flexibility
* config: allow to build BRANCHes not listed in KERNEL_TARGET as long as the config is valid
- useful for `collabora` and other experimental kernels, we don't want to have to add it to each individual board's KERNEL_TARGES one by one
- but we don't want to allow typos in BRANCH to emit very strange unrelated errors
* extensions: mesa-oibaf extension by @monkaBlyat - mainline mesa PPA for Ubuntu
- does nothing on Debian
* extensions: (v3) amazingfated-rk35xx, now `rk-multimedia-amazingfate` - panfork-free
- simply skips if not on Jammy
- deploys Chromium + Widevine if desktop
* rockchip-rk3588: introduce `vendor-boogie-panthor` experimental BRANCH/kernel
- original: https://github.com/hbiyik/linux-rockchip.git (branch `rk-6.1-rkr1-panthor-v6`)
- I picked the commits on top of clean armbian/linux-rockchip `6.1-rkr1` as of 2024-04-01
- At https://github.com/rpardini/armbian-linux-rockchip-rk3588/tree/armbian-rk-6.1-rkr1-plus-boogie-panthor-v6
- Diff: https://github.com/armbian/linux-rockchip/compare/rk-6.1-rkr1...rpardini:armbian-linux-rockchip-rk3588:armbian-rk-6.1-rkr1-plus-boogie-panthor-v6
- rockchip-rk3588: introduce `boogie-bsp` BRANCH
- rockchip-rk3588: copy linux-rk35xx-vendor.config into linux-rk35xx-boogie-bsp.config
- rockchip-rk3588: update linux-rk35xx-boogie-bsp.config, no changes
- rockchip-rk3588: linux-rk35xx-boogie-bsp.config: `CONFIG_DRM_PANTHOR=m`
- rockchip-rk3588: linux-rk35xx-boogie-bsp.config: convert to defconfig
- rockchip-rk3588: rename to `BRANCH=vendor-boogie-panthor` for "clarity" (lol)
- rockchip-rk3588: vendor-boogie-panthor: force SERIALCON, full firmware (for blob needed for panthor) & mesa-oibaf extension
- rockchip-rk3588: vendor-boogie-panthor: enable amazingfated-rk35xx extension sans-panfork
- common.conf delegating to new `mainline_kernel_decide_version` hooks
- special case handling for v6.7-rc7
- some examples
- late-stage "use rolling branch version"
- the interactive menus didn't allow interactive users to select invalid combinations...
- ... but if specified directly on cmdline, no checks were done at all.
- introduce `exit_with_target_not_supported_error()`, which is just `exit_with_error` (code 43) but with code 44
- this way the targets pipeline can just warn instead of break;
- sometimes it's easier multiplying matrixes and skipping the few that can't be built
> tl-dr:
> - maximize OCI cache hit ratio across nightlies/releases/PRs/etc;
> - publish simple `Version:`'s that don't include a crazy hash in repo and images
> - introduce `output/packages-hashed` directory
> - radically change the `output/debs` directory structure
- simplify artifact's `prepare_version()` method for `deb` and `deb-tar` artifacts:
- `artifact_base_dir` and `artifact_final_file` will now be auto-calculated; thus removed from each artifact (except `rootfs`)
- `artifact_deb_repo` ("global", "jammy", "bookworm") is now required; "global" means common across all RELEASES
- `artifact_deb_arch` is now required, "all" is arch-independent, otherwise use `${ARCH}`
- `artifact_map_debs` is now auto-calculated based on the above, and shouldn't be specified manually
- `artifact_final_version_reversioned` is optional, and can force the final version of the artifact (specific for the `base-files` case)
- artifacts that need special handling for reversioning can add function names to `artifact_debs_reversion_functions` array (`base-files` and `bsp-cli` cases)
- artifacts `prepare_version()` should set `artifact_version`, but _never_ include it in other variables; `artifact_version` is now changed by framework after `prepare_version()` returns
- no longer use/refer/mention `${REVISION}` when building packages. All packages should be `${REVISION}`-agnostic.
- `${REVISION}` (actually, `artifact_final_version_reversioned`) will be automatically swapped in the `control` file during reversioning
- `fakeroot_dpkg_deb_build()` now takes exactly two arguments: the directory to pack, and the deb ID (key of `artifact_map_packages` dict); add this change in all the artifact's code for this
- `obtain_complete_artifact()`:
- automatically adds `-Rxxxx` "revisioning-hash" to `artifact_version`, by hashing the revisioning functions and any `artifact_debs_reversion_functions` set
- calculates more complex subdirectory paths for both the `output/packages-hashed` and `output/debs`/`output/debs-beta` directories
- with the new subdirectories we can be sure a re-version is already done correctly and can skip it (eg, for partial `download-debs` re-runs)
- in the future we can automatically clean/remove old versions that are no longer relevant based on the dir structure
- exports a lot more information to JSON, including the new subdirectory paths
- comment-out code that implemented `skip_unpack_if_found_in_caches`, I'm very unsure why we had this in the first place
- `obtain_artifact_from_remote_cache()`
- for `deb` type artifacts, OCI won't preserve the subdirectory structure, so move downloaded files to the correct subdirectory manually
- this is not needed for `deb-tar`, since that can preserve the dir structure itself
- introduce `artifacts-reversion.sh` and its main function `artifact_reversion_for_deployment()`
- this has the logic for reversioning .deb's, by `ar`-unpacking them, changing `control.tar` (and possibly `data.tar`), handling `.xz` compression, etc.
- also handles hashing those functions, for consistency. Any changes in reversioning code actually change the artifact itself so we're not caught by surprise
- by default, it changes `control` file only:
- replace `Version:` (which is the hash-version originally) with `artifact_final_version_reversioned` (which is mostly just `${REVISION}`)
- add a custom field `Armbian-Original-Hash:` with the original hash-version
- `artifact_reversion_for_deployment()` is called by
- new CLI wrapper `cli_obtain_complete_artifact()`, used for CLI building of specific artifact, but also for `download-artifact`
- `build_artifact_for_image()` used during image build
- `armbian-bsp-cli-deb.sh`: move `${REVISION}` related stuff from the main package build to new reversioning functions.
- `artifact-armbian-base-files.sh`: move `${REVISION}` related stuff from the main package build to new reversioning functions.
- `kernel`:
- add some custom fields to `DEBIAN/control`:
- `Armbian-Kernel-Version:` / `Armbian-Kernel-Version-Family:` (for future use: cleanup of usage of `Source: ` field which should be removed)
- declutter the `Description:` field, moving long description out of the first line
- obtain `IMAGE_INSTALLED_KERNEL_VERSION` from the reversioned deb (this is still a hack and has not been fixed)
- `uboot`:
- declutter the `Description:` field, moving long description out of the first line
- use the reversioned .deb when deploying u-boot to the image
- `main_default_build_packages()` now stores reversioned values and complete paths to reversioned .deb's
- `list_installed_packages()` now compares custom field `Armbian-Original-Hash: `, and not the `Version:` to make sure debs in the image are the ones we want
- `install_artifact_deb_chroot()` is a new wrapper around `install_deb_chroot()` for easy handling of reversioned debs
- use it everywhere `install_deb_chroot()` was used in `distro-agnostic.sh` and `distro-specific.sh`
- change deployment dir to not include REVISION or ARCH
- get rid uboot's CHOSEN_UBOOT, REVISION and ARCH in the directory name.
- no two u-boot debs can be installed in the same machine anyway
- split `image` apt .list deployment into `image-early` (after rootfs extract, before building) and `image-late` (after building)
- rationale: having the repo enabled during the image build can cause 'apt upgrade' and others to do inconsistent stuff depending on the (possibly random) state of the repo
also:
- get rid of comments and `SUBREVISION` and `RC` variables
- better debugging in calculate_hash_for_variables() do_normalize_src_path=yes/no
- tag a few places where output/debs might wrong in face of BETA=yes/no
- bsp-cli: now depends on `base-files (>= ${REVISION})`, this way upgrading the bsp-cli causes our base-files to be installed
- bsp-cli no longer does gymnastics with /etc/os-release et al, all done in armbian-base-files now
- general/apt-utils.sh: introduce `apt_find_upstream_package_version_and_download_url()`
- base-files: add release to version, in order to comply with repo restrictions (valid repos can't have two different debs with same name and version, md5 must match)
- partially revert 6ef394d95d (thus bring back the TUI/dialog for selecting KERNEL_CONFIGURE=yes/no)
- partially revert d890b418c7 (thus bring back the capacity to config & build image in one go)
- stop after configuring kernel, but only if command is `kernel-config`, not regular image-build KERNEL_CONFIGURE=yes
- revert #5139 Revert "Apt lists is needed for synpatic index generation"
This reverts commit 63e9bd3baa
- this actually only left junk in rootfs cache for the future to find
- all interactive commands now **don't build the artifact** anymore; just patches/.configs are produced and then build ends
- user is required to put the produced patches in the right place and build again, for full consistency
- split ATF and U-BOOT manual patching process; use CLI command `atf-patch` to patch ATF, and `uboot-patch` to patch u-boot
- non-interactive artifact builds are now 100% sans-stdin
- introduce `uboot-config` CLI command; still experimental, only produces a defconfig and not a patch
- reworked `userpatch_create()` to be (hopefully) more useful:
- detects a previous patch and offers to apply it before continuing
- enters a loop showing the diff, and only proceeds when user indicates he's happy with the patch
- produces `mbox`-formatted patches via `format-patch` and standard Armbian parameters
- uses MAINTAINER and MAINTAINERMAIL instead of git configuration (so it works in containers)
- don't allow image builds with any patching or configuring _at all_ (it has been deprecated with a warning for months already, and results are inconsistent)
Add interactive configurations to Repeat Build Options
Simplify function produce_repeat_args_array()
- make use of $ARMBIAN_NON_PARAM_ARGS and $ARMBIAN_PARSED_CMDLINE_PARAMS (associated array)
- quote parameter values to be on the safe side
Move "Repeat Build Options" logs to start-end.sh
- cli-build.sh, start-end.sh:
- move function function produce_repeat_args_array() to start-end.sh
- add $WHAT as first arg to produce_repeat_args_array() if
$WHAT is not empty
- move early display log of "Repeat Build Options" to
function main_default_start_build()
- move last log of "Repeat Build Options" to
function main_default_end_build()
- add last log of "Repeat Build Options" to
LOG_SECTION="repeat_build_options"
- rootfs: rootfs-create: show a summary of the 20 biggest dirs, right before tarring the rootfs (for debugging)
- rootfs: rootfs-create: show usage of caches between first and second stages
- rootfs: rootfs-create: cleanup junk left by `debootstrap` after second stage
- rootfs: rootfs-create: _always_ clean apt stuff at the end
- rename `apt_purge_unneeded_packages()` to `apt_purge_unneeded_packages_and_clean_apt_caches()` for clarity
- image: `apt_purge_unneeded_packages_and_clean_apt_caches()`: warn if apt caches not empty; clean them off, always.
- host-utils: `local_apt_deb_cache_prepare()`: also test the target, warn if not empty
- extension: cleanup-space-final-image: do NOT clean apt stuff. done in core now
- the metric shit-ton of debugs added should help the next person who faces this in the future
- if CONFIG_DEFS_ONLY during `obtain_complete_artifact()`, just dump JSON to stdout after calculating version/coordinates
- don't prepare ORAS tooling if CONFIG_DEFS_ONLY
- don't run in standard build (and thus don't prepare_host()) if under CONFIG_DEFS_ONLY
- if CONFIG_DEFS_ONLY, don't run `check_basic_host()`
- bsp-cli: does not require aggregation
- bsp-desktop / desktop: requires aggregation
- 'desktop' packaged moved out of 'bsp' folder, it is NOT a bsp
- make consistent naming/folders
- extract .sh generation workflow into 'declare -f' magic functions
- `SHOW_DEBIAN=yes` to bump most of `DEBIAN/xxx` stuff to screen
- artifact armbian-bsp-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`; requires BOARD
- artifact armbian-desktop: use aggregation, require certain vars, CLI forces `BUILD_DESKTOP=yes`, not require `BOARD` anymore
- cli: commands: fix artifact aliases, reorder
- desktop/bsp-desktop: add `config_dump` method; fix `artifact_map_debs` ref subdir and name
- artifact bsps: require aggregation to build, not to list what needs to be built
- distro-agnostic: fix install of `armbian-bsp-cli` (not `armbian-bsp`)
- armbian-bsp-cli: add `_config_dump` method, use configuration with aggregation, add required vars check
- distro-specific: fix installation of `fake-ubuntu-advantage-tools`
- include rudimentary artifact dependency information for config-dump-json: fixes/refactor/cleanup
- artifact-rootfs.sh: include rudimentary artifact dependency information for config-dump-json
- artifact-armbian-(config/plymouth-theme/zsh).sh: include rudimentary artifact dependency information for config-dump-json
- artifact-fake-ubuntu-advantage-tools.sh: include rudimentary artifact dependency information for config-dump-json
- use new capture'd vars scheme
- so `./compile.sh BOARD=xxx BRANCH=yyyy config-dump-json | jq .` now works and is consistent/newline tolerant
- introduce internal `skip_host_config=yes` for `prep_conf_main_minimal_ni()` to skip calling `check_basic_host()`
