Commit Graph

10905 Commits

Author SHA1 Message Date
Ricardo Pardini
c4eaf49f3d armbian-next: cleanup extensions logging, using display_alert new levels extensions (SHOW_EXTENSIONS=yes) and extensionstrace (SHOW_EXTENSIONS_TRACE=yes)
- DEBUG_EXTENSION_CALLS is gone. use SHOW_EXTENSIONS instead.
- LOG_ALL_HOOK_TRACES is gone. use SHOW_EXTENSIONS_TRACE instead
- LOG_ENABLE_EXTENSION is gone. use SHOW_DEBUG=yes instead
2023-02-18 07:39:54 -03:00
Ricardo Pardini
72bc1b09d8 armbian-next: docker: don't even try to to prepare Docker, if it's not available 2023-02-18 07:39:53 -03:00
Ricardo Pardini
3e9c123dd8 armbian-next: docker: try support Rancher Desktop (in dockerd mode), experimental.
- Changes how Docker Desktop works too, there's a `case` for switching back if needed.
- Now the linux-only CONTAINER_COMPAT hacks are used also for Darwin (`DOCKER_SERVER_REQUIRES_LOOP_HACKS=yes`)
  - `/dev/loop*` is spelled out since they don't exist on Darwin (`DOCKER_SERVER_USE_STATIC_LOOPS=yes`)
2023-02-18 07:39:52 -03:00
Ricardo Pardini
3719a059a9 armbian-next: don't force SHOW_LOG=yes during Dockerfile build; we're verbose enough without it 2023-02-18 07:39:51 -03:00
Ricardo Pardini
b3c008b0e0 armbian-next: make sure we've coreutils 2023-02-18 07:39:50 -03:00
Ricardo Pardini
7107368060 armbian-next: make sure we're running on Bash 5.x at least 2023-02-18 07:39:49 -03:00
Ricardo Pardini
19e15b8b35 armbian-next: back to including 'cli' sources/apt sources when building non-desktops (eg, nala etc) 2023-02-18 07:39:48 -03:00
Ricardo Pardini
a612a03678 armbian-next: retry 3 times for loop device's size, then give up; introduce RETRY_RUNS in do_with_retries() 2023-02-18 07:39:47 -03:00
Ricardo Pardini
b9daf6d708 armbian-next: extras/buildpkg.sh code is out [deprecated, later completely dropped] 2023-02-18 07:39:46 -03:00
Ricardo Pardini
a54e7b2fae armbian-next: add gcc-riscv64-linux-gnu to hostdeps (crossbuild-essential-riscv64 is Ubuntu-only) 2023-02-18 07:39:45 -03:00
Ricardo Pardini
bb0e89f017 armbian-next: always include libc6-amd64-cross in hostdeps; qemu is a dummy package now, remove 2023-02-18 07:39:44 -03:00
Ricardo Pardini
d24d3327a8 armbian-next: the great cli entrypoint (+docker) rewrite; introduce USE_LOCAL_APT_DEB_CACHE replacing apt-cacher-ng
- armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker)
  - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for
      - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages
      - debootstrap, by passing it `--cache-dir`
  - utility function to help understand what is happening to cache during usage
  - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_
      - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s
      - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot
  - clean up `install_deb_chroot()` a little, find an unrelated bug there
- WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile
  - it's `drastic red` on non-buildx dockers
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x
  - source configs in a logging section.
  - Docker: silent, fast retries to make sure `docker system df` works
  - shut-up `chown` (no `-v`) output related to  `SET_OWNER_TO_UID`
  - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow
  - use green apple for Mac logging, instead of red apple which might imply error...
- WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker
  - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol
  - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root.
    - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID`
    - use it for userpatches files created, logs, and output files, including images and debs.
  - @TODOs ref. `$SUDO_USER` which I think the old version of this?
  - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command.
    - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3
- WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented
- WiP: the great cli entrypoint (+docker) rewrite, Phase 1
- armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs.
  - using image in my private repo.
  - this has significant speedup to "start building time" on the 1st run
  - move some Linux specific stuff to its own if
  - add comments and todo
- armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences
- armbian-next: WiP: Docker, configure `BUILDKIT_COLORS`
- armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers
- armbian-next: `logging`: add whale indicator if build running under Docker
- armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only
- armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy
- armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots
- WiP: try with debian:bullseye -- can't detect docker at all
- armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working
  - gen .dockerignore together with Dockerfile
  - split in funcs
  - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes
  - mac still works, Linux stuff would break it but I if'fed
- armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing
  - this fails for the right reasons now, causing retries, which are then retried and work ;-)
  - this is related to building under Docker on Linux, using docker.io package (not docker-ce)
- armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated
- armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build
  - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal
-  armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore
-  armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY
  - uses REQUIREMENTS_DEFS_ONLY
  - works on Docker Desktop on Mac;
  - linux TBA
- armbian-next: don't error out if `.git` not present; other small fixes
- armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance"
  - can't assume things, for example:
  - that `sudo` will be available; it might not, and might be already root, no reason to fail
  - that `/etc/timezone` will exist
  - that `systemd-detect-virt` will be available
  - that `git` will be available
  - that `locale-gen` will be available
2023-02-18 07:39:43 -03:00
Ricardo Pardini
2c6751f584 armbian-next: show mkfs ext2/ext4 output (remove -q), it's being run under the logging manager already 2023-02-18 07:39:42 -03:00
Ricardo Pardini
8f26db138a armbian-next: traps: don't duplicate error message/stacktrace when error occurs in subshell; instead indicate clearly SUBSHELL 2023-02-18 07:39:41 -03:00
Ricardo Pardini
775cd6c619 armbian-next: logging: don't bomb due to lack of git or zstdmt
- logging might run super early when dependencies are not installed
- or say in a container, without git
2023-02-18 07:39:40 -03:00
Ricardo Pardini
5e55c61bfc armbian-next: kernel: add hooks fetch_sources_for_kernel_driver and patch_kernel_for_driver to allow migrating EXTRAWIFI stuff to extensions 2023-02-18 07:39:39 -03:00
Ricardo Pardini
c4165abe4e armbian-next: export CHOSEN_KERNEL_WITH_ARCH again, for reporting only 2023-02-18 07:39:38 -03:00
Ricardo Pardini
40e3b642d6 armbian-next: don't break if parsing BOOT_SOC out of BOOTCONFIG fails; sanity checks for BOOT_SOC/DDR_BLOB
- (eg: `none` case: `jetson-nano`)
- but add sanity checks for BOOT_SOC/DDR_BLOB etc
- and fail if BOOT_SOC is not defined when function runs
2023-02-18 07:39:37 -03:00
Lane Jennison
28c7645a14 armbian-next: fix old fuction name refrence cleaning to general_cleaning 2023-02-18 07:39:36 -03:00
Ricardo Pardini
cb38a8071b armbian-next: umount_chroot_recursive: don't try to realpath unless it's a dir to begin with 2023-02-18 07:39:35 -03:00
Ricardo Pardini
c444040740 armbian-next: grub: call VER="generic" update_initramfs when "${DISTRO_GENERIC_KERNEL}" == "yes" (so ddk builds work again) 2023-02-18 07:39:34 -03:00
Ricardo Pardini
4cebf28be4 armbian-next: move resolv.conf symlink to systemd to post_debootstrap_tweaks so customize phase has working DNS 2023-02-18 07:39:33 -03:00
Ricardo Pardini
3e0729b5da armbian-next: normalize mount/umount $targets to have (or not) the trailing slash 2023-02-18 07:39:32 -03:00
Ricardo Pardini
72cf555bce armbian-next: bye bye $FAST_CREATE_IMAGE: always use truncate for blank image 2023-02-18 07:39:31 -03:00
Ricardo Pardini
914348ba52 armbian-next: less verbose umount_chroot(), except after first try, so we can know what is left mounted 2023-02-18 07:39:30 -03:00
Ricardo Pardini
f921c8cd74 armbian-next: let all unmounting of ${MOUNT} be done by umount_chroot_recursive 2023-02-18 07:39:29 -03:00
Ricardo Pardini
d85bf2a155 armbian-next: many fixes after v29 rebase; mostly involving umount
- remove grub's --verbose, it's really too verbose
- trust in TMPDIR, do not use trap at all in compile_plymouth-theme-armbian - settings traps adhoc is forbidden in armbian-next as well
- be more verbose in umount_chroot(), do not try to unmount tmpfs-mounted dir itself, only xx/tmp
- try to be smarter about /tmp being mounted in rootfs-to-image -- in the end the recursive saves us
- run post_debootstrap_tweaks when the SDCARD is still mounted, not after
- don't try to download anything from any mirrors if SKIP_ARMBIAN_REPO==yes
2023-02-18 07:39:28 -03:00
Ricardo Pardini
833a7d9ab6 armbian-next: fix "create list of installed packages for debug" 2023-02-18 07:39:27 -03:00
Ricardo Pardini
b73e72721a armbian-next: HACK: ROOTFSCACHE_VERSION is undefined and failing everytime 2023-02-18 07:39:26 -03:00
Ricardo Pardini
9137a9de3f armbian-next: deploy bootscript even if BOOTCONFIG=none
- fact we're not BUILDING nor deploying uboot does not mean image does not need a bootscript.
2023-02-18 07:39:25 -03:00
Ricardo Pardini
f052196b6c armbian-next: debugs about armbianEnv.txt overlays/fdtfile 2023-02-18 07:39:24 -03:00
Ricardo Pardini
7b47e79780 armbian-next: introduce ARMBIAN_LOGS_TO_JOURNAL and ARMBIAN_LOGS_JOURNAL_IDENTIFIER (logs display_alert to journald) 2023-02-18 07:39:23 -03:00
Ricardo Pardini
19938837b5 armbian-next: re-add dependency fdisk after juggling rebases 2023-02-18 07:39:22 -03:00
Ricardo Pardini
38e2e50328 armbian-next: split early_prepare_host_dependencies() (run early, during config, before aggregation) from prepare_host() (run after aggregation)
- add hook `host_dependencies_known` (different from `host_dependencies_ready` from before)
2023-02-18 07:39:21 -03:00
Ricardo Pardini
1fa825b58e armbian-next: fix for tests of BOOTSCRIPT and BOOTCONFIG!=none 2023-02-18 07:39:20 -03:00
Ricardo Pardini
271fe3fcd0 armbian-next: retry apt update & apt install family packages in addition to board packages 2023-02-18 07:39:19 -03:00
Ricardo Pardini
45c2728851 armbian-next: prepare a bin dir inside WORKDIR with a python symlink to /usr/bin/python2 and add it to PATH 2023-02-18 07:39:18 -03:00
Ricardo Pardini
4055399d15 armbian-next: sources/families: meson64: gx: use python2 explicitly for calling acs_tool.pyc; better logging 2023-02-18 07:39:17 -03:00
Ricardo Pardini
a1ff891b24 armbian-next: cleaning: don't complain about 'none' CLEAN_LEVEL 2023-02-18 07:39:16 -03:00
Ricardo Pardini
188ef0b1af armbian-next: kernel: run kernel's make through unbuffer as well 2023-02-18 07:39:15 -03:00
Ricardo Pardini
98eae22665 armbian-next: uboot: use CFLAGS/KCFLAGS everywhere; run make through unbuffer (from expect package) for full color logging 2023-02-18 07:39:14 -03:00
Ricardo Pardini
2196e957f0 armbian-next: logs: avoid errors by cwd'ing to SRC 2023-02-18 07:39:13 -03:00
Ricardo Pardini
379b7d6f19 armbian-next: write_image_to_device: if SKIP_VERIFY=no, don't waste time checksumming the image, it won't be used 2023-02-18 07:39:12 -03:00
Ricardo Pardini
a02647862a armbian-next: u-boot: add debug info when hacking CONFIG_ENV_IS_IN_EXT4 in u-boot .config 2023-02-18 07:39:11 -03:00
Ricardo Pardini
bfa5813c6e armbian-next: u-boot: add fasthash to u-boot [deprecated, later removed]
- u-boot: fasthash: add `uboot_target_counter` to id and branch, lest `UBOOT_TARGET_MAP` > 2 fails
2023-02-18 07:39:10 -03:00
Ricardo Pardini
f07ec572d8 armbian-next: fix logging headers in HTML/ANSI log output 2023-02-18 07:39:09 -03:00
Ricardo Pardini
578020f42c armbian-next: cleaning: fix shortcircuits, warn about unknown clean levels 2023-02-18 07:39:08 -03:00
Ricardo Pardini
b0cb5c270f armbian-next: traps: cleanups: run cleanups in the reverse order they were added; allow running one single cleanup and removing from list 2023-02-18 07:39:07 -03:00
Ricardo Pardini
d6c43d0cdf armbian-next: kernel-localmodconfig: update kernel_config_mtime after running localmodconfig, otherwise changes not detected 2023-02-18 07:39:06 -03:00
Ricardo Pardini
41e471aa1e armbian-next: introduce KERNEL_EXTRA_DIR to suffix LINUXSOURCEDIR forcefully 2023-02-18 07:39:05 -03:00