Commit Graph

91 Commits

Author SHA1 Message Date
Jacob Hrbek
6556a3a77e Add compatibility for non-FHS 3.0 systems
* Added `shell.nix` definition for temporary development environment for Nix(OS)
* Set uuidgen and other binaries check to not rely on hard-codded paths
2023-07-19 23:16:22 +02:00
Ricardo Pardini
f724ed130e export-logs: introduce PASTE_URL to make easy to change paste service used 2023-06-13 12:07:40 +02:00
Ricardo Pardini
e2f32c53f8 export-logs: add GHA output for logs_url (when SHARE_LOG=yes) 2023-05-17 18:06:21 +02:00
Ricardo Pardini
f0f9159faa logging: curb CI/GHA special errors for cascading errors during build failure
- otherwise we get 4 errors for each "error" in CI
  - one for the real error
  - one for "wait for cleanups"
  - one for "Docker build failed"
  - one from GHA itself since command failed (we can't get rid of this)
2023-05-15 17:43:25 +02:00
Ricardo Pardini
c692676da7 kernel-debs: postinst/etc: fix: unify is_boot_dev_vfat() function; make work during image build too via env var
- determinining if `/boot` is vfat is more complex than it looks;
  - use a common function to unify across all scripts
- during image build, the kernel install is always done with a non-fat, non-mounted /boot
  - use a special variable passed through the env so unified function can know ahead of time
  - introduce `extra_apt_envs` param to `chroot_sdcard_apt_get()`; array with extra environment vars
- if vfat /boot:
  - `linux-update-symlinks` is not called (Debian would just choke on this)
  - don't even try to symlink Armbian stuff, move instead
- more/better logging board-side (unfortunately mixed in all the set -x debugs)
2023-05-01 22:46:43 +02:00
Ricardo Pardini
3d2575639d logging: include Repeat Build Args in markdown/ANSI logs 2023-04-21 14:26:33 +02:00
Ricardo Pardini
953ab276d0 logging: better logging when ANSI_COLOR=none (eg, during info collection) 2023-04-15 22:43:06 +02:00
Ricardo Pardini
ed0b406265 lib: fix: replace undue export statements with declare -g 2023-03-31 09:12:44 +02:00
Ricardo Pardini
fc14d62c52 capture: set globals CAPTURED_VARS_NAMES and CAPTURED_VARS_ARRAY (instead of CAPTURED_VARS_ARRAY that was space-delimited); filter and sort last-minute 2023-03-31 09:12:44 +02:00
Ricardo Pardini
9a74b838ce runners: introduce internal skip_error_info=yes for run_host_command_logged()
- doesn't show nor clear vars `if_error_find_files_sdcard` and `if_error_detail_message`
- used by `chroot_sdcard_apt_get()` which uses `run_host_command_logged()` multiple times, and the first would clear the important/last one
2023-03-31 09:12:44 +02:00
Igor
d4a7a1c816 Revert "Update export-logs.sh to use primary Armbian paste server (#4936)"
This reverts commit bc62c8cd42.
2023-03-18 21:39:02 +01:00
Ricardo Pardini
736fe7ecf4 don't use "bright black" ANSI for logging unless we're running under a known-good terminal (eg, iTerm2)
- giving up on the "bright black" == "gray" thing. There's enough bad default themes out there, and not the first time people (rightfully) complained...
2023-03-17 08:32:11 +01:00
Ricardo Pardini
ee11f5dbd9 chroot_sdcard_apt_get: don't use -qq for all apt operations on chroot if SHOW_DEBUG=yes (or DEBUG=yes)
- helps debugging crazy upgrades happening due to @rpardini's stupid `Version: ` insanity
2023-03-12 18:40:20 -03:00
TheLinuxBug
bc62c8cd42 Update export-logs.sh to use primary Armbian paste server (#4936)
Changing export-logs.sh script from posting to armbian.next.armbian.com to posting to the primary paste server which has now been upgraded at paste.armbian.com.
2023-03-11 22:03:40 +01:00
Igor Pečovnik
b7b8eb7b72 Add / modify (c) in bash scripts (#4922)
* Add / modify (c) in bash scripts

Signed-off-by: Igor <igor@armbian.com>

* Add (c) to the source config files

---------

Signed-off-by: Igor <igor@armbian.com>
2023-03-09 18:30:40 +01:00
Ricardo Pardini
25606ef3c8 logs: remove git diff from GIT_INFO_ANSI, it's somewhat useless and potentially huge leading to argument list too long 2023-02-24 13:46:28 -03:00
Ricardo Pardini
2f7423d0e4 shellfmt/.editorconfig: enable trim whitespaces, run shellfmt, no actual code changes (#4849)
- shellfmt: bump to 3.6.0 (run with `bash lib/tools/shellfmt.sh`)
- .editorconfig: rationalize, remove duplicated logic
2023-02-23 09:24:41 +01:00
Ricardo Pardini
8f23c9d2e4 armbian-next: debug_dict()/debug_var() only if debugging 2023-02-18 07:46:08 -03:00
Ricardo Pardini
ecbf09491d armbian-next: logging: introduce RAW_LOG=yes to export .tar with raw unprocessed logs (for development) 2023-02-18 07:45:46 -03:00
Ricardo Pardini
3bcf89d6c6 armbian-next: logging: introduce SHARE_LOG=yes to auto-curl the logs to paste 2023-02-18 07:45:45 -03:00
Ricardo Pardini
0e480e685e armbian-next: docker: pass down git info via env var, since Docker doesn't get ${SRC}/.git, yet we need that info in the Docker logs 2023-02-18 07:45:40 -03:00
Ricardo Pardini
388c76c91b armbian-next: shellcheck: squash more shellcheck warnings; some long-lost variables being used; some unset's that are better reset's, etc 2023-02-18 07:45:33 -03:00
Ricardo Pardini
ba7487b1a1 armbian-next: docker: fix Dockerfile generation; don't output logs info during Dockerfile build 2023-02-18 07:45:23 -03:00
Ricardo Pardini
c5de5e0f1f armbian-next: logging: ansi-haste sharing beta 2 (now with URL) 2023-02-18 07:45:22 -03:00
Ricardo Pardini
bef7f8fee4 armbian-next: artifacts: introduce kernel/u-boot artifacts; git ref2info; "memoizer"; some hashing
- artifacts: u-boot/kernel - pt7 - adapt legacy/artifact versions; use common `capture_rename_legacy_debs_into_artifacts()`
- artifacts: u-boot - pt6: add artifact for u-boot
  - use artifact version / reason in actual u-boot .deb if present
- artifacts: kernel - pt5: tune kernel version, refactor
- artifacts: kernel - pt4: squash unrelated bugs that show up; move `prepare_compilation_vars()` to default build
- artifacts: kernel - pt3: drivers+patches+.config hashing
  - split file hashing function from drivers-harness; fix it so filenames are relative and sorted; sort from ${SRC}, always
  - aplit prepare_kernel_config_core_or_userpatches() from `kernel_config_initialize()`
- artifacts: kernel - pt2:  memoizing git ref2info
- artifacts: kernel - pt1:  versioning
2023-02-18 07:45:20 -03:00
Ricardo Pardini
b4df4b83c3 armbian-next: logging: detect TERM=alacritty and use normal instead of gray 2023-02-18 07:45:17 -03:00
Ricardo Pardini
41d7d0155e armbian-next: docker: fix: not all find's are made equal, back to -mtime +1 (full 24-hour period needed for match) 2023-02-18 07:45:08 -03:00
Ricardo Pardini
3c5488b002 armbian-next: bump logging archive greet to 14 files 2023-02-18 07:44:59 -03:00
Ricardo Pardini
89d6c686c6 armbian-next: chroot_sdcard_apt_get_install_dry_run(): run again if first try fails, with more logging 2023-02-18 07:44:52 -03:00
Ricardo Pardini
6177b94268 armbian-next: logging: pure ASCII .log; included in Markdown for GHA 2023-02-18 07:44:45 -03:00
Ricardo Pardini
1cec42392e armbian-next: GHA & rootfs adventures, pt 1
- rootfs: create readonly global `rootfs_cache_id` (type+hash+date/version)
- add "oras-upload" CLI command (dumb, no retries, single target/single source)
- docker: mount-bind for & re-pass envs `GITHUB_OUTPUT` and `GITHUB_STEP_SUMMARY` down to Docker
- introduce `github-actions.sh::github_actions_add_output()`
- during logs cleanup, dump the Markdown log into GITHUB_STEP_SUMMARY if it is available
2023-02-18 07:44:41 -03:00
Ricardo Pardini
78dcf01c92 armbian-next: cleanup, kill and add to-do's, remove dead code, turn down logging, squash future shortcircuit bugs
- killed `[[ $ROOTFS_TYPE != ext4 ]] && display_alert "Assuming ${BOARD} ${BRANCH} kernel supports ${ROOTFS_TYPE}" "" "wrn"`
  - which definitely didn't belong in rootfs
- disable usage of run_host_command_logged_long_running for kernel-make
- stop lying about long_running stuff being any different from non-long_running versions
- stop lying about 'set -e' when there's still a bunch of pipes for "pv" and stuff all around
2023-02-18 07:44:33 -03:00
Ricardo Pardini
44ae761927 armbian-next: apt: expand local caching to include package lists, as well as .deb's
- `/var/cache/apt` in chroot was already cached;
- add `/var/lib/apt/lists`
- remove usage of namerefs, replace with global dict
2023-02-18 07:44:19 -03:00
Ricardo Pardini
805085ff45 armbian-next: logging: validate where not already in a logging section when trying to start a new one
- with a small exception for 'entrypoint' ;-)
2023-02-18 07:44:18 -03:00
Ricardo Pardini
3f0c480e9d armbian-next: mark remove_all_trap_handlers() deprecated, it never did anything 2023-02-18 07:44:16 -03:00
Ricardo Pardini
780fe26ea4 armbian-next: logging: (very, very early beta) for uploading/sharing of ANSI log file online 2023-02-18 07:44:13 -03:00
Ricardo Pardini
d95478281b armbian-next: fix ANSI logs, use reset at strategic places 2023-02-18 07:44:06 -03:00
Ricardo Pardini
eb30aac310 armbian-next: extensions: move 'extensions.sh' into library; replace manual source with init function
- wraps globals in `extension_manager_declare_globals()` -- and make them actually global
- whoever wrote this original code is insane, add copyright
2023-02-18 07:44:05 -03:00
Ricardo Pardini
22074a0cde armbian-next: logging: new log file / summary nomenclature 2023-02-18 07:44:01 -03:00
Ricardo Pardini
a6182bbd89 armbian-next: docker: archive logfiles again; warn & greet testers when archiving many old logfiles 2023-02-18 07:44:00 -03:00
Ricardo Pardini
72c61f2f68 armbian-next: replace bare sync invocations by introducing wait_for_disk_sync(); sync early at start of build
- wait 10s for sync call; otherwise warn user to be patient
- wait_for_sync "reason for sync" everywhere
- add @TODO for /run/user/0 etc and XDG_RUNTIME_DIR in chroot
2023-02-18 07:43:53 -03:00
Ricardo Pardini
8af14d4f8c armbian-next: countdown: split off file for countdown functions (countdown-to-abort and countdown-to-continue)
- introduce `countdown.sh`, move `traps.sh::exit_if_countdown_not_aborted()` there, introduce `exit_if_countdown_not_aborted()`
2023-02-18 07:43:50 -03:00
Ricardo Pardini
8cedc82ec2 armbian-next: hopefully improved error logging; avoid logging errors up to 3x; subdued red stacktraces
- reorg `display_alert()` so more frequent levels are handled first
- improve some ANSI color usage, still a mess
- don't write failed commands stackstrace as warnings to screen -- just to file (yes it's redundant, except if in test)
- more useful / subdued stacktraces, to make actual error more obvious...
2023-02-18 07:43:43 -03:00
Ricardo Pardini
242a5dad1f armbian-next: somewhat-better handling/logging when using Docker (don't show two logfiles to inspect)
- do not show path to Docker run-host's logfile, unless the Docker run itself didn't produce logs
- introduce `global_final_exit_code` to allow to maintain/preserve Docker's exit code without triggering an error host-side
2023-02-18 07:43:41 -03:00
Ricardo Pardini
bf3f346bee armbian-next: still fighting tee leaking: now I think I won, again
- avoid using the "last resort, use lazy umount" by killing tee_pid _and descendants_ before hand. Works!
- introduce `get_descendants_of_pid_array()` using recursive trick to get all descendants
- do not `unset CURRENT_LOGFILE` -- during `logging_error_show_log()` (from SHOW_LOG=no days)
  - do `unset CURRENT_LOGFILE` immediately before processing logs -- although it magically survives. mysteries of traps in bash...
- `declare -g` instead of `export` for `start_logging_section()`
- don't `check_and_close_fd_13()` in `run_cleanup_handlers()`
- do run `check_and_close_fd_13()`, but only after archiving the old logs (in logging trap!)
- add a default error for unhandled TRAP's -- of which should be done
- fix reset of `global_tee_pid=0` in section-logging
2023-02-18 07:43:40 -03:00
Ricardo Pardini
ec7639e518 armbian-next: severe bug, there was no prefix-emoji for Windows 2023-02-18 07:43:33 -03:00
Ricardo Pardini
62d948b1b4 armbian-next: still fighting tee leaking under duress, and I think I won
- call `check_and_close_fd_13()` directly when `exit_with_error()` called, don't wait for trap
- store PID spawned by `do_with_logging()` under `$global_tee_pid`
- `check_and_close_fd_13()`:
  - sprinkle with `sync` to give a chance for stuff to die gracefully
  - check if `$global_tee_pid` is set and actually a running PID, if so, kill it
  - do not blindly "wait" for stuff, instead, kill it and wait for it to die -- 100% more likely to not hang ;-)
2023-02-18 07:43:31 -03:00
Ricardo Pardini
35fbb7ed6c armbian-next: small cleanups: squash typos / add function keyword / add types / mark dead code 2023-02-18 07:43:28 -03:00
Ricardo Pardini
a6a5152695 armbian-next: chroot_sdcard_apt_get_update() to replace and better log chroot_sdcard_apt_get update
- also add retries where it was missing
- @TODO: desktop_postinstall() is a mess, needs aggregation work
2023-02-18 07:43:19 -03:00
Ricardo Pardini
2455c55dac armbian-next: firmware: don't build -full firmware if not on CI/noninteractive and board's not going to use it
- move `fakeroot_dpkg_deb_build()` from runners.sh to new utils-dpkgdeb.sh
  - and make it debug show the size of the source
- clean off `.git` directory earlier for full build
- @TODO: get rid of the "hardlinks" funky biz in there and not-copy `.git` to begin with
2023-02-18 07:43:18 -03:00