Commit Graph

278 Commits

Author SHA1 Message Date
Igor Pečovnik
f2b5a34b91 Refactor rootfs caching (#3895) 2022-06-18 23:56:36 +02:00
Igor Pečovnik
887147c82f Fix githash cache and images creation with CI (#3854) 2022-06-15 21:16:23 +02:00
Igor Pečovnik
042cdbd6a3 Force rootfs cache recreation (#3746) 2022-05-03 10:58:58 +02:00
Derek
b1e85c9258 create an option to allow ipv6 downloads (#3640)
* create an option to allow ipv6 downloads

* Coding style fix

Co-authored-by: Igor Pečovnik <igorpecovnik@users.noreply.github.com>
2022-04-07 19:07:40 +02:00
Igor Pečovnik
db408b61c4 Purge bionic, impish, hirsute, stretch (#3600)
* Purge bionic, impish, hirsute, stretch

- up jammy to supported (will be soon anyway)

* Change recommended build env to Jammy
2022-04-05 16:19:51 +02:00
Igor Pečovnik
2274e2658e Force cache recreation due to corruption (#3617) 2022-04-01 21:49:03 +02:00
Igor Pečovnik
2cd1131f72 Bump rootfs cache revision to force recreation (#3576)
We have buggy caches due to varios bugs that has been around but were fixed this week
2022-03-26 14:21:07 +01:00
Peter Young
11f1223ce2 Fix url slash & china mirror (#3525) 2022-03-09 09:24:27 +01:00
Igor Pecovnik
8ac58b806c Change MAKE_FOLDERS into accepting any value for subfolder
Internal function
2022-02-27 21:49:03 +01:00
Igor Pečovnik
c859d0c831 Fixing package name changes in Jammy (#3512)
* Detach  Jammy from focal due to package changes

Fix font package name

* Resolve broken packages for Jammy / i3 / gnome / xmonad

printer-gnome is also obsolete

* Bump rootfs cache number
2022-02-25 19:26:45 +01:00
KevinZønda
3236e5299b upd mirror site (#3483) 2022-02-13 17:42:27 +01:00
Igor Pečovnik
38e496ccee Change RC folder caps 2022-02-09 14:20:27 +01:00
Igor Pečovnik
26cef397c0 Add switch to skip image creation if it already exists (#3454)
* Move artefact creation directory into from deboostrap into configuration

This is needed to support function of early checking for artefacts. Useful for CI

* Introduce IMAGE_PRESENT parameter which we can use when creating a lot of images so we don't need to recreate whole CI if one image fails
2022-02-02 20:54:42 +01:00
csrbzhb
367280b10d fix github mirror.china github speed up (#3451) 2022-02-02 10:55:01 +01:00
Igor Pecovnik
21a4908f15 Force cache rebuild 2022-02-01 12:46:09 +01:00
Igor Pečovnik
d4ee87852c Add option to override default ubuntu arhive (#3446) 2022-01-30 23:10:51 +01:00
Igor Pečovnik
72e8fc7d36 Improve cache creation process by removing /root and /home to be syched (#3439) 2022-01-26 12:48:13 +01:00
Igor Pečovnik
af7bbccf06 Bump with cache rootfs version to force reload (#3417)
Mitigating debootstrap bug.
2022-01-20 14:46:21 +01:00
RadxaYuntian
fe9c032aac Add new options to handle mirror selection better (#3325)
* Add new option UBOOT_MIRROR to select mainline U-Boot mirror

 - set to `github` to use mirror provided by GitHub, the same as `USE_GITHUB_UBOOT_MIRROR=yes`.
 - set to `gitee` to use mirror provided by Gitee, a Chinese git service.
 - leave it unset to use offical `source.denx.de`.

* Add GITHUB_MIRROR option

Usage similar to UBOOT_MIRROR.

* Add REGIONAL_MIRROR option

Chinese users (the main user of mirror options) can have use this one instead of configure many different options.
2021-12-08 16:15:26 +01:00
Ricardo Pardini
90e0fe0ba2 extensions framework + UEFI aarch64/x86 + rpi4b + core changes/fixes (#3300)
* extensions framework (née "fragments")

- this should actually change nothing at this point, just add capabilities
- the framework is implemented in lib/extensions.sh
- the "if function x exists then call x" replaced with call_extension_method()
  - +inline documentation
  - +compatibility names

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions framework; meta-extensions: auto-docs and sample extension gen

- 2 extensions dealing with extensibility itself
  - detect-unused-extensions: shows which extensions are enabled, but never called.
  - gen-sample-extension-docs: generates a sample empty extension & Markdown documentation for extensions

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* new extension methods and features via config variables in core Armbian

- `SKIP_EXTERNAL_TOOLCHAINS=yes` - does not download or use any linaro toolchains, only build host-installed ones
- `SKIP_BOOTSPLASH=yes` - does not patch kernel for splash file
- `EXTRA_BSP_NAME=xyz` - allows for BSP variants, useful for when extensions modify the BSP
- `EXTRA_ROOTFS_MIB_SIZE=x` - add x mib's to rootfs size, for use with very small images
- `KERNEL_EXTRA_TARGETS` - what extra targets to make kernel for, default to "modules dtbs"
- `BOOTCONFIG=none` - does not build nor install u-boot; also doesn't handle bootscripts et al
- `unset KERNELSOURCE` - does not build nor install kernel, nor build initrd, nor build nor install firmware
- `ARMHF_ARCH=skip` - does not add armhf to apt/dpkg, thus pure arm64
- `SKIP_ARMBIAN_REPO=yes` - results in armbian.list.disabled in the final image
- define `APT_EXTRA_DIST_PARAMS` with apt-cacher-ng options and use it for `PACKAGE_LIST_INSTALL/REMOVE` et al
- initial support for targeting x86/amd64 UEFI and BIOS
    - some do's/don'ts for x86/amd64, like a different `UBUNTU_MIRROR` default
    - GPT/EFI(ESP) partitions (fat, `UEFISIZE=256` to enable, mount `UEFI_MOUNT_POINT=/boot/efi`, first on disk but ends
      up at `$uefipart`=15)
    - GPT/BIOS partitions (fat, `BIOSSIZE=1` to enable, second on disk but ends up at partition 14)
    - `UEFI_FS_LABEL="armbiefi"` - to set the FAT label for the EFI partition, visible in Win/Mac
    - hard-requires gdisk package host-side
- add add_host_dependencies() extension method; fill `EXTRA_BUILD_DEPS="pkg pkg2"` to install to host before toolchains
  download
- add pre_prepare_partitions() extension method, for custom partition size calculations
- add create_partition_table() extension method, used to do full-custom partitioning if `USE_HOOK_FOR_PARTITION=yes`
- add post_create_partitions() extension method, mostly for easy debugging
- add post_write_sdcard() extension method, where you can also set `SKIP_VERIFY=yes` to skip sdcard verification
- add post_install_kernel_debs() extension method.
- multiple fixes to bsp to avoid spurious errors when files are not where it expects
- v4: detect `update-initramfs` failure and abort build with useful message if it does
- v4: show useful stacktrace in `exit_with_error`
  - if `ERROR_DEBUG_SHELL=yes`, drop into a shell before unmounting/deleting everything, so we can inspect what went wrong
- v4: display a message before `apt-get remove PACKAGE_LIST_BOARD_REMOVE` packages, so any errors while removing are easy to understand
- v4: preserve kernel .config's dates when copying

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions framework; refactor tool fetching/building into extensions

- a few examples of core refactoring using extensions
- sunxi-tools extension, enabled by 2 different sunxi family includes ("reuse" example)
- marvel-tools extension, enabled by 2 different mvebu family includes
- rkbin-tools extension, enabled by rockship64_common family include
- amlogic-fip/c2-blobs stuff refactored directly into meson64_common.inc ("single-use" example)
- removed the 'testings' fetch_from_repo completely since not used anywhere.

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* .wip's for UEFI arm64 and UEFI/BIOS x86 via new GRUB extension

- v3: added `growroot`-awareness to `armbian-resize-filesystem`
    - the partition-growing part of `armbian-resize-filesystem` does not deal correctly with the UEFI layout
    - `growroot` is installed on UEFI images by default, that handles growing partition during initramfs
    - now `armbian-resize-filesystem` handles `resize2fs` only, and works.
- v4: reworked UEFI board/family/include structure:
    - use Distro's `linux-generic` kernel only for `current`
    - `edge` now builds it's own pure-mainline `5.15.y` kernel, for both x86 and arm64
        - `.config` taken from Ubuntu, probably needs tuning for EXTRAWIFI=yes et al
- v4: introduce `SKIP_KERNEL_SYMLINK=yes`, tested in `builddeb`
    - to avoid symlinking kernel; u-boot likes it, but grub and flash-kernel hates it
- v5: many fixes
- v7: more small fixes.

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* .wip for the RaspberryPi 4B via new flash-kernel extension

- this does not build it's own kernel "yet", but uses default linux-raspi kernel from Ubuntu
- flash-kernel is not really a bootloader
    - it just prepares kernel et al a FAT partition for booting by the RPi4b bootloader
    - flash-kernel is standard Debian package, but has only been tested on Ubuntu releases
- it is really only known-working since Hirsute release.
  - Debian's rpi kernel is armhf only, so out of scope here, at least until we add source-built kernels.
- v3: fixed focal rootfs build. untested.
- v3: better variable names, preparing for source-built kernel.
- v5: new edge build with pure mainline kernel.
- v6: many fixes and some hacks for packaging and layout, also firmware (using Ubuntu's)

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* Added first patch to edge x86 related to wifi drivers

* extensions: leave hostapd alone; remove hackish ext; block reentrancy

- package-list-utils does not belong in this PR
- grub or bcm2711 is not the place to remove hostapd
- block recursive enable_extension() calls, for now.

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* gen-sample-extension-docs: fix: avoid counter in generated sample

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions: dependencies: enable_extension() in extensions with a stack

- and better stacktraces, I hope

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* Remove code from package list since we don't have it in repository
Adjust kernel config to disable driver that needs further polishing.

* Allow amd64 to build the same desktops as aarch64. We only have this limit for armhf, where some desktops don't work

* amd64: allow building amd64 on aarch64 with system toolchain

- conditionally add gcc-x86-64-linux-gnu to hostdeps

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* add libelf-dev directly to hostdeps (and Dockerfile), remove extension

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* packaging: remove SKIP_KERNEL_SYMLINK hack, fix the root cause

- which was the missing $image_name for non-arm64 & non-arm, so: x86 for example

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* grub: really obliterate u-boot stuff from BSP

- for now. soon we'll refactor u-boot so not have to do this

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* flash-kernel: really obliterate u-boot stuff from BSP

- for now. soon we'll refactor u-boot so not have to do this

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions: add host_dependencies_ready() hook

- this passes FINAL_HOST_DEPS containing all hostdeps for the run after they're installed

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* Add verification functions for correct selection.

* If UEFI Skip symlink creation

* Do not create dtb package for amd64

* Skip scripts folder cleaning if build process native.

Skip creating  postinst prerm scripts  for headers.

* Skip applying headers-debian-byteshift.patch if build native

* Fix architecture syntax as x86_64

* Revert "amd64: allow building amd64 on aarch64 with system toolchain"

This reverts commit 0c5ee20bb1.

* Compare architectures before starting compilation.



Signed-off-by: The-going <48602507+The-going@users.noreply.github.com>

* extensions: cleanups after fixes by the-Going

- packaging:
  - there is _no need_ anymore for the symlink hack, CONFIG_EFI or no. But check is great, see below
  - it's not `amd64` that has no DTB's, it's all UEFI, thus: `is_enabled CONFIG_EFI`, thanks!
- Explicitly disallow "reverse cross compile" in amd64.conf.
- whitespace-only-deletions: revert. we shall shellfmt the whole thing one day, but not today.
- fix a few syntax warnings in newly introduced code (floating `$ARCH` vs `"${ARCH}`) - blame shellcheck

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* packaging: fix: turns out a lot of boards have CONFIG_EFI=y, can't use that for dtb/no-dtb decision.

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* grub: remove debug

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* firmware: allow installing `armbian-firmware-full`; make it really full

- can now use `BOARD_FIRMWARE_INSTALL="-full"` to install full firmware for the board. enable for UEFI.
- don't rely on KERNELSOURCE for firmware-related decisions. introduce `INSTALL_ARMBIAN_FIRMWARE` which defaults to `yes`
- rpi4b/flash-kernel: disable Armbian firmware; we need linux-firmware-raspi2, which conflicts.

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions: log to /${LOG_SUBPATH}/ instead of fixed /debug/

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions: introduce cleanup_extension_manager() called by build-all-ng's unset_all()

- to reset/unset everything done by the the initializer, so build can run again

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions: remove 'global' logging, for use with build_all_ng

- enable_extensions() will have to live on without logging to file. it's just too early.
- now init EXTENSION_MANAGER_TMP_DIR in initialize_extension_manager()
- now init EXTENSION_MANAGER_LOG_FILE in initialize_extension_manager()

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* extensions: build-all-ng.sh bugfix due to extension's debug to stdout

- extensions (among other things) can produce output to stdout when activated
- fix: check_hash() produced "idential" (sic, now changed to IDENTICAL) to stdout as a trigger
- debugging output got mixed with "idential", rendering hash cache void for families that used extensions
  - eg: sunxi, others
- fix is to send stdout to the bitbucket when sourcing the board & arch config files
- proper fix would be stop using stdout in this case and use return code for check_hash()
  - one day soon

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* Add CI build targets

Co-authored-by: Igor Pecovnik <igor.pecovnik@gmail.com>
Co-authored-by: The-going <48602507+The-going@users.noreply.github.com>
2021-12-06 09:49:49 +01:00
Igor Pečovnik
e163aa34fb Enable Netplan.io (#3248)
* Enable Netplan in Ubuntu Focal

* Add netplan to other variants

* Force rootfs update
2021-11-15 23:34:36 +01:00
Igor Pečovnik
2a47ad5c78 Add support for next Ubuntu LTS (#3234)
* Add support for next Ubuntu LTS

* Update build-cache.yml

* Update build-cache.yml

* Update build-cache.yml

* Update build-cache.yml

* Jammy repository creation

* Enable docker build support on Jammy
2021-11-09 18:15:04 +01:00
Igor Pecovnik
14904824dd Force rootfs cache rebuild 2021-10-31 16:43:42 +01:00
Ricardo Pardini
be5f2b8c75 introduce DEB_COMPRESS for dpkg-deb+kernel's KDEB_COMPRESS (#3222)
* introduce DEB_COMPRESS for dpkg-deb+kernel's KDEB_COMPRESS

- Avoids zstd which is now default in Ubuntu Impish, but can't be read by Debian dpkg
- Allows for xz, gzip, or none; defaults to xz; use gzip for faster, and none for much faster.

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>

* use KDEB_COMPRESS in packages/armbian/builddep for 5.12+ kernels too

Signed-off-by: Ricardo Pardini <ricardo@pardini.net>
2021-10-27 18:55:12 +02:00
Igor Pecovnik
5f947341ba Bump rootfs cache version number to force rootfs creation
Also add an option to set those two cache parameters from the outside
2021-10-14 21:31:15 +02:00
Igor Pecovnik
26af7fe00c Force rootfs caches rebuild
We have few corrupted caches and almost all desktop caches has to be rebuild due to added packes. It makes sense to refresh it.
2021-09-09 23:21:20 +02:00
lanefu
894cb29ca7 Feature/get_log_path/ (#3064)
* custom log path of debug/ via LOG_SUBPATH=
2021-07-27 07:24:06 -04:00
Igor Pecovnik
02e0deb23d Removing duplicates in packages and bumping with a rootfs cache id 2021-07-27 12:57:33 +02:00
Igor Pečovnik
a7e16da2ce Add Ubuntu Impish (#3006) 2021-07-14 08:45:30 +02:00
The-going
241f015ac7 formatted output of variable values (#2986)
Signed-off-by: The-going <48602507+The-going@users.noreply.github.com>
2021-07-13 20:25:25 +02:00
Igor Pečovnik
9249d3f954 Add support for desktop board support package (#2972)
* Add support for desktop board support package

Extracting from https://github.com/armbian/build/pull/2776 which can be closed after.
* Desktop BSP creation is working, but need broader testing and some quick how-to
* Create empty files as examples where we can put things.
* Fixing Pinebook desktop bsp creation
* We need to have information about ARCH in the desktop bsp package. This ain't universal.
2021-07-06 00:59:50 +02:00
Igor Pečovnik
1f2e97febf Remove Ubuntu Groovy from codebase (#2952) 2021-07-03 21:58:08 +02:00
Igor Pečovnik
fcb8b6f195 Code cleaning attempts (#2925)
* Move function run_on_sdcard to more appropriate location

* Focus on bash headers, unify, adjust (c)

* Adjust (c) to range

* Cleanup compile.sh

* Cleaning main

* Build all, main configs

* fine tunning

* Improve readme
2021-06-26 23:36:47 +02:00
lanefu
1bc196089a use official GitHub repo for u-boot GitHub mirror 2021-05-15 15:56:32 -04:00
Igor Pecovnik
84d7079c5d Bump with roofs cache and small cache recreation optimisation 2021-05-13 21:30:06 +02:00
Przemysław Sztoch
486a821a55 VENDOR, rock-pi-n10, RM_LIST for armbian-config/zsh, Static MAC for networkd (#2789)
* VENDOR, rock-pi-n10, RM_LIST for armbian-config/zsh

VENDOR parameter in configuration
zsh is the best of course...
Static MAC for networkd

ROCK Pi N10 (Radxa) - Initial support, knwon bugs:
- Problem with poweroff (kernel panic).
- DeviceTree is not fully verified, I suspect missing and unnecessary
entries.
- No support for NPU
- u-boot without NVME support
- I have doubts if I chose the right preboot loader for RK3399Pro, NPU
  and installed DDR memory.

* New RK3399Pro boot preloaders.

* rootfs checksum, DEBOOTSTRAP_LIST fix, output.log fix

Repaired checksum for rootfs
Added use PACKAGE_LIST_RM to purge DEBOOTSTRAP_LIST
Better logging to output.log
Fix for new host distribution detection (ie Debian Bullseye).
ubootdebs as new cleaning option
2021-05-11 21:50:14 +02:00
Igor Pecovnik
70b985d90a Bump with version 2021-05-06 11:51:17 +02:00
Igor Pečovnik
d3997fe64c Cache creation improvements (#2806) 2021-05-05 20:49:25 +02:00
Igor Pecovnik
f43db66d32 Cosmetical in backend
Supress error when we rebuild rootfs cache
2021-05-03 00:08:55 +02:00
Igor Pečovnik
9b0dd2b839 Fix broken Docker support (#2804)
* lsb-release package is getting deprecated

It is not present on Hirsute anymore so we better remove its dependency

* - remove lsb-release dependency from docker
- exit if Docker version is too low (check works only on Ubuntu builds, else doesn't do anything)

Reference: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1916485

* Cleanup

* Remove quotes for proper pkg install

* Automatically upgrade Docker engine

* Missing python dependencies

* Bugfix - debian name is not just one word

* Put this back

* provoke error

* Revert

* Adjust Github runner to make bionic and focal and compile u-boot for test

* Adjust cron

* Remove Bionic, cron back to 14 days

* Change to ubuntu-latest
2021-05-02 18:06:38 +02:00
Przemyslaw Sztoch
738996ea33 Better user path config and support for etworkd
add user path for configuration scan
check for NetworkManager/networkd
support for bullseye

(i.e. path: config/cli and files: packages, packages.remove)
2021-04-09 10:27:52 +02:00
Igor Pečovnik
2eb08df632 Use MAINLINE_MIRROR for creating armbian-firmware package (#2703)
We are using several alternative sources for main Linux kernel code, but not for linux-firmware which is also mirrored the exact same way. This will have major impact on users that are located in Asia and use mirrors from there.
2021-03-18 17:14:11 +01:00
Igor Pecovnik
720b8a8259 Adjust mainline u-boot to new location to supress redirect warnings 2021-03-07 11:05:35 +01:00
Igor Pečovnik
5acb2964da Small desktop packages rfc (#2691)
* Small packages refactoring / optimisations

* Update targets, remove non existing group
2021-03-05 21:53:12 +01:00
柚木鉉
98c1cbc889 Add bfsu mirror as another mirror of tuna (#2682)
* add bfsu for china mirror
2021-03-03 17:45:25 +01:00
Igor Pecovnik
7cd178f64a Bugfix - fixing wrong value extraction
Closes https://github.com/armbian/build/issues/2672
2021-03-02 10:55:22 +01:00
Igor Pečovnik
2e594375a9 Improve cache and toolchain download (#2665)
* Improve cache and toolchain download

* Cleanup
2021-02-26 18:27:21 +01:00
Igor Pečovnik
3a74abe43d Merge desktop branch (#2662)
Co-authored-by: Miouyouyou (Myy) <myy@miouyouyou.fr>
Co-authored-by: Igor Pečovnik <igorpecovnik@users.noreply.github.com>
Co-authored-by: Richard Neese <r.neese@gmail.com>
Co-authored-by: iav <iav@iav.lv>
Co-authored-by: lanefu <lanefu@users.noreply.github.com>
Co-authored-by: Werner <EvilOlaf@users.noreply.github.com>
Co-authored-by: Oleg <balbes-150@yandex.ru>
Co-authored-by: Lane Jennison <lane@lane-fu.com>
Co-authored-by: JMCC <jmcc1@gmx.com>
2021-02-24 18:19:09 +01:00
Ash
ab60a86bd9 Do not overwrite EXIT_PATCHING_ERROR if already defined (#2652) 2021-02-23 09:41:19 +01:00
Igor Pecovnik
705bbd431b Bump rootfs cache - it was recreated
- tweak buil all images
- add more info to the error
2021-01-25 13:23:39 +01:00