Bug 26028 - Kernel file compression method for aarch64
Summary: Kernel file compression method for aarch64
Status: RESOLVED OLD
Alias: None
Product: Mageia
Classification: Unclassified
Component: RPM Packages (show other bugs)
Version: 7
Hardware: aarch64 Linux
Priority: Normal normal
Target Milestone: ---
Assignee: Kernel and Drivers maintainers
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-07 14:51 CET by Jybz
Modified: 2021-09-07 14:10 CEST (History)
5 users (show)

See Also:
Source RPM:
CVE:
Status comment:


Attachments

Description Jybz 2020-01-07 14:51:04 CET
I try to make an image of Mageia for the Raspberry Pi 3 B+ which is an ARMv8 SoC.

I manage to start the compiled u-boot, using the extlinux.conf of the ext4 FS.
But it complains when starting the kernel :
```
Raspberry Pi Bootcode

Found SD card, config.txt = 1, start.elf = 1, recovery.elf = 0, timeout = 0
Read File: config.txt, 1506 (bytes)
[...]
U-Boot 2020.01 (Jan 07 2020 - 11:07:48 +0100)

DRAM:  948 MiB
RPI 3 Model B (0xa02082)
MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   No ethernet found.
starting USB...
Bus usb@7e980000: scanning bus usb@7e980000 for devices... 3 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Scanning mmc 0:2...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
424 bytes read in 5 ms (82 KiB/s)
1:      linux
Retrieving file: /boot/initrd.img
96226816 bytes read in 4267 ms (21.5 MiB/s)
Retrieving file: /boot/vmlinuz
8660591 bytes read in 388 ms (21.3 MiB/s)
append: root=UUID=74a0b16e-92f5-4d02-a37f-68637392e159 earlyprintk console=ttyS0,115200 kgdboc=ttyS0,115200 console=tty1 elevator=deadline rootwait
Retrieving file: /usr/lib64/linux-5.4.6-desktop-2.mga7/bcm2837-rpi-3-b-plus.dtb
14446 bytes read in 30 ms (469.7 KiB/s)
Bad Linux ARM64 Image magic!
SCRIPT FAILED: continuing...
Card did not respond to voltage select!

Device 0: unknown device
Waiting for Ethernet connection... unable to connect.
[...]
```
Uboot does not try the bootz command for the kernel, but, as it is aarch64, tries the booti command.
Booti : https://docs.khadas.com/vim2/UBootUsage.html (I didn't find other doc)
```
booti   - boot arm64 Linux Image image from memory
```
From u-boot source, we can link the Error "Bad Linux ARM64 image" with booti command via the function name 'booti_setup' :
https://gitlab.denx.de/u-boot/u-boot/blob/master/arch/arm/lib/image.c#L29

Furthermore, from Raspberry website :
https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md
```
arm_64bit
[...]
NOTE: 64-bit kernels must be uncompressed image files.
```
Maybe it isn't related.

The current kernel is :
```
[jybz@localhost boot]$ file ./vmlinuz-5.4.6-desktop-2.mga7 
./vmlinuz-5.4.6-desktop-2.mga7: gzip compressed data, max compression, from Unix, original size modulo 2^32 24000520
```

---

Oh ! Update (before to submit the bug request)
I ask on #u-boot chan for ideas, I got this answer :
[14:10] <Tartarus> jeeebz: There's some patches to support automatic gunzip of an Image.gz but they aren't merged yet and I need to give some feedback again as doing that automagically is a little tricky
[...]
[14:32] <Tartarus> jeeebz: it's functional the problem is what's the best way to feed in the size of the compressed data automatically

So there is curently no u-boot support for vmlinuz on aarch64. But it is on the way.
There are nowadays many arm devices on aarch64.

Should our kernel wait (how long ?) for U-boot and continuing with gzip, or should we, temporary (as long as u-boot is not able to manage it) not compress the kernel, or should we packages 2 different versions of the kernel, one compressed, one not ?

Jybz
Comment 1 Jani Välimaa 2020-01-07 17:12:19 CET
I think you should ask from #fedora-arm how Fedora boots and uses u-boot with aarch64.

CC: (none) => jani.valimaa

Comment 2 Lewis Smith 2020-01-08 10:07:29 CET
Thanks Jani for your comment.

Assigning this necessarily to the kernel group.

Assignee: bugsquad => kernel
Summary: Kernel file for aarch64 => Kernel file compression method for aarch64

Comment 3 Thomas Backlund 2020-01-08 10:16:04 CET
Also, please use the kernel-5.4.8-3.mga7 that is in updates_testing, as that has several changes to the arm64 defconfigs (compared to the 5.4.6-2 you are testing with)

CC: (none) => tmb

Comment 4 Pascal Terjan 2020-01-08 12:44:45 CET
Can you try decompressing the kernel when building the image? (zcat vmlinuz > vmlinux should do)

Using https://github.com/pbatard/RPi3 or https://github.com/pftf/RPi4 would also allow to use standard bootloader but we can't really ask people to change the firmware :(

CC: (none) => pterjan

Comment 5 Jybz 2020-01-08 13:42:58 CET
Yes, it can be done in the script process to build aarch64 images. But I can't manage it for kernel updates.


I did, one step further !

> U-Boot 2020.01 (Jan 07 2020 - 11:07:48 +0100)
> 
> DRAM:  948 MiB
> RPI 3 Model B (0xa02082)
> MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
> Loading Environment from FAT... *** Warning - bad CRC, using default environment
> 
> In:    serial
> Out:   vidconsole
> Err:   vidconsole
> Net:   No ethernet found.
> starting USB...
> Bus usb@7e980000: scanning bus usb@7e980000 for devices... 3 USB Device(s) found
>        scanning usb for storage devices... 0 Storage Device(s) found
> Hit any key to stop autoboot:  0 
> switch to partitions #0, OK
> mmc0 is current device
> Scanning mmc 0:1...
> Scanning mmc 0:2...
> Found /boot/extlinux/extlinux.conf
> Retrieving file: /boot/extlinux/extlinux.conf
> 411 bytes read in 5 ms (80.1 KiB/s)
> 1:      linux
> Retrieving file: /boot/initrd.img
> 96226816 bytes read in 4267 ms (21.5 MiB/s)
> Retrieving file: /boot/vmlinux
> 24000520 bytes read in 1087 ms (21.1 MiB/s)
> append: root=UUID=74a0b16e-92f5-4d02-a37f-68637392e159 earlyprintk console=ttyS0,115200 kgdboc=ttyS0,115200 elevator=deadline rootwait
> Retrieving file: /usr/lib64/linux-5.4.6-desktop-2.mga7/bcm2837-rpi-3-b-plus.dtb
> 14446 bytes read in 31 ms (455.1 KiB/s)
> ## Flattened Device Tree blob at 02600000
>    Booting using the fdt blob at 0x2600000
>    Using Device Tree in place at 0000000002600000, end 000000000260686d
> 
> Starting kernel ...
> 
> 

No more "Bad Linux ARM64 Image magic!" but it stops the boot process here...
I isn't the kernel from update_testing. I will trigger a build later, but I will not have a Rpi3b+... I will keep you inform for when I will try it.
Comment 6 Pascal Terjan 2020-01-09 15:36:57 CET
I add a quick look last night, and u-boot has also a bootefi command.

https://github.com/u-boot/u-boot/blob/master/doc/uefi/uefi.rst#executing-a-uefi-binary

So it seems possible to just install grub2-efi like our tools know how to do and just use u-boot to load grub on those devices not using UEFI
Comment 7 Jybz 2020-01-09 15:45:29 CET
Just for your information : I could boot with some kernel options and the uncompressed vmlinux :

> Mageia release 7 (Official) for aarch64
> Kernel 5.4.8-desktop-3.mga7 on a 4-processor aarch64 / ttyS0
> rpi login: root
> You are required to change your password immediately (administrator enforced)
> New password: 
> Retype new password: 
> [root@rpi ~]# uname -a
> Linux rpi 5.4.8-desktop-3.mga7 #1 SMP PREEMPT Tue Jan 7 16:37:55 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
> [root@rpi ~]# cat /sys/firmware/devicetree/base/model && echo ""
> Raspberry Pi 3 Model B+

I will take a look on this feature.
But how "our" grub2 will behave on a serial port ? We will see...
Comment 8 Pascal Terjan 2020-01-09 15:54:30 CET
You should also be able to boot the kernel directly using bootefi, the main difference is that our tools know how to update the grub config when adding new kernel, change command line options, etc
Comment 9 Jybz 2020-01-09 16:00:23 CET
Our tools also manage the updates with uboot, modifying the /boot/extlinux/extlinux.conf file, also with reusing the kernel arguments.

I will try the efi method, but I don't know what I should install yet.
Comment 10 Pascal Terjan 2020-01-09 16:02:43 CET
Ah I didn't know it didn't last time I checked many years ago when it did lilo and grub :)

Then just using bootefi instead of booti in the config may be the most simple change
Comment 11 Jybz 2020-01-10 17:19:30 CET
A propos, I see that grub2-efi is only available for aarch64.
I'm trying to use EFI for the Rpi3b+ now, if it is successful, I would like to do all images with EFI, also for armv7hl.

Why is there no grub2-efi on armv7hl ? Is it possible ?
Comment 12 r howard 2020-01-11 03:49:25 CET
@Jybz as far as I know there is no UEFI support for armv7hl.
The only UEFI support I have seen for ARM has been for AARCH64 based boards. 
Just be aware that boards that run armv7hl natively tend to be supported by u-boot.

CC: (none) => rihoward1

Comment 13 Jybz 2020-01-27 12:17:32 CET
How Ubuntu managed to boot compressed kernel on aarch64, as Clever on IRC did it perfectly, I will copy/paste :

> [11:31] <jeeebz> Hello ! few days ago (maybe weeks...) one wrote me here that there are no possibilities for aarch64 to uncompresse vmlinuz, not with uboot, nor with the firmware of the raspberry pi. But a colleage and I wonder how Ubuntu manage with their compressed kernel to boot. Does anyone have an idea ?
> [11:38] <clever> jeeebz: is it a 64bit kernel? on an rpi?
> [11:40] <jeeebz> Yes
> [11:41] <clever> jeeebz: what is the content of config.txt on that install?
> [11:43] <jeeebz> I'm leaving for lunch, I'm also downloading the image to inspect it.
> [11:44] <jeeebz> clever: from that web page : https://ubuntu.com/download/raspberry-pi part 1) 64b for Rpi 3
> [11:53] <clever> [pi3]
> [11:53] <clever> kernel=uboot_rpi_3.bin
> [11:53] <clever> jeeebz: its configured to load a different uboot bin file, for each model
> [11:54] <clever> and it supports pi2, pi3, and pi4
> [11:56] <clever> jeeebz: and the vmlinuz file is a PE executable...
> [11:57] <clever> jeeebz: at a glance, id say its an efi binary?
> [12:00] <clever> and if you read boot.scr ...
> [12:00] <clever> load ${devtype} ${devnum}:${distro_bootpart} ${ramdisk_addr_r} vmlinuz
> [12:00] <clever>     # It's a gzip compressed image (NOTE: *not* a self-extracting gzip
> [12:00] <clever>     # compressed kernel, just a kernel image that has been gzip'd). This is
> [12:00] <clever>     # how all kernels *should* be shipped on Ubuntu for RPi in future,
> [12:00] <clever>     unzip ${ramdisk_addr_r} ${kernel_addr_r}
> [12:00] <clever> jeeebz: its using a custom u-boot script to just unzip it in ram, before executing it

What are your suggestions ?

I tried with EFI, but without success. I would need much more time to debug it. I prefer to use the regular ways, firmware → [uboot →] kernel, and with extlinux.conf if possible.
Comment 14 Jybz 2020-01-29 11:20:00 CET
I've an idea, please comment it :

When there is a kernel update, the script /sbin/installkernel is executed.
What about adding these lines :
> zcat /boot/vmlinuz >/boot/vmlinux
> sed -i -e 's/vmlinuz/vmlinux/g' /boot/extlinux/extlinux.conf
generate a diff patch,
copying this patch inside /boot (in order to revert it in the future, when u-boot will manage vmlinuz for aarch64),
applying this patch in the build process to generate mageia's images for aarch64.


Or maybe, to handle multiple kernel versions :
> for VMLINUZ in /boot/vmlinuz* ; do
>     VMLINUX=$(echo ${VMLINUZ} | sed -e 's/vmlinuz/vmlinux/g' )
>     zcat ${VMLINUZ} >${VMLINUX}
> done
> sed -i -e 's/vmlinuz/vmlinux/g' /boot/extlinux/extlinux.conf

What do you think about ?
Pro : this patch will be applied only on particular images (i.e. boards), no changes on mageia's BS. It can be reversed.
Con : when a package (which one ???) will be updated, it will removes the patched /sbin/installkernel , then a kernel will be uninstalled, the associated vmlinux remains (even if it is only few Mo).
Comment 15 Pascal Terjan 2020-01-29 15:41:07 CET
If we want to provide a decompressed one wen using uboot on aarch64, we should just make the tools do so, they know the bootloader used and the arch.
I have ordered a RPi4 which should be delivered on Monday, so I'll try to have a look at some point next week.
Comment 16 Jybz 2020-05-11 22:02:02 CEST
News from uboot :
https://gitlab.denx.de/u-boot/u-boot/-/commit/414c34ed555b8ce5c260cf

>  image: Add compressed Image parsing support in booti.
> 
> Add compressed Image parsing support so that booti can parse both
> flat and compressed Image to boot Linux. Currently, it is difficult
> to calculate a safe address for every board where the compressed
> image can be decompressed. It is also not possible to figure out the
> size of the compressed file as well. Thus, user need to set two
> additional environment variables kernel_comp_addr_r and filesize to
> make this work.
> 
> Following compression methods are supported for now.
> lzma, lzo, bzip2, gzip.
> 
> lz4 support is not added as ARM64 kernel generates a lz4 compressed
> image with legacy header which U-Boot doesn't know how to parse and
> decompress.

So, I still have to find another Rpi to try if it works now, I hope we can close this bug report afterwards ;)
Comment 17 r howard 2020-05-12 00:58:32 CEST
Jbyz that was committed 3 weeks ago to U-boot so will be in U-boot v2020.07 when it is released. Currently that is at rc2. 07 indicates it will be released in July. There is a U-boot  release each quarter of the year.
Comment 18 Aurelien Oudelet 2020-10-06 16:27:51 CEST
U-Boot 2020.10 is released upstream.
Comment 19 Filip Komar 2021-04-06 01:56:29 CEST
Sill open? It seems to work nicely on my RPi4 4G.

CC: (none) => filip.komar

Comment 20 Aurelien Oudelet 2021-07-06 13:17:18 CEST
Mageia 7 is EOL since July 1st 2021.
There will not have any further bugfix for this release.

You are encouraged to upgrade to Mageia 8 as soon as possible.

@reporter, if this bug still apply with Mageia 8, please let us know it.

@packager, if you work on the Mageia 7 version of your package, please check the Mageia 8 package if issue is also present. In this case, please fix the Mageia 8 version instead.

This bug report will be closed OLD if there is no further notice within 1st September 2021.
Comment 21 Marja Van Waes 2021-09-07 14:10:53 CEST
Hi bug reporter and hi assignee and others involved,

Please reopen this bug report if it is still valid for Mageia 8 or 9(cauldron), and change "Version:" in the upper left of this report accordingly.

This report is being closed as OLD because it was filed against Mageia 7, for which  support ended on June 30th 2021.

Thanks,
Marja

Resolution: (none) => OLD
Status: NEW => RESOLVED


Note You need to log in before you can comment on or make changes to this bug.