Pi 3 image doesn't boot when compiled with GCC version 10 or later

[piersfinlayson]

I've been experimenting with building pi1541 with various ARM GNU toolchain versions. Using up to 9.2-2019.12 produces a working Pi 3 kernel.img. However beyond that (starting with 10.2-2020.11) the kernel.img no longer boots.

I've been unable to find the cause for this, although the 10.2 and onwards binary is marginally smaller (473,008 bytes from 9.2 and 468,880 bytes from 10.2).

The Pi 3 kernel.img version from https://cbm-pi1541.firebaseapp.com/kernel.zip is 502,664 bytes, and looks (from the .travis.yml file) to have been built using GCC 5.4.

The most up to date version of the ARM GNU toolchain which I can getting working is therefore:

gcc-arm-9.2-2019.12-x86_64-arm-none-eabi.tar.xz

I've created a docker based script which builds pi1541 and creates the SD card, using GCC 9.2, here: https://github.com/piersfinlayson/pi1541-build

[piersfinlayson]

I've tracked down the problem as being due to the G++ compiler (when using at least 10.2 and later) as generating code with unaligned data access. Specifically I've seen a STRH (store half word) instruction be applied to an odd memory address in m6522::m6522 causing a data abort exception.

Adding the -mno-unaligned-access flag to $ARCH in Makefile.rules for the RASPPI=3 fixes the problem. Like so:

ARCH ?= -march=armv8-a+crc -mtune=cortex-a53 -mfpu=crypto-neon-fp-armv8 -mfloat-abi=hard -marm -mno-unaligned-access -DRPI3=1

And also for the RASPPI=2 line:

ARCH ?= -march=armv7-a -mtune=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard -marm -mno-unaligned-access -DRPI2=1 -DEXPERIMENTALZERO=1

To debug, as this exception occurs when setting up the globals, you have to move UART intialization from main.cpp to before the __libc_init_array() call in armc-cstartup.c, and then write some text out via the UART. This means the exception handler can then use the UART to indicate the problematic code. It can also be helpful to add -g to the CPPFLAGS (Makefile.rules) and -S to the objdump (Makefile) in order to create instrumented assembler in kernel.lst.

There doesn't seem to be a need to add this flag to any other RASPPI option, probably because per the GCC explanation for -mno-unaligned-access "default unaligned access is disabled for all pre-ARMv6, all ARMv6-M and for ARMv8-M Baseline architectures". So it makes sense that a fix is needed for both RASPPI=2 (ARMv7) and RASPPI=3 (ARMv8-A).

The Cortex A53 (RASPPI=3) technical reference indicates the unaligned memory access is allowed for normal memory, but not for "device" memory. I'm not sure how this normal/device distinction is decided, and it looks like whether alignment is checked on normal memory can be specified using the SCTLR.A bit (and SCRLR_ELx.A bits). Hence it's possible there's another fix here (in setting the A bit in the approriate registers), but I've gone down this rat-hole far enough, so I'm opting the no-unaligned-access setting to get this working.

I used the ARM GNU toolchain 12.3 Rel1 (as well as 10.2 and 10.3) to test this fix. 12.3 Rel1 for an x86_64 host can be downloaded here. arm-gnu-toolchain-12.3.rel1-x86_64-arm-none-linux-gnueabihf.tar.xz

I'll submit a PR.