std: implement basic io and improve alloc in uefi #22226

RossComputerGuy · 2024-12-13T09:04:45Z

Largely based on #19486, this PR marks me starting to work on UEFI support in Zig again. This time, there will be multiple PR's instead of one so it becomes easier to manage. This PR simply makes the zig init example build.

I tested this in QEMU by loading in OVMF, entering the UEFI shell, loading FS0:\bin\$.efi. The zig-out can be shared by adding: -drive file=fat:rw:zig-out.

alexrp · 2024-12-13T15:39:49Z

lib/std/os/uefi.zig

@@ -1,5 +1,7 @@
 const std = @import("../std.zig");

+pub const posix = @import("uefi/posix.zig");


Last I talked with Andrew about the future of std.posix, one thing we agreed on is that probably all Windows code in std.posix should be deleted. I would expect that the exact same thing is true of UEFI, as it is also not a POSIX platform in any meaningful sense?

Yeah, I agree. The UEFI POSIX stuff is just a POSIX wrapper around the UEFI protocols. We can always ditch it. UEFI is close to Windows so I think whatever we do for Windows and POSIX, it would apply to UEFI.

alexrp · 2024-12-13T15:43:05Z

lib/std/Thread/Futex.zig

@@ -66,7 +66,7 @@ pub fn wake(ptr: *const atomic.Value(u32), max_waiters: u32) void {
    Impl.wake(ptr, max_waiters);
 }

-const Impl = if (builtin.single_threaded)
+const Impl = if (builtin.single_threaded or builtin.os.tag == .uefi)


Should UEFI be treated as a proper single-threaded target?

zig/src/target.zig

Lines 66 to 81 in 82f35c5

pub fn alwaysSingleThreaded(target: std.Target) bool {

_ = target;

return false;

}

pub fn defaultSingleThreaded(target: std.Target) bool {

switch (target.cpu.arch) {

.wasm32, .wasm64 => return true,

else => {},

}

switch (target.os.tag) {

.haiku => return true,

else => {},

}

return false;

}

Yes, I just didn't know where the source was which defined that heh.

I've updated this and changed the defaultSingleThreaded for UEFI.

Out of curiosity, is UEFI not always single threaded instead of just default single threaded?

You technically could implement a form of multithreading, just like with wasm. It's just builtin.single_threaded will default to true when single_threaded is not changed in your build. You probably could write a custom scheduler + mutltithreading stuff and use root.os stuff to get it working but no implementation will be made for upstream.

I learned somewhat recently that UEFI does have EFI_MP_SERVICES_PROTOCOL so there is a way to do multiprocessing but not multithreading but likely is still out of the scope.

linusg · 2024-12-19T11:08:15Z

lib/std/fs.zig

@@ -52,7 +52,7 @@ pub const MAX_PATH_BYTES = @compileError("deprecated; renamed to max_path_bytes"
 /// * On other platforms, `[]u8` file paths are opaque sequences of bytes with
 ///   no particular encoding.
 pub const max_path_bytes = switch (native_os) {
-    .linux, .macos, .ios, .freebsd, .openbsd, .netbsd, .dragonfly, .haiku, .solaris, .illumos, .plan9, .emscripten, .wasi => posix.PATH_MAX,
+    .linux, .macos, .ios, .freebsd, .openbsd, .netbsd, .dragonfly, .haiku, .solaris, .illumos, .plan9, .emscripten, .wasi, .uefi => posix.PATH_MAX,


This looks wrong, I'd assume it needs to use similar logic as windows

I think this ties in with #22226 (comment)

linusg · 2024-12-19T11:10:15Z

lib/std/fs/Dir.zig

+
+        pub fn next(self: *Self) Error!?Entry {
+            _ = self;
+            return null;


Not sure if turning a compile error into a non-functional stub is that useful? Most people will assume the standard library functions work for a certain target if they compile.

Yeah, I hadn't figured out how to implement it just yet. I'll leave it out of this PR and work on it later on.

linusg · 2024-12-19T11:13:49Z

lib/std/os/uefi/allocator.zig

+/// Allocates memory in pages.
+///
+/// This allocator is backed by `allocatePages` and is therefore only suitable for usage when Boot Services are available.
+pub const PageAllocator = struct {


std.os.uefi.allocator.{Page,Pool}Allocator violates https://ziglang.org/documentation/master/#Avoid-Redundant-Names-in-Fully-Qualified-Namespaces

I have changed it now, hopefully it doesn't violate the fully qualified namespace stuff.

linusg · 2024-12-19T11:16:26Z

lib/std/start.zig

+                if (@errorReturnTrace()) |trace| {
+                    std.debug.dumpStackTrace(trace.*);
+                }
+                std.time.sleep(5 * std.time.ns_per_s);


Magic sleep needs a comment

This part of the code is based on what @truemedian did. I'm not sure either so CC'ing them for some information.

Magic sleep here was a debugging aid to prevent the uefi firmware from clearing the screen before I could read the any output because once EfiMain returns it will go to the next boot loader. Not necessary at all, unless that window to eead the error is desired.

Ok, I likely will remove it and it'll be up to the program to do that if they want to.

linusg

I'll have to ask you to split this up even more - there are number of changes in here that need more discussion or simply won't be merged as-is. I think the allocator improvements should be their own PR, not touching anything unrelated.

The changes around threading are nonsensical - UEFI is by definition single threaded, so alwaysSingleThreaded() ought to be the function that gets modified, which in turn means the panic handler doesn't need a stub for std.Thread.getCurrentId().

RossComputerGuy · 2025-02-08T02:36:30Z

I'll have to ask you to split this up even more - there are number of changes in here that need more discussion or simply won't be merged as-is. I think the allocator improvements should be their own PR, not touching anything unrelated.

Will do

The changes around threading are nonsensical - UEFI is by definition single threaded, so alwaysSingleThreaded() ought to be the function that gets modified, which in turn means the panic handler doesn't need a stub for std.Thread.getCurrentId().

Welll, kinda. The UEFI PI spec does mention a way to do multiprocessing which in turn can lead to multithreading.

linusg · 2025-02-08T02:45:12Z

Welll, kinda. The UEFI PI spec does mention a way to do multiprocessing which in turn can lead to multithreading.

You don't even need to run code on another core for "threading", but from the standard library's point of view we always have a single thread. Preparing for a scenario that is unlikely to ever be useful and only creates the need for more stub code isn't desirable, let's start small :)

…ad key

RossComputerGuy · 2025-02-08T04:33:39Z

I found out while slicing this PR that we actually do have to mock threading.

/home/ross/zig/lib/std/Thread.zig:558:9: error: Unsupported operating system uefi
        @compileError("Unsupported operating system " ++ @tagName(native_os));
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    getCurrentId: /home/ross/zig/lib/std/Thread.zig:533:27
    getCurrentId: /home/ross/zig/lib/std/Thread.zig:354:29
    lock: /home/ross/zig/lib/std/Thread/Mutex/Recursive.zig:50:54
    lockStdErr: /home/ross/zig/lib/std/Progress.zig:545:22
    lockStdErr: /home/ross/zig/lib/std/debug.zig:198:28
    print__anon_1981: /home/ross/zig/lib/std/debug.zig:208:15
    main: src/main.zig:7:20
    EfiMain: /home/ross/zig/lib/std/start.zig:223:37
    comptime: /home/ross/zig/lib/std/start.zig:75:58
    start: /home/ross/zig/lib/std/std.zig:97:27
    comptime: /home/ross/zig/lib/std/std.zig:168:9

RossComputerGuy · 2025-02-08T04:53:07Z

Ok, things are split up now. I went through and dropped changes which were not necessary to make this work:

$ zig init
$ zig build -Dtarget=$(uname -m)-uefi

alexrp reviewed Dec 13, 2024

View reviewed changes

RossComputerGuy force-pushed the feat/uefi-zig-init branch from 87beafb to 2a2c072 Compare December 13, 2024 23:02

linusg reviewed Dec 19, 2024

View reviewed changes

RossComputerGuy force-pushed the feat/uefi-zig-init branch 2 times, most recently from cb18f91 to 74b7013 Compare December 19, 2024 19:09

RossComputerGuy force-pushed the feat/uefi-zig-init branch 2 times, most recently from 372c006 to ce7569c Compare February 7, 2025 22:57

This was referenced Feb 7, 2025

Several allocators in std no longer compile #22806

Open

UEFI keeps regressing due to lack of CI build coverage #22809

Open

std.os.uefi: fix allocators compiling #22810

Open

RossComputerGuy force-pushed the feat/uefi-zig-init branch from ce7569c to ce1a302 Compare February 8, 2025 01:32

linusg requested changes Feb 8, 2025

View reviewed changes

std.start: handle errors with efi main

2adedd1

RossComputerGuy added 6 commits February 7, 2025 18:46

std.os.uefi.tables.boot_services: wrap memory functions

81019b1

std.os.uefi.tables: make allocate type a tagged union

eb9f336

std.os.uefi.protocol.file: wrap open

205a9a4

std.io: add stdio for uefi

02ef16a

std.os.uefi.protocols.simple_text_input: improve error handling in re…

b032683

…ad key

std.os.uefi: refactor allocators

829d756

RossComputerGuy force-pushed the feat/uefi-zig-init branch from ce1a302 to 31ebae5 Compare February 8, 2025 04:38

RossComputerGuy added 5 commits February 7, 2025 20:47

std.os.uefi: add posix compatibility layer

8815213

std.Thread: fake process id on uefi

de6463c

std.fs.File: disable ansi on uefi

23d4f75

std.debug: add module lookup function prototype for uefi

8fce590

std.fs.Dir: no flock when not defined

3d3cfa3

RossComputerGuy force-pushed the feat/uefi-zig-init branch from 31ebae5 to 3d3cfa3 Compare February 8, 2025 04:47

uefi: mark as always single threaded

9220214

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

std: implement basic io and improve alloc in uefi #22226

std: implement basic io and improve alloc in uefi #22226

RossComputerGuy commented Dec 13, 2024

alexrp Dec 13, 2024

RossComputerGuy Dec 13, 2024

alexrp Dec 13, 2024

RossComputerGuy Dec 13, 2024

RossComputerGuy Dec 13, 2024

mochalins Dec 14, 2024

RossComputerGuy Dec 14, 2024

RossComputerGuy Dec 29, 2024

linusg Dec 19, 2024 •

edited

Loading

RossComputerGuy Dec 19, 2024

linusg Dec 19, 2024

RossComputerGuy Dec 19, 2024

linusg Dec 19, 2024

RossComputerGuy Dec 19, 2024

linusg Dec 19, 2024

RossComputerGuy Dec 19, 2024

truemedian Dec 19, 2024

RossComputerGuy Dec 29, 2024

linusg left a comment

RossComputerGuy commented Feb 8, 2025 •

edited

Loading

linusg commented Feb 8, 2025

RossComputerGuy commented Feb 8, 2025

RossComputerGuy commented Feb 8, 2025

		@@ -1,5 +1,7 @@
		const std = @import("../std.zig");

		pub const posix = @import("uefi/posix.zig");

	pub fn alwaysSingleThreaded(target: std.Target) bool {
	_ = target;
	return false;
	}

	pub fn defaultSingleThreaded(target: std.Target) bool {
	switch (target.cpu.arch) {
	.wasm32, .wasm64 => return true,
	else => {},
	}
	switch (target.os.tag) {
	.haiku => return true,
	else => {},
	}
	return false;
	}

std: implement basic io and improve alloc in uefi #22226

Are you sure you want to change the base?

std: implement basic io and improve alloc in uefi #22226

Conversation

RossComputerGuy commented Dec 13, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linusg Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linusg left a comment

Choose a reason for hiding this comment

RossComputerGuy commented Feb 8, 2025 • edited Loading

linusg commented Feb 8, 2025

RossComputerGuy commented Feb 8, 2025

RossComputerGuy commented Feb 8, 2025

linusg Dec 19, 2024 •

edited

Loading

RossComputerGuy commented Feb 8, 2025 •

edited

Loading