> Suppose you control the entire stack though, from the bare metal up. (Correct me if I'm wrong, but) Toro doesn't seem to run on real hardware, you have to run it atop QEMU or Firecracker.
Some unikernels are intended to run under a hypervisor or on bare metal. Bare metal means you need some drivers, but if you have a use case for a unikernel on bare metal, you probably don't need to support the vast universe of devices, maybe only a few instances of a couple types of things.
I've got a not production ready at all hobby OS that's adjacent to a unikernel; runs in virtio hypervisors and on bare metal, with support for one NIC. In it's intended hypothetical use, it would boot from PXE, with storage on nodes running a traditional OS, so supporting a handful of NICs would probably be sufficient. Modern NICs tend to be fairly similar in interface, so if the manufacturer provides documentation, it shouldn't take too long to add support at least once you've got one driver doing multiple tx/rx queues and all that jazz... plus or minus optimization.
For storage, you can probably get by with two drivers, one for sata/ahci and one for nvme. And likely reuse an existing filesystem.
Do you usually publish your hobby code publicly? If not, consider this an appeal to do so (:
> Modern NICs tend to be fairly similar in interface, so if the manufacturer provides documentation, it shouldn't take too long to add support at least once you've got one driver ... For storage, you can probably get by with two drivers
I take that there aren't any pluggable drivers for NICs like there's for nvme/sata disks?
> I take that there aren't any pluggable drivers for NICs like there's for nvme/sata disks?
I mean, there is NDIS / NDISWrapper. Or, I think it wouldn't be too hard to run netbsd drivers... but I'm crazy and want my drivers in userland, in Erlang, so none of that applies. :)
As a fair warning, there's some concurrency errors in the kernel which I haven't tracked down that results in sometimes getting stuck before the shell prompt comes up, the tcp stack is just ok enough to mostly work, and the dhcp client only works if everything goes right.
Erlang! Indeed a crazy idea (in a good way!), and while I'm not normally a big fan of unikernels, now you've got me seriously intrigued :)
I've been dabbling in Erlang and OS development myself, my biggest inspirations being Microsoft Singularity and QNX. The former is a C# lookalike of what you're making, or at least that's how it seems from my perspective.
The readme mentions a FreeBSD-like system call interface, but then the drivers and the network stack are written in Erlang, and, as you've mentioned, run in the user land. Is that actually a unikernel design with BEAM running in the kernel, or more of a microkernel hosting BEAM, with it providing device handling and the user space?
The original plan was BEAM on metal, but I had a hard time getting that started... so I pivoted to BEAM from pkg, running on a just enough kernel that exposes only the FreeBSD syscalls that actually get called.
Where that fits in the taxonomy of life, I'm not sure. There is a kernel/userspace boundary (and also a c-code/erlang code boundary in userspace), so it's not quite a unikernel. I wouldn't really call it a microkernel; there's none of the usual microkernel stuff... I just let userspace do i/o ports with the x86 task structure and do memory mapped i/o by letting it mmap anything (more or less). The kernel manages timers/timekeeping and interrupts, Erlang drivers open a socket to get notified when an interrupt fires --- level triggered interrupts would be an issue. Kernel also does thread spawning and mutex support, connects pipes, early/late console output, etc.
If I get through my roadmap (networked demo, uefi/amd64 support, maybe arm64 support, run the otp test suite), I might look again and see if I can eliminate the kernel/userspace divide now that I understand the underneath, but the minimal kernel approach lets me play around with the fun parts, so I'm pretty happy with it. I've got a slightly tweaked dist working and can hotload userspace Erlang code over the network, including the tcp stack, which was the itch I wanted to scratch... nevermind that the tcp stack isn't very good at the moment ;)
Really cool! Will definitely take a closer look in my spare time.
>I just let userspace do i/o ports [...] and do memory mapped i/o by letting it mmap anything (more or less). The kernel manages timers/timekeeping and interrupts [...]
This is how QNX does it too, allowing privileged processes to use MAP_PHYS and port I/O instructions on x86, and handle interrupts like they're POSIX signals. It all boils down to how you structure your design, but personally, I think that's not a bad approach at all. The cool thing about it is that, after the initial setup, you can drop the privileges for creating further mappings and handlers, reducing the attack surface.
Unless you're trying to absolutely minimize the cost and amount of context switches, I think moving BEAM into the kernel would be a downgrade, but again, I'm a big proponent of microkernels :)
Some unikernels are intended to run under a hypervisor or on bare metal. Bare metal means you need some drivers, but if you have a use case for a unikernel on bare metal, you probably don't need to support the vast universe of devices, maybe only a few instances of a couple types of things.
I've got a not production ready at all hobby OS that's adjacent to a unikernel; runs in virtio hypervisors and on bare metal, with support for one NIC. In it's intended hypothetical use, it would boot from PXE, with storage on nodes running a traditional OS, so supporting a handful of NICs would probably be sufficient. Modern NICs tend to be fairly similar in interface, so if the manufacturer provides documentation, it shouldn't take too long to add support at least once you've got one driver doing multiple tx/rx queues and all that jazz... plus or minus optimization.
For storage, you can probably get by with two drivers, one for sata/ahci and one for nvme. And likely reuse an existing filesystem.