This seems to be quite similar to the other unikernel that was posted recently [1]. It would be useful to have an in-depth comparison of these (and other) unikernels, especially with regard to performance and compatibility.
I can see that HermiTux has the ability to select only the syscalls needed by the embedded program, and to transform syscalls to function calls, which seems quite interesting.
That's certainly part of the point (and probably one of the main performance benefits). However, even if this doesn't happen, a unikernel still gives the advantage that it's much more difficult to run undesired code and is likely to improve security.
> OSv was originally designed and implemented by Cloudius Systems (now ScyllaDB) however currently it is being maintained and enhanced by a small community of volunteers.
Yeah that doesn't sound conclusive in either direction -- not clear if they made this on the way to ScyllaDB. I took another look just in case and found a source[0]:
> After much research into the market and technology, in mid-2014 the team decided to pivot away from OSv (still a live project). Instead they decided to embrace the Cassandra architecture but rewrite it from scratch in native code with all of the know-how of years of kernel/hypervisor coding. This would become the Scylla database.
I was watching a Jonathan Blow clip recently where he was bemoaning all the layers of abstraction we've put over executables, which are all x86 (or ARM etc) at the end of the day.
It got me wondering if it would be feasible to have a single executable format that works across OSes. My OS knowledge is pretty rusty. What are the main roadblocks? Will we always have to have at least some runtime layer like WebAssembly?
Well, there's two things here. One, typically executable code is packaged in a file along with other content, otherwise, the OS doesn't know that the contents are intended to be executed.
The broader problem is, if you could somehow work around that... what are you going to _do_? You need OS-specific code to read and write files, to display things, to get input from the keyboard/mouse, to talk to the network. It's not like Ye Olden Days where you can just write directly to the display memory and have things show up, the OS controls your access to all of the hardware. An OS-agnostic executable can't accomplish much on a modern PC.
I think the core problem is that actually compiling for different operating systems is essentially none of the work needed for compatability and provides a stage that lets you remove stuff that isn't needed for that OS, while using bytecode makes it possible to support different instruction sets and can be compiled for the platform it is used on as needed or ahead of time.
Going the other way, if an application is distributed as a VM or other method of minimal reliance on the host OS it has all the issues of static binaries (heavier resource use, can't benefit from general updates) plus it won't have the look and feel of different operating systems unless it includes all that complexity on every OS. There are a number of other compromises that could be made vs. the most commonly used ones, but it is all a matter of tradeoffs. E.g. Flatpack has a set of base images as well as the application image, but it seems to have issues with the availability of dependencies.
The closer operating systems are the easier it is for a single executable to work. The BSDs have have been able to run binaries from various other operating systems, however as the other operating systems change it a lot of work to keep up and some OS APIs are just bad ideas that would compromise the operating system if they are adopted (in some cases a better way to allow compatibility could be created with a lot more work).
One option that I am interested in that might gain some popularity at some point would be to run a much simpler OS in a VM that can do some of the useful things you want a computer to do while not trying to do all of them but without trying to make individual applications fit in with the host OS. This would require users to learn two operating systems but would make the small OS easier to understand in detail and would have some significant security advantages due to being able to impose restrictions that would excessively restrict a general purpose OS. However, there will always be pressure to integrate more into the host OS. A major issue limiting this possibility right now is the situtation with GPU access from a VM. Also, commercial OS vendors are more than willing to make gratuitous incompatabilities to limit any alternative that starts to grow in popularity.
There are a few roadblocks that probably will not be overcome, at least not without introducing more of those abstraction layers.
The binary format itself is a roadblock, because the loader of your OS needs to know that in order to load and start your program. To be compatible e.g. for PE and ELF, you would either need another loader or binaries that look like both formats at the same time. The latter isn't really possible (except for some interesting puzzles) the former is what binfmt support in Linux does. On other OSes, you do that manually like in "java foobar.class" or "wine something.exe".
The syscall convention is another problem, every OS does them somewhat differently, and even the same OS on the same architecture can have different conventions one may use: int 0x80 (or whatever number one likes), call gates, SYSENTER, traps or trampolines. I certainly forgot a few. Within those there is another variability around parameter passing and return values (register, stack, magic location, all of the above), buffer handling for larger parameters and preserved registers. Syscall names, numbers and semantics are all totally different. Then there are mechanisms like unix signals and callbacks from the OS, futures and zero-copy-IO that are very very OS specific. There also is absolutely no smallest common denominator.
Memory layout has peculiarities, like randomisation, different sections, who sets up the stack, that kind of stuff. Mostly the linker and loader will do things here, but again, it depends.
And then there are libraries. Even if you use a static binary (which you probably would have to), modern OSes still expect you to dynamically link some libraries like all syscall entry points or stubs, things like libnss, etc.
All in all quite impossible without at least another layer like wine or the java vm, some big loader switch/case and quite a bit of fiddling to adapt to some hard-to-adapt peculiarities. E.g. the Linux kernel has quite a few things built specially for wine to work.
CloudABI (https://github.com/NuxiNL/cloudabi) was an interesting attempt at doing just that. However it seems it didn't gain enough traction and people involved veered towards wasm/wasi which also abstracts over the instruction set (at the expense of performance, unless I'm missing some development on the AOT front)
Everything that needs more than a single process won't run, either because it needs a system service or because it does multi-process-sandboxing. Single-address-space is extremely limiting for a lot of stuff, so most webservers are out. Not sure if threads would even work. Any desktop software is out anyways. Basically you will have to write your own software for this model of operation, or pick stuff that is accidentially suited, which won't be a lot.
I can see that HermiTux has the ability to select only the syscalls needed by the embedded program, and to transform syscalls to function calls, which seems quite interesting.
[1]: https://news.ycombinator.com/item?id=25405672