blog.6admin.io
guest@6admin.io:~$
abi, assembly, kernel, low-level, syscall, windows

Portable assembly does not exist

You sometimes hear that a single assembly file would recompile and run anywhere, as long as you stay on a common x86-64 architecture. It is a pleasant idea and it collides with reality the moment you look at what actually surrounds the machine instruction. The instruction set, admittedly, is shared. Everything else, the tooling that assembles, the format that wraps, the convention that lets functions talk to one another and the door through which you enter the kernel, all of that diverges from one system to the next.

I will not pretend to be exhaustive, you could write a book on this, but here are the concrete points that doom the myth.

Assemblers do not all speak the same language

Before you produce anything, you have to pick an assembler and the syntax already diverges. On Linux, BSD and macOS you get NASM and GAS, the latter accepting both AT&T and Intel syntax. On Linux you can add FASM and YASM. On Windows you find MASM, NASM and GAS.

That table alone should cool the enthusiasm: the same algorithm is written differently depending on the tool, the directives do not overlap and linking is driven differently from one toolchain to another.

Three executable formats, three worlds

Once the code is assembled, it has to be wrapped and each OS family imposes its own. ELF reigns over Linux and the BSDs, Mach-O dresses macOS and Windows relies on PE together with its object cousin COFF.

These formats do more than carry different headers. The address resolution mechanisms differ too, so an external symbol is not resolved the same way whether you sit inside an ELF, a Mach-O or a PE. Stack and runtime handling, very close between Linux and BSD, stands clearly apart on the other two.

ABIs: who puts what in which register

Suppose you have cleared those hurdles. There remains the calling convention, that is, the tacit agreement on how to pass arguments, where to leave the return value and which registers to preserve.

Linux, the BSDs and macOS follow the SysV ABI, even if macOS bends it Apple-style. Windows, for its part, imposes the Microsoft x64 ABI, which does not place arguments in the same registers and does not manage the reserved stack space the same way.

In other words, a routine written for one will call a function compiled for the other all wrong and the error will only show up at runtime, usually in the worst possible fashion.

System calls: the real border

It is when talking to the kernel that the dream of portability collapses for good.

Between Linux and BSD the system calls look alike in spirit, but they do not share the same numbers, so the same syscall does not trigger the same service from one system to the next. macOS, because of its hybrid XNU architecture, a blend of Mach heritage and BSD, has its own brew and you are not supposed to invoke its syscalls directly: the recommended path goes through libSystem. Windows likewise has its own calls, the NtCreateFile, NtAllocateVirtualMemory and the like, which you do not call by hand either: the official route goes from the Win32 API to ntdll.dll, which alone crosses the boundary into the kernel.

In both cases, aiming at the low layer directly amounts to writing fragile code, dependent on a numbering order the vendor never promised to freeze.

An aside for the old-timers

For anyone who lived through DOS, none of this comes as a surprise. Back then we had a lot of fun with MASM, 16-bit real mode staying limited but clearly more comfortable than an 8-bit Z80 in a CPC.

Interrupts went through DOS and the BIOS, registers and segments fit in 16 bits, AX where today you would write EAX in 32-bit then RAX in 64-bit. No paging, no multitasking, no memory protection whatsoever. And for the more curious, a 486DX2 already let you switch into 32-bit protected mode, just to taste something else.

The scenery has changed, the lesson stands: portability at the machine level has always been a matter of layers, never a given.

So much for portable

  • The x86-64 instruction set is shared, but everything around it diverges from one OS to another.
  • Assembler, executable format, address resolution, ABI and system calls form that many independent borders.
  • SysV on one side, Microsoft x64 on the other: calling conventions are not interchangeable.
  • You do not hit syscalls directly on Windows or macOS, you go through ntdll.dll or libSystem.
  • “Portable assembly” is an oxymoron until you commit to an abstraction layer, which itself carries a cost.

To carry the descent further, see Linux syscalls for the kernel interface mentioned here, then sockets in C and raw sockets for what the libc absorbs when you stay one level above.