a nerd's tale

ho ho ho! elves at work

Potentially controversial opinion: putting a price on a piece of software is a questionable thing to do, though putting it behind a subscription is definitely a criminal offense. How do you put a price on a thing whose manufacturing cost drops to practically nothing after a point? It definitely makes sense if it comes with hardware, since that has a fixed cost.

Regardless, it is questionable but not wrong, though I do expect to have the source code too, if I am paying for it. It doesn’t necessarily need to be open source, but I should at least have access to it. Why am I rambling about this? Cause if I have the source code, I have a better chance of figuring out why something doesn’t work as expected, instead of having to go dissecting the compiled binary.

</preface>

A couple of years back, I was pulling my hair out trying to figure out why my packaging script wouldn’t work. I have a compiled binary, I copy that binary, figure out which libraries it links to, copy them too, rewrite the rpaths to point to the right places, and that should be enough, right? right? Nope, it almost worked, minus one single DLL.

I only figured out the why behind this one after I almost gave myself an amateur haircut without using any scissors or clippers.

It started with, what am I doing wrong? Is there something wrong with the CMake configuration? Maybe something was off with setting up the deps with Conan? Took a day or 2 to systematically iron out the known possibilities on my side, but then I started having doubts about those pylon binaries from Basler.

Never in my life did I think I would have to read the ELF spec to fix someone else’s mistakes. Well, maybe not never, everyone has a past, but definitely never to fix a bug.

The Wikipedia entry is a good start but definitely doesn’t give you enough to work with. The next stop is the specification itself, but goodness, that is a dry piece of text, just like the other spec documents. Compared to that, the OpenType spec was pretty decent, even though it came from Mr. Immaruineverythingnice, Microsoft.

Shit gets interesting if we look into patchelf’s source code. There, it becomes reasonably clear how the headers are structured and what information is available. readelf was pretty much what I needed to piece things together.

At this point, I was fairly convinced something was wrong with the metadata rather than the actual code. Comparing the dynamic sections of the DLLs that worked with the one that didn’t, I found that this particular DLL was missing the DT_SONAME entry.

I guess there’s nothing technically wrong with that, cause the entry is optional, but come on, if you distribute a library without source, you could at least give it a proper runtime identity. The SONAME is commonly used to identify a library’s ABI compatibility version, which is rather useful when other binaries are supposed to link against it.

The hand-holding of modern tooling makes me feel glad and annoyed at the same time. Glad, cause I don’t have to deal with, or even learn about, what goes on behind the scenes. Annoyed, cause the moment one non-standard thing comes along, I not only need a reasonably good grasp of the internals, but also have to deal with the added baggage of understanding all the abstractions the tooling has piled on top.

And that brings us to the actual problem.

linuxdeploy runs ldd on the binary and parses stdout to figure out its dynamic dependencies. The parser expects the familiar ldd output, something along the lines of,

text
libfoo.so => /some/path/libfoo.so (0x...)

Except, ldd doesn’t always print things that way. It can also produce,

text
/some/path/libfoo.so (0x...)

without the => in the middle.

And given those lines in glibc, that difference isn’t exactly arbitrary. Depending on how the dependency is recorded and resolved, the loader can end up printing the latter form.

Now, when a shared library has a DT_SONAME, the linker generally records that name in the consuming binary’s DT_NEEDED entry. Without it, the linker can instead record the name or path used while linking. And if that happens to be an absolute path, congratulations, you have now got that path stuffed into DT_NEEDED.

The dependency is still recorded as DT_NEEDED, just not necessarily with the nice library name you’d expect. And if the recorded name matches the path of the loaded object, glibc can print it without the =>. Which is perfectly fine as far as the loader is concerned, but not so fine for linuxdeploy, whose parser expects that => to be there.

So the library loads, ldd reports it, but linuxdeploy fails to pick it up. One missing DT_SONAME, a path-based dependency, and a parser making assumptions about the output format. Nothing necessarily invalid about the ELF itself, just a bunch of individually reasonable decisions coming together to ruin my day.

Luckily, with that figured out, messing with the metadata isn’t that hard with patchelf in hand. A bit of patching to give the library a proper SONAME, and making sure the consuming binary records the right dependency, and things can finally behave themselves.

After all this shebang, I think I can now brag about the fact that I might be able to hand-roll ELFs. Would be a cool party trick, right? I can go, “look at me, I am the compiler now”.

Of course, an ELF needs something to execute, and if I am going to pretend to be a compiler, I might as well translate the instructions myself.

Though that kind of power could only be possessed by people with rainbow socks. Translating aarch64 assembly instructions by hand is no joke. A simple exit 0 in assembly would be,

asm
mov x8, #93
mov x0, #0
svc 0

and with the help of this reference, that’s roughly,

assembly instruction grouped by bytes in decimal in hex little endian
mov x8, #93 1 10100101 00 0000000001011101 01000 11010010 10000000 00001011 10101000 210 128 011 168 0xd2 0x80 0x0b 0xa8 0xa8 0x0b 0x80 0xd2
mov x0, #0 1 10100101 00 0000000000000000 00000 11010010 10000000 00000000 00000000 210 128 000 000 0xd2 0x80 0x00 0x00 0x00 0x00 0x80 0xd2
svc 0 1 10101000 00 0000000000000000 00001 11010100 00000000 00000000 00000001 212 000 000 001 0xd4 0x00 0x00 0x01 0x01 0x00 0x00 0xd4

So now I can stuff 0xa8 0x0b 0x80 0xd2 0x00 0x00 0x80 0xd2 0x01 0x00 0x00 0xd4 into a file, wrap it in a valid ELF with an executable PT_LOAD segment, point e_entry in the header to the virtual address of those instructions, and we are good to go.

Thats a fucking lot of mental calculation, gotta take a nap now :wq.