I disagree about the structs vs out-parameters thing. I’ve found it makes functions that could return an error much harder to compose and leads to a proliferation of types all over the place. In practice almost all functions can fail (assuming you are handling OOM), so having a predictable style of returning errors is more important.
Which almost noone ever does. It's very hard and almost never has any benefit. At that point you have way different problems than programming style choices...
You'd get normal errno/out param semantics if you had access to semantic struct unpacking. But you don't. Even then, composing optional values in C has always been a bit of a pain. If you're not doing exceptions, your two choices seem to be exceptions and monads in every language, and neither work in C, or would even be compatible with the philosophy that most C programmers have. I guess you could attempt to pull some kind of macro but it only works on simple one-to-the-other calls. C++ optionals, as terrible as C++ is, are certainly more fun to use than
Returning option<foo> (or sum<foo, error>) is the right thing but a real pain to write in C. I'm not sure the pattern of `if (thing(...)) goto fail` on every function call is particularly wonderful either, though the Go crowd seem to like it.
Otherwise there's thread_local mylibrary_errno, which might actually be the right thing for within a library, translating it to an enum return on the boundaries.