Linkers

I spent this week learning about Node.js C++ add ons, node-wap, node-gyp and linker behavior. What a week.

The monkey wrench in the whole thing was that this C++ add on depended on a particular static library (".a" file). The resulting file (from "node-wap build" or "node-gyp build") is a dynamic library (".node" file).

As we all know, static libraries are collections of object files. The crazy-important detail that eluded me was that during linking, the linker only copies over the object files (from the ".a" file) that are required in that context (i.e., amongst the other object files being linked).

Fast forward, and I use "node" to execute some JavaScript which imports (via dlopen()) the .node dynamic library and craps out immediately due to missing symbols that are referenced from within the .node. Those symbols originated in the static library. Of course, I didn't know any of this at first because node.js HIDES the dlopen() error message. It doesn't actually even bother to get it. I just returns a generic error and tosses an exception. I fired up lldb to get some debugging action going on. Of course, the node binaries don't have any debugging symbols. So I downloaded the source, compiled and installed it. It was only then that lldb could show me the dlerror() message (which contained the missing symbol name).

I verified: the symbol in question was definitely NOT in the .node, and it definitely WAS in the .a. But some of the symbols from the .a were in the .node. Hrmm.

After spending some quality time with Google I learned of the "--whole-archive" linker option which prevents the default linker behavior described above. Alas, that didn't work on OS X. Different linker. But I did find "-all_load" and "-force_load" which each accomplish the same thing. Hurray! Run-time loading of dynamic libraries! And the JS could even call my C++. Fancy stuff.

Then I fussed with node-gyp for a while, trying to get it to pass the "-force_load" argument to the linker. node-gyp has a comfort zone, and it doesn't like to leave it. There's a very obvious "ldflags" option that can be used in a .gyp file, but it was _completely_ ignored. I can't count the number of variations I managed to think of, trying to get that to work. Then I discovered "xcode_settings". But I'm not producing an Xcode project. Why would I want Xcode settings? It's not like I had any better ideas though. And, sure enough, on the Mac, that's all node-gyp seems to care about.

Then I fussed with node-gyp for a while, trying to get it to create a fat binary (i386, x86_64), but I finally gave up. And since that's not actually required, it's no loss. But I feel defeated.

Here's a binding.gyp file representing hours of suffering:

{
  'targets': [

    {
      'target_name': 'ModuleName',
      'sources': ['ModuleName.cc'],
      'conditions': [
        ['OS == "mac"', {
          'xcode_settings': {
            'ARCHS': ['x86_64'],
            'OTHER_LDFLAGS': ['-Wl,-force_load,../StaticLibrary.a'],
          }
        }],
      ],
    },
  ],
}