I've had a student working on creating a Scheme interface for Berkeley DbXml for the last few weeks. He left for an internship at Microsoft last week and so I thought I'd pick up where he left off, normalize the naming, and so on. One of the things I wanted to do was to port his work to OS X since it's my development platform of choice. Consequently, yesterday, I downloaded the latest copy of DbXml to my PowerBook and stated the build.

We've been using DbXml in my lab for a while and my students have been frustrated with build problems, particularly on the Perl wrappers. We could get it to build (sometimes) on Redhat, but not Fedora, for example. I ran into the same problems on OS X, but not to worry, I didn't need the Perl wrappers, so when they failed their test, I just moved on. Alert readers will discern that this will play into the story later.

I grabbed Mike's files and started playing. Mike's code creates a set of wrapper functions for the DbXml C++ interface and then links those, together with all the libraries for DbXml, into a single library that the Scheme foreign function interface can call. I played with the naming and redid the build so that it used the MzScheme mzc program since that helps normalize some of the build for specific platforms. I ran into some problems, but no more than I usually expect when porting something.

An couple hours into it, however, I was sure I'd done everything I could and still, when I'd fire it up, it complained that it couldn't find functions that should have been linked into the library. I relinked, read the documentation for gcc, and pulled my hair out for a good long while. I solved a couple small problems, but nothing I did put a dent in the primary problem of not being able to find some functions in the library I'd created.

Finally, primarily because I was googling anything I could think of, I ran across this page on creating working dynamic libraries on OS X. It suggested that sometimes dynamic libraries are created that don't contain the full path information of the included libraries and thus they can't be found. On OS X, there is a tool, called otool, that you can use to display the table of contents for a dynamic library (among other things). Sure enough, when I looked at my library, the Xerces library was the only one without full path information. There's another handy tool called install_name_tool that you can use to correct this problem. Once I did, the code ran just like it was meant to.

Here's where we get back to the Perl problems. Perl built fine, but wouldn't run. Tonight I got to thinking that it was likely the same problem. Sure enough, running otool on the Perl bundle (Perl's name for the dynamic library) revealed that Xerces didn't have full path information in the bundle. I fixed the bundle and the Perl wrapper for DbXml ran without a hitch. A simple 60 second job, once you know what to do.

The underlying problem is that the Xerces library doesn't use the GNU libtool script (with the --install_name flag) to install the library. Instead, it uses its own install-sh script and this apparently needs a little work. Consequently, the Xerces library doesn't know where it's at and thus can't give the linker that information when it's linked into another library. So anything that includes the Xerces library (at least on OS X) will exhibit this problem. Nice, huh?