OpenMethod works with shared libraries on Linux, other POSIX-like platforms, and Windows. A program and its shared libraries can all contribute classes, methods and overriders to the same registry, provided they share its state.

Sharing a Registry

All of a registry’s mutable state lives in a single variable. One module owns it and exports it; every other module - a client - imports it. Three macros do this. Each takes the registry as an argument, so they work the same for default_registry, indirect_registry and custom registries:

Macro Where

BOOST_OPENMETHOD_IMPORT_REGISTRY

every translation unit of a client module

BOOST_OPENMETHOD_EXPORT_REGISTRY

every translation unit of the owning module

BOOST_OPENMETHOD_INSTANTIATE_REGISTRY

exactly one translation unit of the owning module

The examples below put IMPORT and EXPORT in a header that every translation unit includes, and choose between them with a macro that the owning module defines:

#ifdef OWNS_REGISTRY_STATE
BOOST_OPENMETHOD_EXPORT_REGISTRY(boost::openmethod::default_registry);
#else
BOOST_OPENMETHOD_IMPORT_REGISTRY(boost::openmethod::default_registry);
#endif

The owner then adds INSTANTIATE to one of its .cpp files. Use the macros at namespace scope, after <boost/openmethod.hpp> and the definition of the registry. They emit fully qualified names, so nothing needs to be added to namespace boost::openmethod.

a translation unit that uses neither macro still compiles, with a private copy of the state. In a client module, that module then stops sharing classes, methods and overriders with the rest of the program, which shows as missing_class or unexpected bad_call errors. In the owning module, under -fvisibility=hidden on ELF, it hides the state: the module exports nothing, and clients fail to link with an undefined reference to registry_state<…​>::st. Putting the macros in a header, as above, prevents both.

Do not replace the macros with hand-written explicit instantiations. Different platforms need different decorations, and the wrong one compiles silently on some of them (see Platform Details).

Nothing else needs decorating. Methods, overriders and class registrations are consolidated across modules by initialize (see Implementation Notes). A library that uses open-methods only internally, and not in its API, does not need to share the state at all.

Implicit Linking

When a program links against a shared library at build time, every module is loaded, and its static constructors have run, before main starts. One call to initialize then sets up the dispatch tables.

The library owns the state. The program links against the library, and on Windows and macOS, which resolve imported symbols at link time, the library cannot also import the state from the program: each would have to link against the other. In the implicit_linking example, the header shared by both modules defines the classes and the meet method, and selects EXPORT or IMPORT:

// animals.hpp

#include <string>
#include <boost/openmethod.hpp>

#ifdef OWNS_REGISTRY_STATE
BOOST_OPENMETHOD_EXPORT_REGISTRY(boost::openmethod::default_registry);
#else
BOOST_OPENMETHOD_IMPORT_REGISTRY(boost::openmethod::default_registry);
#endif

struct Animal { virtual ~Animal() {} };
struct Herbivore : Animal {};
struct Carnivore : Animal {};

struct Cow : Herbivore {};
struct Wolf : Carnivore {};

BOOST_OPENMETHOD(
    meet, (
        boost::openmethod::virtual_ptr<Animal>,
        boost::openmethod::virtual_ptr<Animal>),
    std::string);

The library defines OWNS_REGISTRY_STATE and instantiates the state. It registers the classes, and provides the default overrider:

// extensions.cpp

#define OWNS_REGISTRY_STATE

#include "animals.hpp"

#include <string>

using namespace boost::openmethod;

BOOST_OPENMETHOD_INSTANTIATE_REGISTRY(boost::openmethod::default_registry);

BOOST_OPENMETHOD_CLASSES(Animal, Herbivore, Cow, Carnivore, Wolf);

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Animal>, virtual_ptr<Animal>), std::string) {
    return "greet";
}

The program imports the state, and adds a class and two overriders:

// main.cpp

#include "animals.hpp"
#include <boost/openmethod/initialize.hpp>
#include <iostream>
#include <memory>
#include <string>

using namespace boost::openmethod::aliases;

struct Tiger : Carnivore {};

BOOST_OPENMETHOD_CLASSES(Tiger, Carnivore);
BOOST_OPENMETHOD_CLASSES(Animal, Herbivore, Cow, Carnivore, Wolf);

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Herbivore> a, virtual_ptr<Carnivore> b), std::string) {
    auto base = next(a, b);
    return "do not " + base + ", run";
}

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Carnivore>, virtual_ptr<Herbivore>), std::string) {
    return "hunt";
}

auto main() -> int {
    boost::openmethod::initialize();

    std::unique_ptr<Animal> gracie(new Cow());
    std::unique_ptr<Animal> willy(new Wolf());
    std::unique_ptr<Animal> hobbes(new Tiger());

    std::cout << "cow meets wolf -> " << meet(*gracie, *willy)
              << "\n"; // do not greet, run
    std::cout << "wolf meets cow -> " << meet(*willy, *gracie) << "\n"; // hunt
    std::cout << "cow meets tiger -> " << meet(*gracie, *hobbes)
              << "\n"; // do not greet, run

    return 0;
}

next crosses the module boundary in the middle of a dispatch: the program’s Herbivore, Carnivore overrider calls the library’s Animal, Animal overrider, so cow meets wolf prints do not greet, run.

The program registers the library’s classes again, because a class must be registered in every module that has its own type id for it (see use_classes). Where the dynamic linker merges a class’s type_info objects - on ELF, for a class with default visibility - the second registration is redundant but harmless. Elsewhere, and on Windows in particular, each module has its own type_info; without the second registration, the program’s Wolf, created with new, is unknown to the registry, and dispatching on it fails with unknown class Wolf.

Dynamic Linking

A program can also load a shared library after it has started - a plugin, typically - and the library can add classes, methods and overriders to the registry. Call initialize again after loading or unloading a library.

The program is then the natural owner of the state: it must work before the library is loaded, and whether or not it is. The dynamic_loading example uses a header like the one above. The program defines OWNS_REGISTRY_STATE, instantiates the state, provides a catch-all overrider, and calls meet:

// main.cpp

#define OWNS_REGISTRY_STATE

#include "animals.hpp"

#include <boost/openmethod.hpp>
#include <boost/openmethod/initialize.hpp>
#include <boost/dll/shared_library.hpp>
#include <iostream>
#include <memory>
#include <string>

using namespace boost::openmethod;

BOOST_OPENMETHOD_INSTANTIATE_REGISTRY(boost::openmethod::default_registry);

BOOST_OPENMETHOD_CLASSES(Herbivore, Cow, Carnivore, Wolf);

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Animal>, virtual_ptr<Animal>), std::string) {
    return "greet";
}

int main() {

    try {
        std::cout << "Before loading the shared library.\n";

        boost::openmethod::initialize(trace::from_env());

        std::cout << "cow meets wolf -> "
                  << meet(*std::make_unique<Cow>(), *std::make_unique<Wolf>())
                  << "\n"; // greet
        std::cout << "wolf meets cow -> "
                  << meet(*std::make_unique<Wolf>(), *std::make_unique<Cow>())
                  << "\n"; // greet

        // to be continued...

The library imports the state. It adds two overriders, a class, Tiger, and a factory function:

// extensions.cpp

#include "animals.hpp"

#include <string>

using namespace boost::openmethod;

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Herbivore> a, virtual_ptr<Carnivore> b), std::string) {
    auto base = next(a, b);
    return "do not " + base + ", run";
}

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Carnivore>, virtual_ptr<Herbivore>), std::string) {
    return "hunt";
}

struct Tiger : Carnivore {};

BOOST_OPENMETHOD_CLASSES(Tiger, Carnivore);

extern "C" {
BOOST_SYMBOL_EXPORT auto make_tiger() -> Animal* {
    return new Tiger;
}
}

Once the library is loaded with Boost.DLL and initialize has run, its overriders are in effect, and the program can use Tiger objects without knowing the class:

int main() {
        // ...

        std::cout << "\nLoading shared object / DLL.\n";

        boost::dll::shared_library lib(
            "boost_openmethod-shared",
            boost::dll::load_mode::rtld_global |
                boost::dll::load_mode::append_decorations);

        boost::openmethod::initialize(trace::from_env());

        std::cout << "cow meets wolf -> "
                  << meet(*std::make_unique<Cow>(), *std::make_unique<Wolf>())
                  << "\n"; // do not greet, run
        std::cout << "wolf meets cow -> "
                  << meet(*std::make_unique<Wolf>(), *std::make_unique<Cow>())
                  << "\n"; // hunt

        auto make_tiger = lib.get<Animal*()>("make_tiger");
        std::cout << "cow meets tiger -> "
                  << meet(
                         *std::make_unique<Cow>(),
                         *std::unique_ptr<Animal>(make_tiger()))
                  << "\n"; // do not greet, run

Once it is unloaded and initialize has run again, they are gone:

int main() {
        // ...

        std::cout << "\nAfter unloading the shared library.\n";

        lib.unload();
        boost::openmethod::initialize(trace::from_env());

        std::cout << "cow meets wolf -> "
                  << meet(*std::make_unique<Cow>(), *std::make_unique<Wolf>())
                  << "\n"; // greet
        std::cout << "wolf meets cow -> "
                  << meet(*std::make_unique<Wolf>(), *std::make_unique<Cow>())
                  << "\n"; // greet
with GCC’s -rdynamic, dlclose does not necessarily unload the library. It may stay loaded until the program exits, and the last meet call then prints the same as the previous one.
a library that installs an error handler (see error_handling.adoc) with the registry’s error_handler::set must restore the previous one before it is unloaded. The handler is part of the shared state, and would otherwise point into unloaded code.

CMake Setup

The library imports a symbol that the executable defines, so it links against the executable - the reverse of the usual direction:

add_executable(my_app main.cpp)
set_target_properties(my_app PROPERTIES ENABLE_EXPORTS ON)
target_link_libraries(my_app Boost::openmethod Boost::dll)

add_library(my_plugin SHARED extensions.cpp)
target_link_libraries(my_plugin PRIVATE Boost::openmethod my_app)

ENABLE_EXPORTS ON makes CMake generate an import library for the executable on Windows, and pass -rdynamic (or its equivalent) on POSIX, so that the executable’s symbols are visible to the libraries it loads.

Type Ids Across Modules

Sharing the state is one question. Where the type ids come from is another: it decides how well the registry’s type_hash policy does its job, and it is where dlopen differs from implicit linking.

Under std_rtti, a type id is &typeid(X). On ELF, for an implicitly linked library, the dynamic linker gives a class a single type_info across modules, by copying the record into the executable and redirecting the library’s references to the copy. A program and its libraries thus present one compact set of type ids. The classes of a dlopened plugin get no such treatment: their records stay in the plugin’s mapping, which the loader places anywhere - with address-space randomization, somewhere different on every run, and often terabytes away from the program’s.

under -fvisibility=hidden, a class can have a different type_info in each module. initialize treats them as several ids for the same class, but each one is one more id for the hash to separate. On ELF, declaring the class BOOST_SYMBOL_VISIBLE gives it a single type_info again.
under static_rtti, a type id is the address of a static variable in a template of the library. Under -fvisibility=hidden, and always with Windows DLLs, each module has its own: initialize takes the copies of a class for unrelated classes, and a call made in one module on an object created in another lands in the wrong overrider. Use named_static_rtti, which maps the ids to the names of the classes so that initialize merges them.

What it costs

fast_perfect_hash, the default, looks for a multiplier M and a shift S such that (M * x) >> S is collision-free over the type ids. It does well when the ids are evenly spread, and badly on a program with a few plugins: several tight clusters, far apart. The search becomes much more expensive, and on a large enough set it gives up after half a million attempts, calling the error handler with a search_error - which terminates the program by default. And since vptr_vector sizes its table from the hash’s range, a struggling hash costs memory as well as time.

A plugin host that registers more than a few hundred classes is the program most likely to meet both problems. A plugin that registers no classes of its own, and only adds overriders for the program’s, contributes no type ids, and does not cause them.

The alternatives

Three other type_hash policies avoid the problem, and vptr_map removes the hash altogether:

policy what it does

minimal_perfect_hash

One slot per type id, whatever the addresses, and a search whose cost depends only on the number of classes. Costs a second dependent load on every dispatch - a nanosecond or two per call. The one to reach for in a plugin host.

two_level_hash

The same idea, with the final reduction replaced by a shift. Cheaper per call than minimal_perfect_hash where the compiler hoists the shift out of the dispatch loop, but the table rounds up to a power of two: between one and two slots per type id.

minimal_cover_hash

Indexes by the smallest set of bit positions that still separates the type ids. As fast per call as the default, and finds its table deterministically, in milliseconds. Needs BMI2 in every translation unit of the program - see its documentation before choosing it.

vptr_map

Not a hash: a map keyed on the type id, so there is no table to size and no search to fail. Slower per dispatch than all of the above, and the only option that asks nothing of the type ids.

Switching takes one declaration, since with replaces a policy with the one of the same category, in place. The registry is then a custom registry, shared as described in Custom Registries:

struct plugin_registry :
    boost::openmethod::default_registry::with<
        boost::openmethod::policies::minimal_perfect_hash<>> {};

vptr_map replaces the vptr policy rather than the hash, which is then dead weight, so drop it:

struct plugin_registry :
    boost::openmethod::default_registry::with<
        boost::openmethod::policies::vptr_map<>>::without<
        boost::openmethod::policies::type_hash> {};

Indirect Vptrs

initialize rebuilds the registry’s v-tables, which invalidates its virtual_ptrs, and the v-table pointers that inplace_vptr_base stores in objects. Most programs load their shared libraries at the very beginning, before any of those exist, and need not care.

The others need a registry with the indirect_vptr policy, such as indirect_registry: default_registry plus indirect_vptr. To make it the registry that the BOOST_OPENMETHOD macros use, define BOOST_OPENMETHOD_DEFAULT_REGISTRY before including <boost/openmethod/core.hpp>, or a header that includes it, like <boost/openmethod.hpp>.

The indirect_vptr example does it in the header that both modules share, so that one place settles both questions - which registry, and how its state is shared - and no translation unit can disagree. indirect_registry has its own state, shared with the same three macros:

// animals.hpp

#include <string>

#define BOOST_OPENMETHOD_DEFAULT_REGISTRY boost::openmethod::indirect_registry
#include <boost/openmethod.hpp>

#ifdef OWNS_REGISTRY_STATE
BOOST_OPENMETHOD_EXPORT_REGISTRY(boost::openmethod::indirect_registry);
#else
BOOST_OPENMETHOD_IMPORT_REGISTRY(boost::openmethod::indirect_registry);
#endif

struct Animal { virtual ~Animal() {} };
struct Herbivore : Animal {};
struct Carnivore : Animal {};

struct Cow : Herbivore {};
struct Wolf : Carnivore {};

BOOST_OPENMETHOD_CLASSES(Animal, Herbivore, Cow, Carnivore, Wolf);

BOOST_OPENMETHOD(
    meet, (
        boost::openmethod::virtual_ptr<Animal>,
        boost::openmethod::virtual_ptr<Animal>),
    std::string);

The program owns the state, and carries virtual_ptrs across initialize calls:

// main.cpp

#define OWNS_REGISTRY_STATE

#include "animals.hpp"

#include <boost/openmethod.hpp>
#include <boost/openmethod/initialize.hpp>
#include <boost/openmethod/interop/std_unique_ptr.hpp>
#include <boost/dll/shared_library.hpp>
#include <iostream>
#include <string>

using namespace boost::openmethod::aliases;

BOOST_OPENMETHOD_INSTANTIATE_REGISTRY(boost::openmethod::indirect_registry);

BOOST_OPENMETHOD_OVERRIDE(
    meet, (virtual_ptr<Animal>, virtual_ptr<Animal>), std::string) {
    return "greet";
}

auto main() -> int {
    std::cout << "Before loading the shared library.\n";
    boost::openmethod::initialize();

    auto gracie = make_unique_virtual<Cow>();
    auto willy = make_unique_virtual<Wolf>();

    std::cout << "cow meets wolf -> " << meet(*gracie, *willy) << "\n"; // greet
    std::cout << "wolf meets cow -> " << meet(*willy, *gracie) << "\n"; // greet

    std::cout << "\nAfter loading the shared library.\n";

    boost::dll::shared_library lib(
        "boost_openmethod-indirect_shared",
        boost::dll::load_mode::rtld_global |
            boost::dll::load_mode::append_decorations);

    boost::openmethod::initialize();

    // The virtual_ptrs made before the reload still dispatch correctly:
    std::cout << "cow meets wolf -> " << meet(*gracie, *willy)
              << "\n"; // do not greet, run
    std::cout << "wolf meets cow -> " << meet(*willy, *gracie) << "\n"; // hunt

    return 0;
}

The library it loads includes the same header, and imports the state. The complete example is in the indirect_vptr directory.

Custom Registries

A custom registry is shared the same way - name it instead of default_registry:

// my_registry.hpp
#ifndef MY_REGISTRY_DEFINED
#define MY_REGISTRY_DEFINED

struct my_registry;
#define BOOST_OPENMETHOD_DEFAULT_REGISTRY my_registry

#include <boost/openmethod.hpp>

struct my_registry : boost::openmethod::registry</* policies... */> {};

#ifdef OWNS_REGISTRY_STATE
BOOST_OPENMETHOD_EXPORT_REGISTRY(my_registry);
#else
BOOST_OPENMETHOD_IMPORT_REGISTRY(my_registry);
#endif

#endif
// registry.cpp - exactly one translation unit of the owning module
#define OWNS_REGISTRY_STATE
#include "my_registry.hpp"

BOOST_OPENMETHOD_INSTANTIATE_REGISTRY(my_registry);

The #define makes my_registry the registry that the BOOST_OPENMETHOD macros use; drop it if the methods name their registry explicitly. The sharing macros come after the definition.

Implementation Notes

One state variable per registry

Everything mutable in a registry is a member of one object: the class and method lists, the dispatch tables, and the state of every stateful policy - for example, the hash parameters of fast_perfect_hash, or the v-table pointers of vptr_vector. Policies keep no globals of their own; template machinery gathers their state into a tuple inside that object. Whatever the policies, there is exactly one symbol to export and import.

The variable is the sole member of registry_state, a deliberately thin, function-free class template, a form dictated by MSVC. dllexport/dllimport are only honored on a whole-class explicit instantiation - not on a variable template, nor on a lone static data member - and exporting a class decorates its member functions and nested types too, which the class holding the state could not tolerate. A one-member class without functions is the one form that every platform can export and import.

Registrations are consolidated, not shared

Only the state variable is shared. The registrars - the static objects generated by BOOST_OPENMETHOD, BOOST_OPENMETHOD_OVERRIDE and BOOST_OPENMETHOD_CLASSES - are not: each module keeps its own method objects, registration records, and per-class v-table pointers.

The copies are consolidated at run time instead. When a module is loaded, its static constructors append its class and method records to the lists in the shared state, each record pointing back into its module. initialize groups the records that describe the same class or method, computes the dispatch tables once, then copies the resulting slots and strides into every module’s copy of each method, and updates every module’s v-table pointers through the pointers captured at registration. Each module dispatches through its own statics, at full speed, and all the copies agree.

This is why methods need no dllexport or dllimport, and why modules need not agree on who "owns" a method: only the registry state has an owner.

Platform Details

On Windows, each module - executable or DLL - has its own copy of a global variable by default. Without the macros, a DLL’s static constructors would register its classes and overriders in the DLL’s own copy of the state, invisible to the program.

On ELF, the dynamic linker merges the modules' copies of an implicitly instantiated variable into one - provided it is visible, which -rdynamic (ENABLE_EXPORTS ON in CMake) ensures for an executable. Under -fvisibility=hidden, each copy stays local to its module, and the modules end up with separate registries, as on Windows.

The two need different decorations, which is why the macros branch on the platform:

  • On Windows, Cygwin and MinGW, __declspec(dllexport) and extern cannot be combined on an explicit instantiation, and the owner’s other translation units may instantiate the state implicitly, since visibility is not a PE concept. So EXPORT expands to nothing, and INSTANTIATE carries the dllexport.

  • On ELF and Mach-O, the attribute must be on the declaration, so that every translation unit of the owner gives the symbol default visibility. GCC rejects it if repeated on the definition, so INSTANTIATE carries none. An implicit instantiation in a translation unit without EXPORT would hide the symbol: ELF merges COMDATs at the most restrictive visibility.

IMPORT is an extern template declaration everywhere - with dllimport on Windows - which keeps a client from instantiating the state and makes it use the owner’s.

The platforms also differ in when the imported symbol is resolved. On ELF, a shared library may be linked with unresolved references, which the dynamic linker satisfies at load time, so a plugin can import the state from the executable that loads it. Mach-O and PE resolve every reference at link time, so a client library must link against the owning module - hence the reverse link in the CMake setup above.

on Windows, every module that shares a registry must use the same dynamic C++ runtime (the /MD or /MDd family). The state contains standard containers, allocated by whichever module runs initialize. A module that links the static runtime (/MT), or another version of the runtime, frees memory it did not allocate when the state is destroyed at exit, which corrupts the heap.
Edit this Page