diff options
Diffstat (limited to 'ext/pybind11/docs/advanced/classes.rst')
| -rw-r--r-- | ext/pybind11/docs/advanced/classes.rst | 484 |
1 files changed, 414 insertions, 70 deletions
diff --git a/ext/pybind11/docs/advanced/classes.rst b/ext/pybind11/docs/advanced/classes.rst index 8896441b6..93deeec62 100644 --- a/ext/pybind11/docs/advanced/classes.rst +++ b/ext/pybind11/docs/advanced/classes.rst @@ -45,9 +45,7 @@ Normally, the binding code for these classes would look as follows: .. code-block:: cpp - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::class_<Animal> animal(m, "Animal"); animal .def("go", &Animal::go); @@ -56,8 +54,6 @@ Normally, the binding code for these classes would look as follows: .def(py::init<>()); m.def("call_go", &call_go); - - return m.ptr(); } However, these bindings are impossible to extend: ``Animal`` is not @@ -90,18 +86,16 @@ functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have a default implementation. There are also two alternate macros :func:`PYBIND11_OVERLOAD_PURE_NAME` and :func:`PYBIND11_OVERLOAD_NAME` which take a string-valued name argument between the *Parent class* and *Name of the -function* slots, which defines the name of function in Python. This is required +function* slots, which defines the name of function in Python. This is required when the C++ and Python versions of the function have different names, e.g. ``operator()`` vs ``__call__``. The binding code also needs a few minor adaptations (highlighted): .. code-block:: cpp - :emphasize-lines: 4,6,7 - - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); + :emphasize-lines: 2,4,5 + PYBIND11_MODULE(example, m) { py::class_<Animal, PyAnimal /* <--- trampoline*/> animal(m, "Animal"); animal .def(py::init<>()) @@ -111,8 +105,6 @@ The binding code also needs a few minor adaptations (highlighted): .def(py::init<>()); m.def("call_go", &call_go); - - return m.ptr(); } Importantly, pybind11 is made aware of the trampoline helper class by @@ -131,7 +123,7 @@ Bindings should be made against the actual class, not the trampoline helper clas .def("go", &PyAnimal::go); /* <--- THIS IS WRONG, use &Animal::go */ Note, however, that the above is sufficient for allowing python classes to -extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the +extend ``Animal``, but not ``Dog``: see :ref:`virtual_and_inheritance` for the necessary steps required to providing proper overload support for inherited classes. @@ -152,6 +144,30 @@ a virtual method call. >>> call_go(c) u'meow! meow! meow! ' +If you are defining a custom constructor in a derived Python class, you *must* +ensure that you explicitly call the bound C++ constructor using ``__init__``, +*regardless* of whether it is a default constructor or not. Otherwise, the +memory for the C++ portion of the instance will be left uninitialized, which +will generally leave the C++ instance in an invalid state and cause undefined +behavior if the C++ instance is subsequently used. + +Here is an example: + +.. code-block:: python + + class Dachschund(Dog): + def __init__(self, name): + Dog.__init__(self) # Without this, undefind behavior may occur if the C++ portions are referenced. + self.name = name + def bark(self): + return "yap!" + +Note that a direct ``__init__`` constructor *should be called*, and ``super()`` +should not be used. For simple cases of linear inheritance, ``super()`` +may work, but once you begin mixing Python and C++ multiple inheritance, +things will fall apart due to differences between Python's MRO and C++'s +mechanisms. + Please take a look at the :ref:`macro_notes` before using this feature. .. note:: @@ -306,6 +322,8 @@ can now create a python class that inherits from ``Dog``: See the file :file:`tests/test_virtual_functions.cpp` for complete examples using both the duplication and templated trampoline approaches. +.. _extended_aliases: + Extended trampoline class functionality ======================================= @@ -333,7 +351,7 @@ ensuring member initialization and (eventual) destruction. .. seealso:: - See the file :file:`tests/test_alias_initialization.cpp` for complete examples + See the file :file:`tests/test_virtual_functions.cpp` for complete examples showing both normal and forced trampoline instantiation. .. _custom_constructors: @@ -342,29 +360,129 @@ Custom constructors =================== The syntax for binding constructors was previously introduced, but it only -works when a constructor with the given parameters actually exists on the C++ -side. To extend this to more general cases, let's take a look at what actually -happens under the hood: the following statement +works when a constructor of the appropriate arguments actually exists on the +C++ side. To extend this to more general cases, pybind11 makes it possible +to bind factory functions as constructors. For example, suppose you have a +class like this: .. code-block:: cpp + class Example { + private: + Example(int); // private constructor + public: + // Factory function: + static Example create(int a) { return Example(a); } + }; + py::class_<Example>(m, "Example") - .def(py::init<int>()); + .def(py::init(&Example::create)); -is short hand notation for +While it is possible to create a straightforward binding of the static +``create`` method, it may sometimes be preferable to expose it as a constructor +on the Python side. This can be accomplished by calling ``.def(py::init(...))`` +with the function reference returning the new instance passed as an argument. +It is also possible to use this approach to bind a function returning a new +instance by raw pointer or by the holder (e.g. ``std::unique_ptr``). + +The following example shows the different approaches: .. code-block:: cpp + class Example { + private: + Example(int); // private constructor + public: + // Factory function - returned by value: + static Example create(int a) { return Example(a); } + + // These constructors are publicly callable: + Example(double); + Example(int, int); + Example(std::string); + }; + py::class_<Example>(m, "Example") - .def("__init__", - [](Example &instance, int arg) { - new (&instance) Example(arg); - } - ); + // Bind the factory function as a constructor: + .def(py::init(&Example::create)) + // Bind a lambda function returning a pointer wrapped in a holder: + .def(py::init([](std::string arg) { + return std::unique_ptr<Example>(new Example(arg)); + })) + // Return a raw pointer: + .def(py::init([](int a, int b) { return new Example(a, b); })) + // You can mix the above with regular C++ constructor bindings as well: + .def(py::init<double>()) + ; + +When the constructor is invoked from Python, pybind11 will call the factory +function and store the resulting C++ instance in the Python instance. + +When combining factory functions constructors with :ref:`virtual function +trampolines <overriding_virtuals>` there are two approaches. The first is to +add a constructor to the alias class that takes a base value by +rvalue-reference. If such a constructor is available, it will be used to +construct an alias instance from the value returned by the factory function. +The second option is to provide two factory functions to ``py::init()``: the +first will be invoked when no alias class is required (i.e. when the class is +being used but not inherited from in Python), and the second will be invoked +when an alias is required. + +You can also specify a single factory function that always returns an alias +instance: this will result in behaviour similar to ``py::init_alias<...>()``, +as described in the :ref:`extended trampoline class documentation +<extended_aliases>`. + +The following example shows the different factory approaches for a class with +an alias: + +.. code-block:: cpp + + #include <pybind11/factory.h> + class Example { + public: + // ... + virtual ~Example() = default; + }; + class PyExample : public Example { + public: + using Example::Example; + PyExample(Example &&base) : Example(std::move(base)) {} + }; + py::class_<Example, PyExample>(m, "Example") + // Returns an Example pointer. If a PyExample is needed, the Example + // instance will be moved via the extra constructor in PyExample, above. + .def(py::init([]() { return new Example(); })) + // Two callbacks: + .def(py::init([]() { return new Example(); } /* no alias needed */, + []() { return new PyExample(); } /* alias needed */)) + // *Always* returns an alias instance (like py::init_alias<>()) + .def(py::init([]() { return new PyExample(); })) + ; + +Brace initialization +-------------------- + +``pybind11::init<>`` internally uses C++11 brace initialization to call the +constructor of the target class. This means that it can be used to bind +*implicit* constructors as well: + +.. code-block:: cpp + + struct Aggregate { + int a; + std::string b; + }; + + py::class_<Aggregate>(m, "Aggregate") + .def(py::init<int, const std::string &>()); -In other words, :func:`init` creates an anonymous function that invokes an -in-place constructor. Memory allocation etc. is already take care of beforehand -within pybind11. +.. note:: + + Note that brace initialization preferentially invokes constructor overloads + taking a ``std::initializer_list``. In the rare event that this causes an + issue, you can work around it by using ``py::init(...)`` with a lambda + function that constructs the new object as desired. .. _classes_with_non_public_destructors: @@ -435,6 +553,10 @@ Python side: Implicit conversions from ``A`` to ``B`` only work when ``B`` is a custom data type that is exposed to Python via pybind11. + To prevent runaway recursion, implicit conversions are non-reentrant: an + implicit conversion invoked as part of another implicit conversion of the + same type (i.e. from ``A`` to ``B``) will fail. + .. _static_properties: Static properties @@ -491,9 +613,7 @@ to Python. #include <pybind11/operators.h> - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::class_<Vector2>(m, "Vector2") .def(py::init<float, float>()) .def(py::self + py::self) @@ -502,8 +622,6 @@ to Python. .def(float() * py::self) .def(py::self * float()) .def("__repr__", &Vector2::toString); - - return m.ptr(); } Note that a line like @@ -537,13 +655,15 @@ throwing a type error. complete example that demonstrates how to work with overloaded operators in more detail. +.. _pickling: + Pickling support ================ Python's ``pickle`` module provides a powerful facility to serialize and de-serialize a Python object graph into a binary data stream. To pickle and -unpickle C++ classes using pybind11, two additional functions must be provided. -Suppose the class in question has the following signature: +unpickle C++ classes using pybind11, a ``py::pickle()`` definition must be +provided. Suppose the class in question has the following signature: .. code-block:: cpp @@ -559,8 +679,9 @@ Suppose the class in question has the following signature: int m_extra = 0; }; -The binding code including the requisite ``__setstate__`` and ``__getstate__`` methods [#f3]_ -looks as follows: +Pickling support in Python is enabled by defining the ``__setstate__`` and +``__getstate__`` methods [#f3]_. For pybind11 classes, use ``py::pickle()`` +to bind these two functions: .. code-block:: cpp @@ -569,21 +690,28 @@ looks as follows: .def("value", &Pickleable::value) .def("extra", &Pickleable::extra) .def("setExtra", &Pickleable::setExtra) - .def("__getstate__", [](const Pickleable &p) { - /* Return a tuple that fully encodes the state of the object */ - return py::make_tuple(p.value(), p.extra()); - }) - .def("__setstate__", [](Pickleable &p, py::tuple t) { - if (t.size() != 2) - throw std::runtime_error("Invalid state!"); - - /* Invoke the in-place constructor. Note that this is needed even - when the object just has a trivial default constructor */ - new (&p) Pickleable(t[0].cast<std::string>()); - - /* Assign any additional state */ - p.setExtra(t[1].cast<int>()); - }); + .def(py::pickle( + [](const Pickleable &p) { // __getstate__ + /* Return a tuple that fully encodes the state of the object */ + return py::make_tuple(p.value(), p.extra()); + }, + [](py::tuple t) { // __setstate__ + if (t.size() != 2) + throw std::runtime_error("Invalid state!"); + + /* Create a new C++ instance */ + Pickleable p(t[0].cast<std::string>()); + + /* Assign any additional state */ + p.setExtra(t[1].cast<int>()); + + return p; + } + )); + +The ``__setstate__`` part of the ``py::picke()`` definition follows the same +rules as the single-argument version of ``py::init()``. The return type can be +a value, pointer or holder type. See :ref:`custom_constructors` for details. An instance can now be pickled as follows: @@ -631,27 +759,243 @@ interspersed with alias types and holder types (discussed earlier in this document)---pybind11 will automatically find out which is which. The only requirement is that the first template argument is the type to be declared. -There are two caveats regarding the implementation of this feature: +It is also permitted to inherit multiply from exported C++ classes in Python, +as well as inheriting from multiple Python and/or pybind-exported classes. + +There is one caveat regarding the implementation of this feature: + +When only one base type is specified for a C++ type that actually has multiple +bases, pybind11 will assume that it does not participate in multiple +inheritance, which can lead to undefined behavior. In such cases, add the tag +``multiple_inheritance`` to the class constructor: + +.. code-block:: cpp + + py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance()); + +The tag is redundant and does not need to be specified when multiple base types +are listed. + +.. _module_local: + +Module-local class bindings +=========================== + +When creating a binding for a class, pybind by default makes that binding +"global" across modules. What this means is that a type defined in one module +can be returned from any module resulting in the same Python type. For +example, this allows the following: + +.. code-block:: cpp + + // In the module1.cpp binding code for module1: + py::class_<Pet>(m, "Pet") + .def(py::init<std::string>()) + .def_readonly("name", &Pet::name); + +.. code-block:: cpp + + // In the module2.cpp binding code for module2: + m.def("create_pet", [](std::string name) { return new Pet(name); }); + +.. code-block:: pycon + + >>> from module1 import Pet + >>> from module2 import create_pet + >>> pet1 = Pet("Kitty") + >>> pet2 = create_pet("Doggy") + >>> pet2.name() + 'Doggy' + +When writing binding code for a library, this is usually desirable: this +allows, for example, splitting up a complex library into multiple Python +modules. + +In some cases, however, this can cause conflicts. For example, suppose two +unrelated modules make use of an external C++ library and each provide custom +bindings for one of that library's classes. This will result in an error when +a Python program attempts to import both modules (directly or indirectly) +because of conflicting definitions on the external type: + +.. code-block:: cpp + + // dogs.cpp -1. When only one base type is specified for a C++ type that actually has - multiple bases, pybind11 will assume that it does not participate in - multiple inheritance, which can lead to undefined behavior. In such cases, - add the tag ``multiple_inheritance``: + // Binding for external library class: + py::class<pets::Pet>(m, "Pet") + .def("name", &pets::Pet::name); - .. code-block:: cpp + // Binding for local extension class: + py::class<Dog, pets::Pet>(m, "Dog") + .def(py::init<std::string>()); - py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance()); +.. code-block:: cpp + + // cats.cpp, in a completely separate project from the above dogs.cpp. + + // Binding for external library class: + py::class<pets::Pet>(m, "Pet") + .def("get_name", &pets::Pet::name); + + // Binding for local extending class: + py::class<Cat, pets::Pet>(m, "Cat") + .def(py::init<std::string>()); + +.. code-block:: pycon + + >>> import cats + >>> import dogs + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + ImportError: generic_type: type "Pet" is already registered! + +To get around this, you can tell pybind11 to keep the external class binding +localized to the module by passing the ``py::module_local()`` attribute into +the ``py::class_`` constructor: + +.. code-block:: cpp + + // Pet binding in dogs.cpp: + py::class<pets::Pet>(m, "Pet", py::module_local()) + .def("name", &pets::Pet::name); + +.. code-block:: cpp + + // Pet binding in cats.cpp: + py::class<pets::Pet>(m, "Pet", py::module_local()) + .def("get_name", &pets::Pet::name); + +This makes the Python-side ``dogs.Pet`` and ``cats.Pet`` into distinct classes, +avoiding the conflict and allowing both modules to be loaded. C++ code in the +``dogs`` module that casts or returns a ``Pet`` instance will result in a +``dogs.Pet`` Python instance, while C++ code in the ``cats`` module will result +in a ``cats.Pet`` Python instance. + +This does come with two caveats, however: First, external modules cannot return +or cast a ``Pet`` instance to Python (unless they also provide their own local +bindings). Second, from the Python point of view they are two distinct classes. + +Note that the locality only applies in the C++ -> Python direction. When +passing such a ``py::module_local`` type into a C++ function, the module-local +classes are still considered. This means that if the following function is +added to any module (including but not limited to the ``cats`` and ``dogs`` +modules above) it will be callable with either a ``dogs.Pet`` or ``cats.Pet`` +argument: + +.. code-block:: cpp + + m.def("pet_name", [](const pets::Pet &pet) { return pet.name(); }); + +For example, suppose the above function is added to each of ``cats.cpp``, +``dogs.cpp`` and ``frogs.cpp`` (where ``frogs.cpp`` is some other module that +does *not* bind ``Pets`` at all). + +.. code-block:: pycon + + >>> import cats, dogs, frogs # No error because of the added py::module_local() + >>> mycat, mydog = cats.Cat("Fluffy"), dogs.Dog("Rover") + >>> (cats.pet_name(mycat), dogs.pet_name(mydog)) + ('Fluffy', 'Rover') + >>> (cats.pet_name(mydog), dogs.pet_name(mycat), frogs.pet_name(mycat)) + ('Rover', 'Fluffy', 'Fluffy') + +It is possible to use ``py::module_local()`` registrations in one module even +if another module registers the same type globally: within the module with the +module-local definition, all C++ instances will be cast to the associated bound +Python type. In other modules any such values are converted to the global +Python type created elsewhere. + +.. note:: + + STL bindings (as provided via the optional :file:`pybind11/stl_bind.h` + header) apply ``py::module_local`` by default when the bound type might + conflict with other modules; see :ref:`stl_bind` for details. + +.. note:: + + The localization of the bound types is actually tied to the shared object + or binary generated by the compiler/linker. For typical modules created + with ``PYBIND11_MODULE()``, this distinction is not significant. It is + possible, however, when :ref:`embedding` to embed multiple modules in the + same binary (see :ref:`embedding_modules`). In such a case, the + localization will apply across all embedded modules within the same binary. + +.. seealso:: + + The file :file:`tests/test_local_bindings.cpp` contains additional examples + that demonstrate how ``py::module_local()`` works. - The tag is redundant and does not need to be specified when multiple base - types are listed. +Binding protected member functions +================================== + +It's normally not possible to expose ``protected`` member functions to Python: + +.. code-block:: cpp + + class A { + protected: + int foo() const { return 42; } + }; + + py::class_<A>(m, "A") + .def("foo", &A::foo); // error: 'foo' is a protected member of 'A' + +On one hand, this is good because non-``public`` members aren't meant to be +accessed from the outside. But we may want to make use of ``protected`` +functions in derived Python classes. + +The following pattern makes this possible: + +.. code-block:: cpp + + class A { + protected: + int foo() const { return 42; } + }; + + class Publicist : public A { // helper type for exposing protected functions + public: + using A::foo; // inherited with different access modifier + }; + + py::class_<A>(m, "A") // bind the primary class + .def("foo", &Publicist::foo); // expose protected methods via the publicist + +This works because ``&Publicist::foo`` is exactly the same function as +``&A::foo`` (same signature and address), just with a different access +modifier. The only purpose of the ``Publicist`` helper class is to make +the function name ``public``. + +If the intent is to expose ``protected`` ``virtual`` functions which can be +overridden in Python, the publicist pattern can be combined with the previously +described trampoline: + +.. code-block:: cpp + + class A { + public: + virtual ~A() = default; + + protected: + virtual int foo() const { return 42; } + }; + + class Trampoline : public A { + public: + int foo() const override { PYBIND11_OVERLOAD(int, A, foo, ); } + }; + + class Publicist : public A { + public: + using A::foo; + }; + + py::class_<A, Trampoline>(m, "A") // <-- `Trampoline` here + .def("foo", &Publicist::foo); // <-- `Publicist` here, not `Trampoline`! + +.. note:: -2. As was previously discussed in the section on :ref:`overriding_virtuals`, it - is easy to create Python types that derive from C++ classes. It is even - possible to make use of multiple inheritance to declare a Python class which - has e.g. a C++ and a Python class as bases. However, any attempt to create a - type that has *two or more* C++ classes in its hierarchy of base types will - fail with a fatal error message: ``TypeError: multiple bases have instance - lay-out conflict``. Core Python types that are implemented in C (e.g. - ``dict``, ``list``, ``Exception``, etc.) also fall under this combination - and cannot be combined with C++ types bound using pybind11 via multiple - inheritance. + MSVC 2015 has a compiler bug (fixed in version 2017) which + requires a more explicit function binding in the form of + ``.def("foo", static_cast<int (A::*)() const>(&Publicist::foo));`` + where ``int (A::*)() const`` is the type of ``A::foo``. |