diff options
| author | Jason Lowe-Power <jason@lowepower.com> | 2017-11-17 17:02:05 -0800 |
|---|---|---|
| committer | Jason Lowe-Power <jason@lowepower.com> | 2017-12-14 00:27:59 +0000 |
| commit | f07d5069d86e31ecf195664850f79fb00c445bd3 (patch) | |
| tree | f54ac06896fa828f873d199a0e9b25bd94911c79 /ext/pybind11/docs | |
| parent | 3f64b374c49491f18dc2ca538ed8c8597e4aac83 (diff) | |
| download | gem5-f07d5069d86e31ecf195664850f79fb00c445bd3.tar.xz | |
ext: Upgrade PyBind11 to version 2.2.1
This upgrade is necessary for pybind to build with GCC 7.2.
We still need to add the patch for stl.h. MSC_FULL_VER change is no longer
needed.
See https://gem5-review.googlesource.com/c/public/gem5/+/2230
Change-Id: I806729217d022070583994c2dfcaa74476aef30f
Signed-off-by: Jason Lowe-Power <jason@lowepower.com>
Reviewed-on: https://gem5-review.googlesource.com/5801
Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
Maintainer: Andreas Sandberg <andreas.sandberg@arm.com>
Diffstat (limited to 'ext/pybind11/docs')
27 files changed, 2277 insertions, 307 deletions
diff --git a/ext/pybind11/docs/Doxyfile b/ext/pybind11/docs/Doxyfile index 4dc8bf059..1b9d1297c 100644 --- a/ext/pybind11/docs/Doxyfile +++ b/ext/pybind11/docs/Doxyfile @@ -1,5 +1,6 @@ PROJECT_NAME = pybind11 INPUT = ../include/pybind11/ +RECURSIVE = YES GENERATE_HTML = NO GENERATE_LATEX = NO diff --git a/ext/pybind11/docs/advanced/cast/custom.rst b/ext/pybind11/docs/advanced/cast/custom.rst index c854e7fcd..e4f99ac5b 100644 --- a/ext/pybind11/docs/advanced/cast/custom.rst +++ b/ext/pybind11/docs/advanced/cast/custom.rst @@ -78,6 +78,12 @@ type is explicitly allowed. }; }} // namespace pybind11::detail +.. note:: + + A ``type_caster<T>`` defined with ``PYBIND11_TYPE_CASTER(T, ...)`` requires + that ``T`` is default-constructible (``value`` is first default constructed + and then ``load()`` assigns to it). + .. warning:: When using custom type casters, it's important to declare them consistently diff --git a/ext/pybind11/docs/advanced/cast/eigen.rst b/ext/pybind11/docs/advanced/cast/eigen.rst index 5b0b08ca6..acdb51de6 100644 --- a/ext/pybind11/docs/advanced/cast/eigen.rst +++ b/ext/pybind11/docs/advanced/cast/eigen.rst @@ -57,7 +57,7 @@ expected: .. code-block:: cpp - void scale_by_2(Eigen::Ref<Eigen::VectorXd> m) { + void scale_by_2(Eigen::Ref<Eigen::VectorXd> v) { v *= 2; } @@ -252,11 +252,11 @@ copying to take place: using namespace pybind11::literals; // for "arg"_a py::class_<MyClass>(m, "MyClass") // ... other class definitions - .def("some_method", &MyClass::some_method, py::arg().nocopy()); + .def("some_method", &MyClass::some_method, py::arg().noconvert()); m.def("some_function", &some_function, - "big"_a.nocopy(), // <- Don't allow copying for this arg - "small"_a // <- This one can be copied if needed + "big"_a.noconvert(), // <- Don't allow copying for this arg + "small"_a // <- This one can be copied if needed ); With the above binding code, attempting to call the the ``some_method(m)`` diff --git a/ext/pybind11/docs/advanced/cast/functional.rst b/ext/pybind11/docs/advanced/cast/functional.rst index 5d0a01d13..d9b460575 100644 --- a/ext/pybind11/docs/advanced/cast/functional.rst +++ b/ext/pybind11/docs/advanced/cast/functional.rst @@ -56,14 +56,10 @@ trivial to generate binding code for all of these functions. #include <pybind11/functional.h> - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { m.def("func_arg", &func_arg); m.def("func_ret", &func_ret); m.def("func_cpp", &func_cpp); - - return m.ptr(); } The following interactive session shows how to call them from Python. diff --git a/ext/pybind11/docs/advanced/cast/overview.rst b/ext/pybind11/docs/advanced/cast/overview.rst index 54c11a90a..2ac7d3009 100644 --- a/ext/pybind11/docs/advanced/cast/overview.rst +++ b/ext/pybind11/docs/advanced/cast/overview.rst @@ -116,6 +116,9 @@ as arguments and return values, refer to the section on binding :ref:`classes`. +------------------------------------+---------------------------+-------------------------------+ | ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` | +------------------------------------+---------------------------+-------------------------------+ +| ``std::string_view``, | STL C++17 string views | :file:`pybind11/pybind11.h` | +| ``std::u16string_view``, etc. | | | ++------------------------------------+---------------------------+-------------------------------+ | ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` | +------------------------------------+---------------------------+-------------------------------+ | ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` | @@ -144,6 +147,8 @@ as arguments and return values, refer to the section on binding :ref:`classes`. +------------------------------------+---------------------------+-------------------------------+ | ``std::experimental::optional<T>`` | STL optional type (exp.) | :file:`pybind11/stl.h` | +------------------------------------+---------------------------+-------------------------------+ +| ``std::variant<...>`` | Type-safe union (C++17) | :file:`pybind11/stl.h` | ++------------------------------------+---------------------------+-------------------------------+ | ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` | +------------------------------------+---------------------------+-------------------------------+ | ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` | diff --git a/ext/pybind11/docs/advanced/cast/stl.rst b/ext/pybind11/docs/advanced/cast/stl.rst index c76da5ca1..3f30c0290 100644 --- a/ext/pybind11/docs/advanced/cast/stl.rst +++ b/ext/pybind11/docs/advanced/cast/stl.rst @@ -23,9 +23,62 @@ next sections for more details and alternative approaches that avoid this. .. seealso:: - The file :file:`tests/test_python_types.cpp` contains a complete + The file :file:`tests/test_stl.cpp` contains a complete example that demonstrates how to pass STL data types in more detail. +.. _cpp17_container_casters: + +C++17 library containers +======================== + +The :file:`pybind11/stl.h` header also includes support for ``std::optional<>`` +and ``std::variant<>``. These require a C++17 compiler and standard library. +In C++14 mode, ``std::experimental::optional<>`` is supported if available. + +Various versions of these containers also exist for C++11 (e.g. in Boost). +pybind11 provides an easy way to specialize the ``type_caster`` for such +types: + +.. code-block:: cpp + + // `boost::optional` as an example -- can be any `std::optional`-like container + namespace pybind11 { namespace detail { + template <typename T> + struct type_caster<boost::optional<T>> : optional_caster<boost::optional<T>> {}; + }} + +The above should be placed in a header file and included in all translation units +where automatic conversion is needed. Similarly, a specialization can be provided +for custom variant types: + +.. code-block:: cpp + + // `boost::variant` as an example -- can be any `std::variant`-like container + namespace pybind11 { namespace detail { + template <typename... Ts> + struct type_caster<boost::variant<Ts...>> : variant_caster<boost::variant<Ts...>> {}; + + // Specifies the function used to visit the variant -- `apply_visitor` instead of `visit` + template <> + struct visit_helper<boost::variant> { + template <typename... Args> + static auto call(Args &&...args) -> decltype(boost::apply_visitor(args...)) { + return boost::apply_visitor(args...); + } + }; + }} // namespace pybind11::detail + +The ``visit_helper`` specialization is not required if your ``name::variant`` provides +a ``name::visit()`` function. For any other function name, the specialization must be +included to tell pybind11 how to visit the variant. + +.. note:: + + pybind11 only supports the modern implementation of ``boost::variant`` + which makes use of variadic templates. This requires Boost 1.56 or newer. + Additionally, on Windows, MSVC 2017 is required because ``boost::variant`` + falls back to the old non-variadic implementation on MSVC 2015. + .. _opaque: Making opaque types @@ -105,10 +158,10 @@ the declaration before any binding code (e.g. invocations to ``class_::def()``, etc.). This macro must be specified at the top level (and outside of any namespaces), since it instantiates a partial template overload. If your binding code consists of -multiple compilation units, it must be present in every file preceding any -usage of ``std::vector<int>``. Opaque types must also have a corresponding -``class_`` declaration to associate them with a name in Python, and to define a -set of available operations, e.g.: +multiple compilation units, it must be present in every file (typically via a +common header) preceding any usage of ``std::vector<int>``. Opaque types must +also have a corresponding ``class_`` declaration to associate them with a name +in Python, and to define a set of available operations, e.g.: .. code-block:: cpp @@ -122,6 +175,20 @@ set of available operations, e.g.: }, py::keep_alive<0, 1>()) /* Keep vector alive while iterator is used */ // .... +Please take a look at the :ref:`macro_notes` before using the +``PYBIND11_MAKE_OPAQUE`` macro. + +.. seealso:: + + The file :file:`tests/test_opaque_types.cpp` contains a complete + example that demonstrates how to create and expose opaque types using + pybind11 in more detail. + +.. _stl_bind: + +Binding STL containers +====================== + The ability to expose STL containers as native Python objects is a fairly common request, hence pybind11 also provides an optional header file named :file:`pybind11/stl_bind.h` that does exactly this. The mapped containers try @@ -143,14 +210,34 @@ The following example showcases usage of :file:`pybind11/stl_bind.h`: py::bind_vector<std::vector<int>>(m, "VectorInt"); py::bind_map<std::map<std::string, double>>(m, "MapStringDouble"); -Please take a look at the :ref:`macro_notes` before using the -``PYBIND11_MAKE_OPAQUE`` macro. +When binding STL containers pybind11 considers the types of the container's +elements to decide whether the container should be confined to the local module +(via the :ref:`module_local` feature). If the container element types are +anything other than already-bound custom types bound without +``py::module_local()`` the container binding will have ``py::module_local()`` +applied. This includes converting types such as numeric types, strings, Eigen +types; and types that have not yet been bound at the time of the stl container +binding. This module-local binding is designed to avoid potential conflicts +between module bindings (for example, from two separate modules each attempting +to bind ``std::vector<int>`` as a python type). + +It is possible to override this behavior to force a definition to be either +module-local or global. To do so, you can pass the attributes +``py::module_local()`` (to make the binding module-local) or +``py::module_local(false)`` (to make the binding global) into the +``py::bind_vector`` or ``py::bind_map`` arguments: -.. seealso:: +.. code-block:: cpp - The file :file:`tests/test_opaque_types.cpp` contains a complete - example that demonstrates how to create and expose opaque types using - pybind11 in more detail. + py::bind_vector<std::vector<int>>(m, "VectorInt", py::module_local(false)); + +Note, however, that such a global binding would make it impossible to load this +module at the same time as any other pybind module that also attempts to bind +the same container type (``std::vector<int>`` in the above example). + +See :ref:`module_local` for more details on module-local bindings. + +.. seealso:: The file :file:`tests/test_stl_binders.cpp` shows how to use the convenience STL container wrappers. diff --git a/ext/pybind11/docs/advanced/cast/strings.rst b/ext/pybind11/docs/advanced/cast/strings.rst index c70fb0bec..2cdbade3a 100644 --- a/ext/pybind11/docs/advanced/cast/strings.rst +++ b/ext/pybind11/docs/advanced/cast/strings.rst @@ -3,14 +3,23 @@ Strings, bytes and Unicode conversions .. note:: - This section discusses string handling in terms of Python 3 strings. For Python 2.7, replace all occurrences of ``str`` with ``unicode`` and ``bytes`` with ``str``. Python 2.7 users may find it best to use ``from __future__ import unicode_literals`` to avoid unintentionally using ``str`` instead of ``unicode``. + This section discusses string handling in terms of Python 3 strings. For + Python 2.7, replace all occurrences of ``str`` with ``unicode`` and + ``bytes`` with ``str``. Python 2.7 users may find it best to use ``from + __future__ import unicode_literals`` to avoid unintentionally using ``str`` + instead of ``unicode``. Passing Python strings to C++ ============================= -When a Python ``str`` is passed from Python to a C++ function that accepts ``std::string`` or ``char *`` as arguments, pybind11 will encode the Python string to UTF-8. All Python ``str`` can be encoded in UTF-8, so this operation does not fail. +When a Python ``str`` is passed from Python to a C++ function that accepts +``std::string`` or ``char *`` as arguments, pybind11 will encode the Python +string to UTF-8. All Python ``str`` can be encoded in UTF-8, so this operation +does not fail. -The C++ language is encoding agnostic. It is the responsibility of the programmer to track encodings. It's often easiest to simply `use UTF-8 everywhere <http://utf8everywhere.org/>`_. +The C++ language is encoding agnostic. It is the responsibility of the +programmer to track encodings. It's often easiest to simply `use UTF-8 +everywhere <http://utf8everywhere.org/>`_. .. code-block:: c++ @@ -39,20 +48,27 @@ The C++ language is encoding agnostic. It is the responsibility of the programme .. note:: - Some terminal emulators do not support UTF-8 or emoji fonts and may not display the example above correctly. + Some terminal emulators do not support UTF-8 or emoji fonts and may not + display the example above correctly. -The results are the same whether the C++ function accepts arguments by value or reference, and whether or not ``const`` is used. +The results are the same whether the C++ function accepts arguments by value or +reference, and whether or not ``const`` is used. Passing bytes to C++ -------------------- -A Python ``bytes`` object will be passed to C++ functions that accept ``std::string`` or ``char*`` *without* conversion. +A Python ``bytes`` object will be passed to C++ functions that accept +``std::string`` or ``char*`` *without* conversion. Returning C++ strings to Python =============================== -When a C++ function returns a ``std::string`` or ``char*`` to a Python caller, **pybind11 will assume that the string is valid UTF-8** and will decode it to a native Python ``str``, using the same API as Python uses to perform ``bytes.decode('utf-8')``. If this implicit conversion fails, pybind11 will raise a ``UnicodeDecodeError``. +When a C++ function returns a ``std::string`` or ``char*`` to a Python caller, +**pybind11 will assume that the string is valid UTF-8** and will decode it to a +native Python ``str``, using the same API as Python uses to perform +``bytes.decode('utf-8')``. If this implicit conversion fails, pybind11 will +raise a ``UnicodeDecodeError``. .. code-block:: c++ @@ -68,16 +84,22 @@ When a C++ function returns a ``std::string`` or ``char*`` to a Python caller, * True -Because UTF-8 is inclusive of pure ASCII, there is never any issue with returning a pure ASCII string to Python. If there is any possibility that the string is not pure ASCII, it is necessary to ensure the encoding is valid UTF-8. +Because UTF-8 is inclusive of pure ASCII, there is never any issue with +returning a pure ASCII string to Python. If there is any possibility that the +string is not pure ASCII, it is necessary to ensure the encoding is valid +UTF-8. .. warning:: - Implicit conversion assumes that a returned ``char *`` is null-terminated. If there is no null terminator a buffer overrun will occur. + Implicit conversion assumes that a returned ``char *`` is null-terminated. + If there is no null terminator a buffer overrun will occur. Explicit conversions -------------------- -If some C++ code constructs a ``std::string`` that is not a UTF-8 string, one can perform a explicit conversion and return a ``py::str`` object. Explicit conversion has the same overhead as implicit conversion. +If some C++ code constructs a ``std::string`` that is not a UTF-8 string, one +can perform a explicit conversion and return a ``py::str`` object. Explicit +conversion has the same overhead as implicit conversion. .. code-block:: c++ @@ -95,15 +117,20 @@ If some C++ code constructs a ``std::string`` that is not a UTF-8 string, one ca >>> str_output() 'Send your résumé to Alice in HR' -The `Python C API <https://docs.python.org/3/c-api/unicode.html#built-in-codecs>`_ provides several built-in codecs. +The `Python C API +<https://docs.python.org/3/c-api/unicode.html#built-in-codecs>`_ provides +several built-in codecs. -One could also use a third party encoding library such as libiconv to transcode to UTF-8. +One could also use a third party encoding library such as libiconv to transcode +to UTF-8. Return C++ strings without conversion ------------------------------------- -If the data in a C++ ``std::string`` does not represent text and should be returned to Python as ``bytes``, then one can return the data as a ``py::bytes`` object. +If the data in a C++ ``std::string`` does not represent text and should be +returned to Python as ``bytes``, then one can return the data as a +``py::bytes`` object. .. code-block:: c++ @@ -120,7 +147,8 @@ If the data in a C++ ``std::string`` does not represent text and should be retur b'\xba\xd0\xba\xd0' -Note the asymmetry: pybind11 will convert ``bytes`` to ``std::string`` without encoding, but cannot convert ``std::string`` back to ``bytes`` implicitly. +Note the asymmetry: pybind11 will convert ``bytes`` to ``std::string`` without +encoding, but cannot convert ``std::string`` back to ``bytes`` implicitly. .. code-block:: c++ @@ -128,7 +156,7 @@ Note the asymmetry: pybind11 will convert ``bytes`` to ``std::string`` without e [](std::string s) { // Accepts str or bytes from Python return s; // Looks harmless, but implicitly converts to str } - ); + ); .. code-block:: python @@ -142,7 +170,12 @@ Note the asymmetry: pybind11 will convert ``bytes`` to ``std::string`` without e Wide character strings ====================== -When a Python ``str`` is passed to a C++ function expecting ``std::wstring``, ``wchar_t*``, ``std::u16string`` or ``std::u32string``, the ``str`` will be encoded to UTF-16 or UTF-32 depending on how the C++ compiler implements each type, in the platform's endian. When strings of these types are returned, they are assumed to contain valid UTF-16 or UTF-32, and will be decoded to Python ``str``. +When a Python ``str`` is passed to a C++ function expecting ``std::wstring``, +``wchar_t*``, ``std::u16string`` or ``std::u32string``, the ``str`` will be +encoded to UTF-16 or UTF-32 depending on how the C++ compiler implements each +type, in the platform's native endianness. When strings of these types are +returned, they are assumed to contain valid UTF-16 or UTF-32, and will be +decoded to Python ``str``. .. code-block:: c++ @@ -171,17 +204,23 @@ When a Python ``str`` is passed to a C++ function expecting ``std::wstring``, `` .. warning:: - Wide character strings may not work as described on Python 2.7 or Python 3.3 compiled with ``--enable-unicode=ucs2``. + Wide character strings may not work as described on Python 2.7 or Python + 3.3 compiled with ``--enable-unicode=ucs2``. -Strings in multibyte encodings such as Shift-JIS must transcoded to a UTF-8/16/32 before being returned to Python. +Strings in multibyte encodings such as Shift-JIS must transcoded to a +UTF-8/16/32 before being returned to Python. Character literals ================== -C++ functions that accept character literals as input will receive the first character of a Python ``str`` as their input. If the string is longer than one Unicode character, trailing characters will be ignored. +C++ functions that accept character literals as input will receive the first +character of a Python ``str`` as their input. If the string is longer than one +Unicode character, trailing characters will be ignored. -When a character literal is returned from C++ (such as a ``char`` or a ``wchar_t``), it will be converted to a ``str`` that represents the single character. +When a character literal is returned from C++ (such as a ``char`` or a +``wchar_t``), it will be converted to a ``str`` that represents the single +character. .. code-block:: c++ @@ -189,26 +228,34 @@ When a character literal is returned from C++ (such as a ``char`` or a ``wchar_t m.def("pass_wchar", [](wchar_t w) { return w; }); .. code-block:: python - + >>> example.pass_char('A') 'A' -While C++ will cast integers to character types (``char c = 0x65;``), pybind11 does not convert Python integers to characters implicitly. The Python function ``chr()`` can be used to convert integers to characters. +While C++ will cast integers to character types (``char c = 0x65;``), pybind11 +does not convert Python integers to characters implicitly. The Python function +``chr()`` can be used to convert integers to characters. .. code-block:: python - + >>> example.pass_char(0x65) TypeError >>> example.pass_char(chr(0x65)) 'A' -If the desire is to work with an 8-bit integer, use ``int8_t`` or ``uint8_t`` as the argument type. +If the desire is to work with an 8-bit integer, use ``int8_t`` or ``uint8_t`` +as the argument type. Grapheme clusters ----------------- -A single grapheme may be represented by two or more Unicode characters. For example 'é' is usually represented as U+00E9 but can also be expressed as the combining character sequence U+0065 U+0301 (that is, the letter 'e' followed by a combining acute accent). The combining character will be lost if the two-character sequence is passed as an argument, even though it renders as a single grapheme. +A single grapheme may be represented by two or more Unicode characters. For +example 'é' is usually represented as U+00E9 but can also be expressed as the +combining character sequence U+0065 U+0301 (that is, the letter 'e' followed by +a combining acute accent). The combining character will be lost if the +two-character sequence is passed as an argument, even though it renders as a +single grapheme. .. code-block:: python @@ -226,18 +273,31 @@ A single grapheme may be represented by two or more Unicode characters. For exam >>> example.pass_wchar(combining_e_acute) 'e' -Normalizing combining characters before passing the character literal to C++ may resolve *some* of these issues: +Normalizing combining characters before passing the character literal to C++ +may resolve *some* of these issues: .. code-block:: python >>> example.pass_wchar(unicodedata.normalize('NFC', combining_e_acute)) 'é' -In some languages (Thai for example), there are `graphemes that cannot be expressed as a single Unicode code point <http://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries>`_, so there is no way to capture them in a C++ character type. +In some languages (Thai for example), there are `graphemes that cannot be +expressed as a single Unicode code point +<http://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries>`_, so there is +no way to capture them in a C++ character type. + + +C++17 string views +================== +C++17 string views are automatically supported when compiling in C++17 mode. +They follow the same rules for encoding and decoding as the corresponding STL +string type (for example, a ``std::u16string_view`` argument will be passed +UTF-16-encoded data, and a returned ``std::string_view`` will be decoded as +UTF-8). References ========== * `The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) <https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/>`_ -* `C++ - Using STL Strings at Win32 API Boundaries <https://msdn.microsoft.com/en-ca/magazine/mt238407.aspx>`_
\ No newline at end of file +* `C++ - Using STL Strings at Win32 API Boundaries <https://msdn.microsoft.com/en-ca/magazine/mt238407.aspx>`_ diff --git a/ext/pybind11/docs/advanced/classes.rst b/ext/pybind11/docs/advanced/classes.rst index 8896441b6..93deeec62 100644 --- a/ext/pybind11/docs/advanced/classes.rst +++ b/ext/pybind11/docs/advanced/classes.rst @@ -45,9 +45,7 @@ Normally, the binding code for these classes would look as follows: .. code-block:: cpp - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::class_<Animal> animal(m, "Animal"); animal .def("go", &Animal::go); @@ -56,8 +54,6 @@ Normally, the binding code for these classes would look as follows: .def(py::init<>()); m.def("call_go", &call_go); - - return m.ptr(); } However, these bindings are impossible to extend: ``Animal`` is not @@ -90,18 +86,16 @@ functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have a default implementation. There are also two alternate macros :func:`PYBIND11_OVERLOAD_PURE_NAME` and :func:`PYBIND11_OVERLOAD_NAME` which take a string-valued name argument between the *Parent class* and *Name of the -function* slots, which defines the name of function in Python. This is required +function* slots, which defines the name of function in Python. This is required when the C++ and Python versions of the function have different names, e.g. ``operator()`` vs ``__call__``. The binding code also needs a few minor adaptations (highlighted): .. code-block:: cpp - :emphasize-lines: 4,6,7 - - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); + :emphasize-lines: 2,4,5 + PYBIND11_MODULE(example, m) { py::class_<Animal, PyAnimal /* <--- trampoline*/> animal(m, "Animal"); animal .def(py::init<>()) @@ -111,8 +105,6 @@ The binding code also needs a few minor adaptations (highlighted): .def(py::init<>()); m.def("call_go", &call_go); - - return m.ptr(); } Importantly, pybind11 is made aware of the trampoline helper class by @@ -131,7 +123,7 @@ Bindings should be made against the actual class, not the trampoline helper clas .def("go", &PyAnimal::go); /* <--- THIS IS WRONG, use &Animal::go */ Note, however, that the above is sufficient for allowing python classes to -extend ``Animal``, but not ``Dog``: see ref:`virtual_and_inheritance` for the +extend ``Animal``, but not ``Dog``: see :ref:`virtual_and_inheritance` for the necessary steps required to providing proper overload support for inherited classes. @@ -152,6 +144,30 @@ a virtual method call. >>> call_go(c) u'meow! meow! meow! ' +If you are defining a custom constructor in a derived Python class, you *must* +ensure that you explicitly call the bound C++ constructor using ``__init__``, +*regardless* of whether it is a default constructor or not. Otherwise, the +memory for the C++ portion of the instance will be left uninitialized, which +will generally leave the C++ instance in an invalid state and cause undefined +behavior if the C++ instance is subsequently used. + +Here is an example: + +.. code-block:: python + + class Dachschund(Dog): + def __init__(self, name): + Dog.__init__(self) # Without this, undefind behavior may occur if the C++ portions are referenced. + self.name = name + def bark(self): + return "yap!" + +Note that a direct ``__init__`` constructor *should be called*, and ``super()`` +should not be used. For simple cases of linear inheritance, ``super()`` +may work, but once you begin mixing Python and C++ multiple inheritance, +things will fall apart due to differences between Python's MRO and C++'s +mechanisms. + Please take a look at the :ref:`macro_notes` before using this feature. .. note:: @@ -306,6 +322,8 @@ can now create a python class that inherits from ``Dog``: See the file :file:`tests/test_virtual_functions.cpp` for complete examples using both the duplication and templated trampoline approaches. +.. _extended_aliases: + Extended trampoline class functionality ======================================= @@ -333,7 +351,7 @@ ensuring member initialization and (eventual) destruction. .. seealso:: - See the file :file:`tests/test_alias_initialization.cpp` for complete examples + See the file :file:`tests/test_virtual_functions.cpp` for complete examples showing both normal and forced trampoline instantiation. .. _custom_constructors: @@ -342,29 +360,129 @@ Custom constructors =================== The syntax for binding constructors was previously introduced, but it only -works when a constructor with the given parameters actually exists on the C++ -side. To extend this to more general cases, let's take a look at what actually -happens under the hood: the following statement +works when a constructor of the appropriate arguments actually exists on the +C++ side. To extend this to more general cases, pybind11 makes it possible +to bind factory functions as constructors. For example, suppose you have a +class like this: .. code-block:: cpp + class Example { + private: + Example(int); // private constructor + public: + // Factory function: + static Example create(int a) { return Example(a); } + }; + py::class_<Example>(m, "Example") - .def(py::init<int>()); + .def(py::init(&Example::create)); -is short hand notation for +While it is possible to create a straightforward binding of the static +``create`` method, it may sometimes be preferable to expose it as a constructor +on the Python side. This can be accomplished by calling ``.def(py::init(...))`` +with the function reference returning the new instance passed as an argument. +It is also possible to use this approach to bind a function returning a new +instance by raw pointer or by the holder (e.g. ``std::unique_ptr``). + +The following example shows the different approaches: .. code-block:: cpp + class Example { + private: + Example(int); // private constructor + public: + // Factory function - returned by value: + static Example create(int a) { return Example(a); } + + // These constructors are publicly callable: + Example(double); + Example(int, int); + Example(std::string); + }; + py::class_<Example>(m, "Example") - .def("__init__", - [](Example &instance, int arg) { - new (&instance) Example(arg); - } - ); + // Bind the factory function as a constructor: + .def(py::init(&Example::create)) + // Bind a lambda function returning a pointer wrapped in a holder: + .def(py::init([](std::string arg) { + return std::unique_ptr<Example>(new Example(arg)); + })) + // Return a raw pointer: + .def(py::init([](int a, int b) { return new Example(a, b); })) + // You can mix the above with regular C++ constructor bindings as well: + .def(py::init<double>()) + ; + +When the constructor is invoked from Python, pybind11 will call the factory +function and store the resulting C++ instance in the Python instance. + +When combining factory functions constructors with :ref:`virtual function +trampolines <overriding_virtuals>` there are two approaches. The first is to +add a constructor to the alias class that takes a base value by +rvalue-reference. If such a constructor is available, it will be used to +construct an alias instance from the value returned by the factory function. +The second option is to provide two factory functions to ``py::init()``: the +first will be invoked when no alias class is required (i.e. when the class is +being used but not inherited from in Python), and the second will be invoked +when an alias is required. + +You can also specify a single factory function that always returns an alias +instance: this will result in behaviour similar to ``py::init_alias<...>()``, +as described in the :ref:`extended trampoline class documentation +<extended_aliases>`. + +The following example shows the different factory approaches for a class with +an alias: + +.. code-block:: cpp + + #include <pybind11/factory.h> + class Example { + public: + // ... + virtual ~Example() = default; + }; + class PyExample : public Example { + public: + using Example::Example; + PyExample(Example &&base) : Example(std::move(base)) {} + }; + py::class_<Example, PyExample>(m, "Example") + // Returns an Example pointer. If a PyExample is needed, the Example + // instance will be moved via the extra constructor in PyExample, above. + .def(py::init([]() { return new Example(); })) + // Two callbacks: + .def(py::init([]() { return new Example(); } /* no alias needed */, + []() { return new PyExample(); } /* alias needed */)) + // *Always* returns an alias instance (like py::init_alias<>()) + .def(py::init([]() { return new PyExample(); })) + ; + +Brace initialization +-------------------- + +``pybind11::init<>`` internally uses C++11 brace initialization to call the +constructor of the target class. This means that it can be used to bind +*implicit* constructors as well: + +.. code-block:: cpp + + struct Aggregate { + int a; + std::string b; + }; + + py::class_<Aggregate>(m, "Aggregate") + .def(py::init<int, const std::string &>()); -In other words, :func:`init` creates an anonymous function that invokes an -in-place constructor. Memory allocation etc. is already take care of beforehand -within pybind11. +.. note:: + + Note that brace initialization preferentially invokes constructor overloads + taking a ``std::initializer_list``. In the rare event that this causes an + issue, you can work around it by using ``py::init(...)`` with a lambda + function that constructs the new object as desired. .. _classes_with_non_public_destructors: @@ -435,6 +553,10 @@ Python side: Implicit conversions from ``A`` to ``B`` only work when ``B`` is a custom data type that is exposed to Python via pybind11. + To prevent runaway recursion, implicit conversions are non-reentrant: an + implicit conversion invoked as part of another implicit conversion of the + same type (i.e. from ``A`` to ``B``) will fail. + .. _static_properties: Static properties @@ -491,9 +613,7 @@ to Python. #include <pybind11/operators.h> - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::class_<Vector2>(m, "Vector2") .def(py::init<float, float>()) .def(py::self + py::self) @@ -502,8 +622,6 @@ to Python. .def(float() * py::self) .def(py::self * float()) .def("__repr__", &Vector2::toString); - - return m.ptr(); } Note that a line like @@ -537,13 +655,15 @@ throwing a type error. complete example that demonstrates how to work with overloaded operators in more detail. +.. _pickling: + Pickling support ================ Python's ``pickle`` module provides a powerful facility to serialize and de-serialize a Python object graph into a binary data stream. To pickle and -unpickle C++ classes using pybind11, two additional functions must be provided. -Suppose the class in question has the following signature: +unpickle C++ classes using pybind11, a ``py::pickle()`` definition must be +provided. Suppose the class in question has the following signature: .. code-block:: cpp @@ -559,8 +679,9 @@ Suppose the class in question has the following signature: int m_extra = 0; }; -The binding code including the requisite ``__setstate__`` and ``__getstate__`` methods [#f3]_ -looks as follows: +Pickling support in Python is enabled by defining the ``__setstate__`` and +``__getstate__`` methods [#f3]_. For pybind11 classes, use ``py::pickle()`` +to bind these two functions: .. code-block:: cpp @@ -569,21 +690,28 @@ looks as follows: .def("value", &Pickleable::value) .def("extra", &Pickleable::extra) .def("setExtra", &Pickleable::setExtra) - .def("__getstate__", [](const Pickleable &p) { - /* Return a tuple that fully encodes the state of the object */ - return py::make_tuple(p.value(), p.extra()); - }) - .def("__setstate__", [](Pickleable &p, py::tuple t) { - if (t.size() != 2) - throw std::runtime_error("Invalid state!"); - - /* Invoke the in-place constructor. Note that this is needed even - when the object just has a trivial default constructor */ - new (&p) Pickleable(t[0].cast<std::string>()); - - /* Assign any additional state */ - p.setExtra(t[1].cast<int>()); - }); + .def(py::pickle( + [](const Pickleable &p) { // __getstate__ + /* Return a tuple that fully encodes the state of the object */ + return py::make_tuple(p.value(), p.extra()); + }, + [](py::tuple t) { // __setstate__ + if (t.size() != 2) + throw std::runtime_error("Invalid state!"); + + /* Create a new C++ instance */ + Pickleable p(t[0].cast<std::string>()); + + /* Assign any additional state */ + p.setExtra(t[1].cast<int>()); + + return p; + } + )); + +The ``__setstate__`` part of the ``py::picke()`` definition follows the same +rules as the single-argument version of ``py::init()``. The return type can be +a value, pointer or holder type. See :ref:`custom_constructors` for details. An instance can now be pickled as follows: @@ -631,27 +759,243 @@ interspersed with alias types and holder types (discussed earlier in this document)---pybind11 will automatically find out which is which. The only requirement is that the first template argument is the type to be declared. -There are two caveats regarding the implementation of this feature: +It is also permitted to inherit multiply from exported C++ classes in Python, +as well as inheriting from multiple Python and/or pybind-exported classes. + +There is one caveat regarding the implementation of this feature: + +When only one base type is specified for a C++ type that actually has multiple +bases, pybind11 will assume that it does not participate in multiple +inheritance, which can lead to undefined behavior. In such cases, add the tag +``multiple_inheritance`` to the class constructor: + +.. code-block:: cpp + + py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance()); + +The tag is redundant and does not need to be specified when multiple base types +are listed. + +.. _module_local: + +Module-local class bindings +=========================== + +When creating a binding for a class, pybind by default makes that binding +"global" across modules. What this means is that a type defined in one module +can be returned from any module resulting in the same Python type. For +example, this allows the following: + +.. code-block:: cpp + + // In the module1.cpp binding code for module1: + py::class_<Pet>(m, "Pet") + .def(py::init<std::string>()) + .def_readonly("name", &Pet::name); + +.. code-block:: cpp + + // In the module2.cpp binding code for module2: + m.def("create_pet", [](std::string name) { return new Pet(name); }); + +.. code-block:: pycon + + >>> from module1 import Pet + >>> from module2 import create_pet + >>> pet1 = Pet("Kitty") + >>> pet2 = create_pet("Doggy") + >>> pet2.name() + 'Doggy' + +When writing binding code for a library, this is usually desirable: this +allows, for example, splitting up a complex library into multiple Python +modules. + +In some cases, however, this can cause conflicts. For example, suppose two +unrelated modules make use of an external C++ library and each provide custom +bindings for one of that library's classes. This will result in an error when +a Python program attempts to import both modules (directly or indirectly) +because of conflicting definitions on the external type: + +.. code-block:: cpp + + // dogs.cpp -1. When only one base type is specified for a C++ type that actually has - multiple bases, pybind11 will assume that it does not participate in - multiple inheritance, which can lead to undefined behavior. In such cases, - add the tag ``multiple_inheritance``: + // Binding for external library class: + py::class<pets::Pet>(m, "Pet") + .def("name", &pets::Pet::name); - .. code-block:: cpp + // Binding for local extension class: + py::class<Dog, pets::Pet>(m, "Dog") + .def(py::init<std::string>()); - py::class_<MyType, BaseType2>(m, "MyType", py::multiple_inheritance()); +.. code-block:: cpp + + // cats.cpp, in a completely separate project from the above dogs.cpp. + + // Binding for external library class: + py::class<pets::Pet>(m, "Pet") + .def("get_name", &pets::Pet::name); + + // Binding for local extending class: + py::class<Cat, pets::Pet>(m, "Cat") + .def(py::init<std::string>()); + +.. code-block:: pycon + + >>> import cats + >>> import dogs + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + ImportError: generic_type: type "Pet" is already registered! + +To get around this, you can tell pybind11 to keep the external class binding +localized to the module by passing the ``py::module_local()`` attribute into +the ``py::class_`` constructor: + +.. code-block:: cpp + + // Pet binding in dogs.cpp: + py::class<pets::Pet>(m, "Pet", py::module_local()) + .def("name", &pets::Pet::name); + +.. code-block:: cpp + + // Pet binding in cats.cpp: + py::class<pets::Pet>(m, "Pet", py::module_local()) + .def("get_name", &pets::Pet::name); + +This makes the Python-side ``dogs.Pet`` and ``cats.Pet`` into distinct classes, +avoiding the conflict and allowing both modules to be loaded. C++ code in the +``dogs`` module that casts or returns a ``Pet`` instance will result in a +``dogs.Pet`` Python instance, while C++ code in the ``cats`` module will result +in a ``cats.Pet`` Python instance. + +This does come with two caveats, however: First, external modules cannot return +or cast a ``Pet`` instance to Python (unless they also provide their own local +bindings). Second, from the Python point of view they are two distinct classes. + +Note that the locality only applies in the C++ -> Python direction. When +passing such a ``py::module_local`` type into a C++ function, the module-local +classes are still considered. This means that if the following function is +added to any module (including but not limited to the ``cats`` and ``dogs`` +modules above) it will be callable with either a ``dogs.Pet`` or ``cats.Pet`` +argument: + +.. code-block:: cpp + + m.def("pet_name", [](const pets::Pet &pet) { return pet.name(); }); + +For example, suppose the above function is added to each of ``cats.cpp``, +``dogs.cpp`` and ``frogs.cpp`` (where ``frogs.cpp`` is some other module that +does *not* bind ``Pets`` at all). + +.. code-block:: pycon + + >>> import cats, dogs, frogs # No error because of the added py::module_local() + >>> mycat, mydog = cats.Cat("Fluffy"), dogs.Dog("Rover") + >>> (cats.pet_name(mycat), dogs.pet_name(mydog)) + ('Fluffy', 'Rover') + >>> (cats.pet_name(mydog), dogs.pet_name(mycat), frogs.pet_name(mycat)) + ('Rover', 'Fluffy', 'Fluffy') + +It is possible to use ``py::module_local()`` registrations in one module even +if another module registers the same type globally: within the module with the +module-local definition, all C++ instances will be cast to the associated bound +Python type. In other modules any such values are converted to the global +Python type created elsewhere. + +.. note:: + + STL bindings (as provided via the optional :file:`pybind11/stl_bind.h` + header) apply ``py::module_local`` by default when the bound type might + conflict with other modules; see :ref:`stl_bind` for details. + +.. note:: + + The localization of the bound types is actually tied to the shared object + or binary generated by the compiler/linker. For typical modules created + with ``PYBIND11_MODULE()``, this distinction is not significant. It is + possible, however, when :ref:`embedding` to embed multiple modules in the + same binary (see :ref:`embedding_modules`). In such a case, the + localization will apply across all embedded modules within the same binary. + +.. seealso:: + + The file :file:`tests/test_local_bindings.cpp` contains additional examples + that demonstrate how ``py::module_local()`` works. - The tag is redundant and does not need to be specified when multiple base - types are listed. +Binding protected member functions +================================== + +It's normally not possible to expose ``protected`` member functions to Python: + +.. code-block:: cpp + + class A { + protected: + int foo() const { return 42; } + }; + + py::class_<A>(m, "A") + .def("foo", &A::foo); // error: 'foo' is a protected member of 'A' + +On one hand, this is good because non-``public`` members aren't meant to be +accessed from the outside. But we may want to make use of ``protected`` +functions in derived Python classes. + +The following pattern makes this possible: + +.. code-block:: cpp + + class A { + protected: + int foo() const { return 42; } + }; + + class Publicist : public A { // helper type for exposing protected functions + public: + using A::foo; // inherited with different access modifier + }; + + py::class_<A>(m, "A") // bind the primary class + .def("foo", &Publicist::foo); // expose protected methods via the publicist + +This works because ``&Publicist::foo`` is exactly the same function as +``&A::foo`` (same signature and address), just with a different access +modifier. The only purpose of the ``Publicist`` helper class is to make +the function name ``public``. + +If the intent is to expose ``protected`` ``virtual`` functions which can be +overridden in Python, the publicist pattern can be combined with the previously +described trampoline: + +.. code-block:: cpp + + class A { + public: + virtual ~A() = default; + + protected: + virtual int foo() const { return 42; } + }; + + class Trampoline : public A { + public: + int foo() const override { PYBIND11_OVERLOAD(int, A, foo, ); } + }; + + class Publicist : public A { + public: + using A::foo; + }; + + py::class_<A, Trampoline>(m, "A") // <-- `Trampoline` here + .def("foo", &Publicist::foo); // <-- `Publicist` here, not `Trampoline`! + +.. note:: -2. As was previously discussed in the section on :ref:`overriding_virtuals`, it - is easy to create Python types that derive from C++ classes. It is even - possible to make use of multiple inheritance to declare a Python class which - has e.g. a C++ and a Python class as bases. However, any attempt to create a - type that has *two or more* C++ classes in its hierarchy of base types will - fail with a fatal error message: ``TypeError: multiple bases have instance - lay-out conflict``. Core Python types that are implemented in C (e.g. - ``dict``, ``list``, ``Exception``, etc.) also fall under this combination - and cannot be combined with C++ types bound using pybind11 via multiple - inheritance. + MSVC 2015 has a compiler bug (fixed in version 2017) which + requires a more explicit function binding in the form of + ``.def("foo", static_cast<int (A::*)() const>(&Publicist::foo));`` + where ``int (A::*)() const`` is the type of ``A::foo``. diff --git a/ext/pybind11/docs/advanced/embedding.rst b/ext/pybind11/docs/advanced/embedding.rst new file mode 100644 index 000000000..393031603 --- /dev/null +++ b/ext/pybind11/docs/advanced/embedding.rst @@ -0,0 +1,261 @@ +.. _embedding: + +Embedding the interpreter +######################### + +While pybind11 is mainly focused on extending Python using C++, it's also +possible to do the reverse: embed the Python interpreter into a C++ program. +All of the other documentation pages still apply here, so refer to them for +general pybind11 usage. This section will cover a few extra things required +for embedding. + +Getting started +=============== + +A basic executable with an embedded interpreter can be created with just a few +lines of CMake and the ``pybind11::embed`` target, as shown below. For more +information, see :doc:`/compiling`. + +.. code-block:: cmake + + cmake_minimum_required(VERSION 3.0) + project(example) + + find_package(pybind11 REQUIRED) # or `add_subdirectory(pybind11)` + + add_executable(example main.cpp) + target_link_libraries(example PRIVATE pybind11::embed) + +The essential structure of the ``main.cpp`` file looks like this: + +.. code-block:: cpp + + #include <pybind11/embed.h> // everything needed for embedding + namespace py = pybind11; + + int main() { + py::scoped_interpreter guard{}; // start the interpreter and keep it alive + + py::print("Hello, World!"); // use the Python API + } + +The interpreter must be initialized before using any Python API, which includes +all the functions and classes in pybind11. The RAII guard class `scoped_interpreter` +takes care of the interpreter lifetime. After the guard is destroyed, the interpreter +shuts down and clears its memory. No Python functions can be called after this. + +Executing Python code +===================== + +There are a few different ways to run Python code. One option is to use `eval`, +`exec` or `eval_file`, as explained in :ref:`eval`. Here is a quick example in +the context of an executable with an embedded interpreter: + +.. code-block:: cpp + + #include <pybind11/embed.h> + namespace py = pybind11; + + int main() { + py::scoped_interpreter guard{}; + + py::exec(R"( + kwargs = dict(name="World", number=42) + message = "Hello, {name}! The answer is {number}".format(**kwargs) + print(message) + )"); + } + +Alternatively, similar results can be achieved using pybind11's API (see +:doc:`/advanced/pycpp/index` for more details). + +.. code-block:: cpp + + #include <pybind11/embed.h> + namespace py = pybind11; + using namespace py::literals; + + int main() { + py::scoped_interpreter guard{}; + + auto kwargs = py::dict("name"_a="World", "number"_a=42); + auto message = "Hello, {name}! The answer is {number}"_s.format(**kwargs); + py::print(message); + } + +The two approaches can also be combined: + +.. code-block:: cpp + + #include <pybind11/embed.h> + #include <iostream> + + namespace py = pybind11; + using namespace py::literals; + + int main() { + py::scoped_interpreter guard{}; + + auto locals = py::dict("name"_a="World", "number"_a=42); + py::exec(R"( + message = "Hello, {name}! The answer is {number}".format(**locals()) + )", py::globals(), locals); + + auto message = locals["message"].cast<std::string>(); + std::cout << message; + } + +Importing modules +================= + +Python modules can be imported using `module::import()`: + +.. code-block:: cpp + + py::module sys = py::module::import("sys"); + py::print(sys.attr("path")); + +For convenience, the current working directory is included in ``sys.path`` when +embedding the interpreter. This makes it easy to import local Python files: + +.. code-block:: python + + """calc.py located in the working directory""" + + def add(i, j): + return i + j + + +.. code-block:: cpp + + py::module calc = py::module::import("calc"); + py::object result = calc.attr("add")(1, 2); + int n = result.cast<int>(); + assert(n == 3); + +Modules can be reloaded using `module::reload()` if the source is modified e.g. +by an external process. This can be useful in scenarios where the application +imports a user defined data processing script which needs to be updated after +changes by the user. Note that this function does not reload modules recursively. + +.. _embedding_modules: + +Adding embedded modules +======================= + +Embedded binary modules can be added using the `PYBIND11_EMBEDDED_MODULE` macro. +Note that the definition must be placed at global scope. They can be imported +like any other module. + +.. code-block:: cpp + + #include <pybind11/embed.h> + namespace py = pybind11; + + PYBIND11_EMBEDDED_MODULE(fast_calc, m) { + // `m` is a `py::module` which is used to bind functions and classes + m.def("add", [](int i, int j) { + return i + j; + }); + } + + int main() { + py::scoped_interpreter guard{}; + + auto fast_calc = py::module::import("fast_calc"); + auto result = fast_calc.attr("add")(1, 2).cast<int>(); + assert(result == 3); + } + +Unlike extension modules where only a single binary module can be created, on +the embedded side an unlimited number of modules can be added using multiple +`PYBIND11_EMBEDDED_MODULE` definitions (as long as they have unique names). + +These modules are added to Python's list of builtins, so they can also be +imported in pure Python files loaded by the interpreter. Everything interacts +naturally: + +.. code-block:: python + + """py_module.py located in the working directory""" + import cpp_module + + a = cpp_module.a + b = a + 1 + + +.. code-block:: cpp + + #include <pybind11/embed.h> + namespace py = pybind11; + + PYBIND11_EMBEDDED_MODULE(cpp_module, m) { + m.attr("a") = 1; + } + + int main() { + py::scoped_interpreter guard{}; + + auto py_module = py::module::import("py_module"); + + auto locals = py::dict("fmt"_a="{} + {} = {}", **py_module.attr("__dict__")); + assert(locals["a"].cast<int>() == 1); + assert(locals["b"].cast<int>() == 2); + + py::exec(R"( + c = a + b + message = fmt.format(a, b, c) + )", py::globals(), locals); + + assert(locals["c"].cast<int>() == 3); + assert(locals["message"].cast<std::string>() == "1 + 2 = 3"); + } + + +Interpreter lifetime +==================== + +The Python interpreter shuts down when `scoped_interpreter` is destroyed. After +this, creating a new instance will restart the interpreter. Alternatively, the +`initialize_interpreter` / `finalize_interpreter` pair of functions can be used +to directly set the state at any time. + +Modules created with pybind11 can be safely re-initialized after the interpreter +has been restarted. However, this may not apply to third-party extension modules. +The issue is that Python itself cannot completely unload extension modules and +there are several caveats with regard to interpreter restarting. In short, not +all memory may be freed, either due to Python reference cycles or user-created +global data. All the details can be found in the CPython documentation. + +.. warning:: + + Creating two concurrent `scoped_interpreter` guards is a fatal error. So is + calling `initialize_interpreter` for a second time after the interpreter + has already been initialized. + + Do not use the raw CPython API functions ``Py_Initialize`` and + ``Py_Finalize`` as these do not properly handle the lifetime of + pybind11's internal data. + + +Sub-interpreter support +======================= + +Creating multiple copies of `scoped_interpreter` is not possible because it +represents the main Python interpreter. Sub-interpreters are something different +and they do permit the existence of multiple interpreters. This is an advanced +feature of the CPython API and should be handled with care. pybind11 does not +currently offer a C++ interface for sub-interpreters, so refer to the CPython +documentation for all the details regarding this feature. + +We'll just mention a couple of caveats the sub-interpreters support in pybind11: + + 1. Sub-interpreters will not receive independent copies of embedded modules. + Instead, these are shared and modifications in one interpreter may be + reflected in another. + + 2. Managing multiple threads, multiple interpreters and the GIL can be + challenging and there are several caveats here, even within the pure + CPython API (please refer to the Python docs for details). As for + pybind11, keep in mind that `gil_scoped_release` and `gil_scoped_acquire` + do not take sub-interpreters into account. diff --git a/ext/pybind11/docs/advanced/functions.rst b/ext/pybind11/docs/advanced/functions.rst index e0b0fe095..c7892b5d3 100644 --- a/ext/pybind11/docs/advanced/functions.rst +++ b/ext/pybind11/docs/advanced/functions.rst @@ -162,22 +162,25 @@ Additional call policies ======================== In addition to the above return value policies, further *call policies* can be -specified to indicate dependencies between parameters. In general, call policies -are required when the C++ object is any kind of container and another object is being -added to the container. - -There is currently just -one policy named ``keep_alive<Nurse, Patient>``, which indicates that the -argument with index ``Patient`` should be kept alive at least until the -argument with index ``Nurse`` is freed by the garbage collector. Argument +specified to indicate dependencies between parameters or ensure a certain state +for the function call. + +Keep alive +---------- + +In general, this policy is required when the C++ object is any kind of container +and another object is being added to the container. ``keep_alive<Nurse, Patient>`` +indicates that the argument with index ``Patient`` should be kept alive at least +until the argument with index ``Nurse`` is freed by the garbage collector. Argument indices start at one, while zero refers to the return value. For methods, index ``1`` refers to the implicit ``this`` pointer, while regular arguments begin at index ``2``. Arbitrarily many call policies can be specified. When a ``Nurse`` with value ``None`` is detected at runtime, the call policy does nothing. -This feature internally relies on the ability to create a *weak reference* to -the nurse object, which is permitted by all classes exposed via pybind11. When -the nurse object does not support weak references, an exception will be thrown. +When the nurse is not a pybind11-registered type, the implementation internally +relies on the ability to create a *weak reference* to the nurse object. When +the nurse object is not a pybind11-registered type and does not support weak +references, an exception will be thrown. Consider the following example: here, the binding code for a list append operation ties the lifetime of the newly added element to the underlying @@ -188,16 +191,53 @@ container: py::class_<List>(m, "List") .def("append", &List::append, py::keep_alive<1, 2>()); +For consistency, the argument indexing is identical for constructors. Index +``1`` still refers to the implicit ``this`` pointer, i.e. the object which is +being constructed. Index ``0`` refers to the return type which is presumed to +be ``void`` when a constructor is viewed like a function. The following example +ties the lifetime of the constructor element to the constructed object: + +.. code-block:: cpp + + py::class_<Nurse>(m, "Nurse") + .def(py::init<Patient &>(), py::keep_alive<1, 2>()); + .. note:: ``keep_alive`` is analogous to the ``with_custodian_and_ward`` (if Nurse, Patient != 0) and ``with_custodian_and_ward_postcall`` (if Nurse/Patient == 0) policies from Boost.Python. +Call guard +---------- + +The ``call_guard<T>`` policy allows any scope guard type ``T`` to be placed +around the function call. For example, this definition: + +.. code-block:: cpp + + m.def("foo", foo, py::call_guard<T>()); + +is equivalent to the following pseudocode: + +.. code-block:: cpp + + m.def("foo", [](args...) { + T scope_guard; + return foo(args...); // forwarded arguments + }); + +The only requirement is that ``T`` is default-constructible, but otherwise any +scope guard will work. This is very useful in combination with `gil_scoped_release`. +See :ref:`gil`. + +Multiple guards can also be specified as ``py::call_guard<T1, T2, T3...>``. The +constructor order is left to right and destruction happens in reverse. + .. seealso:: - The file :file:`tests/test_keep_alive.cpp` contains a complete example - that demonstrates using :class:`keep_alive` in more detail. + The file :file:`tests/test_call_policies.cpp` contains a complete example + that demonstrates using `keep_alive` and `call_guard` in more detail. .. _python_objects_as_args: @@ -378,6 +418,55 @@ name, i.e. by specifying ``py::arg().noconvert()``. need to specify a ``py::arg()`` annotation for each argument with the no-convert argument modified to ``py::arg().noconvert()``. +.. _none_arguments: + +Allow/Prohibiting None arguments +================================ + +When a C++ type registered with :class:`py::class_` is passed as an argument to +a function taking the instance as pointer or shared holder (e.g. ``shared_ptr`` +or a custom, copyable holder as described in :ref:`smart_pointers`), pybind +allows ``None`` to be passed from Python which results in calling the C++ +function with ``nullptr`` (or an empty holder) for the argument. + +To explicitly enable or disable this behaviour, using the +``.none`` method of the :class:`py::arg` object: + +.. code-block:: cpp + + py::class_<Dog>(m, "Dog").def(py::init<>()); + py::class_<Cat>(m, "Cat").def(py::init<>()); + m.def("bark", [](Dog *dog) -> std::string { + if (dog) return "woof!"; /* Called with a Dog instance */ + else return "(no dog)"; /* Called with None, d == nullptr */ + }, py::arg("dog").none(true)); + m.def("meow", [](Cat *cat) -> std::string { + // Can't be called with None argument + return "meow"; + }, py::arg("cat").none(false)); + +With the above, the Python call ``bark(None)`` will return the string ``"(no +dog)"``, while attempting to call ``meow(None)`` will raise a ``TypeError``: + +.. code-block:: pycon + + >>> from animals import Dog, Cat, bark, meow + >>> bark(Dog()) + 'woof!' + >>> meow(Cat()) + 'meow' + >>> bark(None) + '(no dog)' + >>> meow(None) + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + TypeError: meow(): incompatible function arguments. The following argument types are supported: + 1. (cat: animals.Cat) -> str + + Invoked with: None + +The default behaviour when the tag is unspecified is to allow ``None``. + Overload resolution order ========================= diff --git a/ext/pybind11/docs/advanced/misc.rst b/ext/pybind11/docs/advanced/misc.rst index d98466512..87481ba32 100644 --- a/ext/pybind11/docs/advanced/misc.rst +++ b/ext/pybind11/docs/advanced/misc.rst @@ -15,6 +15,7 @@ T2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates the beginning of the next parameter. Use a ``typedef`` to bind the template to another name and use it in the macro to avoid this problem. +.. _gil: Global Interpreter Lock (GIL) ============================= @@ -27,7 +28,7 @@ multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this could be realized as follows (important changes highlighted): .. code-block:: cpp - :emphasize-lines: 8,9,33,34 + :emphasize-lines: 8,9,31,32 class PyAnimal : public Animal { public: @@ -48,9 +49,7 @@ could be realized as follows (important changes highlighted): } }; - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::class_<Animal, PyAnimal> animal(m, "Animal"); animal .def(py::init<>()) @@ -64,10 +63,15 @@ could be realized as follows (important changes highlighted): py::gil_scoped_release release; return call_go(animal); }); - - return m.ptr(); } +The ``call_go`` wrapper can also be simplified using the `call_guard` policy +(see :ref:`call_policies`) which yields the same result: + +.. code-block:: cpp + + m.def("call_go", &call_go, py::call_guard<py::gil_scoped_release>()); + Binding sequence data types, iterators, the slicing protocol, etc. ================================================================== @@ -131,22 +135,16 @@ has been executed: Naturally, both methods will fail when there are cyclic dependencies. -Note that compiling code which has its default symbol visibility set to -*hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the -ability to access types defined in another extension module. Workarounds -include changing the global symbol visibility (not recommended, because it will -lead unnecessarily large binaries) or manually exporting types that are -accessed by multiple extension modules: +Note that pybind11 code compiled with hidden-by-default symbol visibility (e.g. +via the command line flag ``-fvisibility=hidden`` on GCC/Clang), which is +required proper pybind11 functionality, can interfere with the ability to +access types defined in another extension module. Working around this requires +manually exporting types that are accessed by multiple extension modules; +pybind11 provides a macro to do just this: .. code-block:: cpp - #ifdef _WIN32 - # define EXPORT_TYPE __declspec(dllexport) - #else - # define EXPORT_TYPE __attribute__ ((visibility("default"))) - #endif - - class EXPORT_TYPE Dog : public Animal { + class PYBIND11_EXPORT Dog : public Animal { ... }; @@ -175,7 +173,8 @@ Module Destructors pybind11 does not provide an explicit mechanism to invoke cleanup code at module destruction time. In rare cases where such functionality is required, it -is possible to emulate it using Python capsules with a destruction callback. +is possible to emulate it using Python capsules or weak references with a +destruction callback. .. code-block:: cpp @@ -185,6 +184,39 @@ is possible to emulate it using Python capsules with a destruction callback. m.add_object("_cleanup", py::capsule(cleanup_callback)); +This approach has the potential downside that instances of classes exposed +within the module may still be alive when the cleanup callback is invoked +(whether this is acceptable will generally depend on the application). + +Alternatively, the capsule may also be stashed within a type object, which +ensures that it not called before all instances of that type have been +collected: + +.. code-block:: cpp + + auto cleanup_callback = []() { /* ... */ }; + m.attr("BaseClass").attr("_cleanup") = py::capsule(cleanup_callback); + +Both approaches also expose a potentially dangerous ``_cleanup`` attribute in +Python, which may be undesirable from an API standpoint (a premature explicit +call from Python might lead to undefined behavior). Yet another approach that +avoids this issue involves weak reference with a cleanup callback: + +.. code-block:: cpp + + // Register a callback function that is invoked when the BaseClass object is colelcted + py::cpp_function cleanup_callback( + [](py::handle weakref) { + // perform cleanup here -- this function is called with the GIL held + + weakref.dec_ref(); // release weak reference + } + ); + + // Create a weak reference with a cleanup callback and initially leak it + (void) py::weakref(m.attr("BaseClass"), cleanup_callback).release(); + + Generating documentation using Sphinx ===================================== @@ -225,15 +257,11 @@ The class ``options`` allows you to selectively suppress auto-generated signatur .. code-block:: cpp - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::options options; options.disable_function_signatures(); - + m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers"); - - return m.ptr(); } Note that changes to the settings affect only function bindings created during the diff --git a/ext/pybind11/docs/advanced/pycpp/numpy.rst b/ext/pybind11/docs/advanced/pycpp/numpy.rst index 6bcc46719..98b0c25b9 100644 --- a/ext/pybind11/docs/advanced/pycpp/numpy.rst +++ b/ext/pybind11/docs/advanced/pycpp/numpy.rst @@ -57,11 +57,11 @@ specification. struct buffer_info { void *ptr; - size_t itemsize; + ssize_t itemsize; std::string format; - int ndim; - std::vector<size_t> shape; - std::vector<size_t> strides; + ssize_t ndim; + std::vector<ssize_t> shape; + std::vector<ssize_t> strides; }; To create a C++ function that can take a Python buffer object as an argument, @@ -95,11 +95,11 @@ buffer objects (e.g. a NumPy matrix). throw std::runtime_error("Incompatible buffer dimension!"); auto strides = Strides( - info.strides[rowMajor ? 0 : 1] / sizeof(Scalar), - info.strides[rowMajor ? 1 : 0] / sizeof(Scalar)); + info.strides[rowMajor ? 0 : 1] / (py::ssize_t)sizeof(Scalar), + info.strides[rowMajor ? 1 : 0] / (py::ssize_t)sizeof(Scalar)); auto map = Eigen::Map<Matrix, 0, Strides>( - static_cat<Scalar *>(info.ptr), info.shape[0], info.shape[1], strides); + static_cast<Scalar *>(info.ptr), info.shape[0], info.shape[1], strides); new (&m) Matrix(map); }); @@ -111,18 +111,14 @@ as follows: .def_buffer([](Matrix &m) -> py::buffer_info { return py::buffer_info( - m.data(), /* Pointer to buffer */ - sizeof(Scalar), /* Size of one scalar */ - /* Python struct-style format descriptor */ - py::format_descriptor<Scalar>::format(), - /* Number of dimensions */ - 2, - /* Buffer dimensions */ - { (size_t) m.rows(), - (size_t) m.cols() }, - /* Strides (in bytes) for each index */ + m.data(), /* Pointer to buffer */ + sizeof(Scalar), /* Size of one scalar */ + py::format_descriptor<Scalar>::format(), /* Python struct-style format descriptor */ + 2, /* Number of dimensions */ + { m.rows(), m.cols() }, /* Buffer dimensions */ { sizeof(Scalar) * (rowMajor ? m.cols() : 1), sizeof(Scalar) * (rowMajor ? 1 : m.rows()) } + /* Strides (in bytes) for each index */ ); }) @@ -194,7 +190,7 @@ expects the type followed by field names: }; // ... - PYBIND11_PLUGIN(test) { + PYBIND11_MODULE(test, m) { // ... PYBIND11_NUMPY_DTYPE(A, x, y); @@ -202,6 +198,13 @@ expects the type followed by field names: /* now both A and B can be used as template arguments to py::array_t */ } +The structure should consist of fundamental arithmetic types, ``std::complex``, +previously registered substructures, and arrays of any of the above. Both C++ +arrays and ``std::array`` are supported. While there is a static assertion to +prevent many types of unsupported structures, it is still the user's +responsibility to use only "plain" structures that can be safely manipulated as +raw memory without violating invariants. + Vectorizing functions ===================== @@ -236,27 +239,13 @@ by the compiler. The result is returned as a NumPy array of type The scalar argument ``z`` is transparently replicated 4 times. The input arrays ``x`` and ``y`` are automatically converted into the right types (they are of type ``numpy.dtype.int64`` but need to be ``numpy.dtype.int32`` and -``numpy.dtype.float32``, respectively) - -Sometimes we might want to explicitly exclude an argument from the vectorization -because it makes little sense to wrap it in a NumPy array. For instance, -suppose the function signature was - -.. code-block:: cpp +``numpy.dtype.float32``, respectively). - double my_func(int x, float y, my_custom_type *z); +.. note:: -This can be done with a stateful Lambda closure: - -.. code-block:: cpp - - // Vectorize a lambda function with a capture object (e.g. to exclude some arguments from the vectorization) - m.def("vectorized_func", - [](py::array_t<int> x, py::array_t<float> y, my_custom_type *z) { - auto stateful_closure = [z](int x, float y) { return my_func(x, y, z); }; - return py::vectorize(stateful_closure)(x, y); - } - ); + Only arithmetic, complex, and POD types passed by value or by ``const &`` + reference are vectorized; all other arguments are passed through as-is. + Functions taking rvalue reference arguments cannot be vectorized. In cases where the computation is too complicated to be reduced to ``vectorize``, it will be necessary to create and access the buffer contents @@ -295,10 +284,8 @@ simply using ``vectorize``). return result; } - PYBIND11_PLUGIN(test) { - py::module m("test"); + PYBIND11_MODULE(test, m) { m.def("add_arrays", &add_arrays, "Add two NumPy arrays"); - return m.ptr(); } .. seealso:: @@ -322,17 +309,17 @@ where ``N`` gives the required dimensionality of the array: m.def("sum_3d", [](py::array_t<double> x) { auto r = x.unchecked<3>(); // x must have ndim = 3; can be non-writeable double sum = 0; - for (size_t i = 0; i < r.shape(0); i++) - for (size_t j = 0; j < r.shape(1); j++) - for (size_t k = 0; k < r.shape(2); k++) + for (ssize_t i = 0; i < r.shape(0); i++) + for (ssize_t j = 0; j < r.shape(1); j++) + for (ssize_t k = 0; k < r.shape(2); k++) sum += r(i, j, k); return sum; }); m.def("increment_3d", [](py::array_t<double> x) { auto r = x.mutable_unchecked<3>(); // Will throw if ndim != 3 or flags.writeable is false - for (size_t i = 0; i < r.shape(0); i++) - for (size_t j = 0; j < r.shape(1); j++) - for (size_t k = 0; k < r.shape(2); k++) + for (ssize_t i = 0; i < r.shape(0); i++) + for (ssize_t j = 0; j < r.shape(1); j++) + for (ssize_t k = 0; k < r.shape(2); k++) r(i, j, k) += 1.0; }, py::arg().noconvert()); diff --git a/ext/pybind11/docs/advanced/pycpp/object.rst b/ext/pybind11/docs/advanced/pycpp/object.rst index ae58876de..117131edc 100644 --- a/ext/pybind11/docs/advanced/pycpp/object.rst +++ b/ext/pybind11/docs/advanced/pycpp/object.rst @@ -33,12 +33,50 @@ The reverse direction uses the following syntax: When conversion fails, both directions throw the exception :class:`cast_error`. +.. _python_libs: + +Accessing Python libraries from C++ +=================================== + +It is also possible to import objects defined in the Python standard +library or available in the current Python environment (``sys.path``) and work +with these in C++. + +This example obtains a reference to the Python ``Decimal`` class. + +.. code-block:: cpp + + // Equivalent to "from decimal import Decimal" + py::object Decimal = py::module::import("decimal").attr("Decimal"); + +.. code-block:: cpp + + // Try to import scipy + py::object scipy = py::module::import("scipy"); + return scipy.attr("__version__"); + .. _calling_python_functions: Calling Python functions ======================== -It is also possible to call python functions via ``operator()``. +It is also possible to call Python classes, functions and methods +via ``operator()``. + +.. code-block:: cpp + + // Construct a Python object of class Decimal + py::object pi = Decimal("3.14159"); + +.. code-block:: cpp + + // Use Python to make our directories + py::object os = py::module::import("os"); + py::object makedirs = os.attr("makedirs"); + makedirs("/tmp/path/to/somewhere"); + +One can convert the result obtained from Python to a pure C++ version +if a ``py::class_`` or type conversion is defined. .. code-block:: cpp @@ -46,6 +84,37 @@ It is also possible to call python functions via ``operator()``. py::object result_py = f(1234, "hello", some_instance); MyClass &result = result_py.cast<MyClass>(); +.. _calling_python_methods: + +Calling Python methods +======================== + +To call an object's method, one can again use ``.attr`` to obtain access to the +Python method. + +.. code-block:: cpp + + // Calculate e^π in decimal + py::object exp_pi = pi.attr("exp")(); + py::print(py::str(exp_pi)); + +In the example above ``pi.attr("exp")`` is a *bound method*: it will always call +the method for that same instance of the class. Alternately one can create an +*unbound method* via the Python class (instead of instance) and pass the ``self`` +object explicitly, followed by other arguments. + +.. code-block:: cpp + + py::object decimal_exp = Decimal.attr("exp"); + + // Compute the e^n for n=0..4 + for (int n = 0; n < 5; n++) { + py::print(decimal_exp(Decimal(n)); + } + +Keyword arguments +================= + Keyword arguments are also supported. In Python, there is the usual call syntax: .. code-block:: python @@ -62,6 +131,9 @@ In C++, the same call can be made using: using namespace pybind11::literals; // to bring in the `_a` literal f(1234, "say"_a="hello", "to"_a=some_instance); // keyword call in C++ +Unpacking arguments +=================== + Unpacking of ``*args`` and ``**kwargs`` is also possible and can be mixed with other arguments: @@ -90,7 +162,7 @@ Generalized unpacking according to PEP448_ is also supported: .. seealso:: - The file :file:`tests/test_python_types.cpp` contains a complete + The file :file:`tests/test_pytypes.cpp` contains a complete example that demonstrates passing native Python types in more detail. The file :file:`tests/test_callbacks.cpp` presents a few examples of calling Python functions from C++, including keywords arguments and unpacking. diff --git a/ext/pybind11/docs/advanced/pycpp/utilities.rst b/ext/pybind11/docs/advanced/pycpp/utilities.rst index ba0dbef88..369e7c94d 100644 --- a/ext/pybind11/docs/advanced/pycpp/utilities.rst +++ b/ext/pybind11/docs/advanced/pycpp/utilities.rst @@ -21,19 +21,81 @@ expected in Python: auto args = py::make_tuple("unpacked", true); py::print("->", *args, "end"_a="<-"); // -> unpacked True <- +.. _ostream_redirect: + +Capturing standard output from ostream +====================================== + +Often, a library will use the streams ``std::cout`` and ``std::cerr`` to print, +but this does not play well with Python's standard ``sys.stdout`` and ``sys.stderr`` +redirection. Replacing a library's printing with `py::print <print>` may not +be feasible. This can be fixed using a guard around the library function that +redirects output to the corresponding Python streams: + +.. code-block:: cpp + + #include <pybind11/iostream.h> + + ... + + // Add a scoped redirect for your noisy code + m.def("noisy_func", []() { + py::scoped_ostream_redirect stream( + std::cout, // std::ostream& + py::module::import("sys").attr("stdout") // Python output + ); + call_noisy_func(); + }); + +This method respects flushes on the output streams and will flush if needed +when the scoped guard is destroyed. This allows the output to be redirected in +real time, such as to a Jupyter notebook. The two arguments, the C++ stream and +the Python output, are optional, and default to standard output if not given. An +extra type, `py::scoped_estream_redirect <scoped_estream_redirect>`, is identical +except for defaulting to ``std::cerr`` and ``sys.stderr``; this can be useful with +`py::call_guard`, which allows multiple items, but uses the default constructor: + +.. code-block:: py + + // Alternative: Call single function using call guard + m.def("noisy_func", &call_noisy_function, + py::call_guard<py::scoped_ostream_redirect, + py::scoped_estream_redirect>()); + +The redirection can also be done in Python with the addition of a context +manager, using the `py::add_ostream_redirect() <add_ostream_redirect>` function: + +.. code-block:: cpp + + py::add_ostream_redirect(m, "ostream_redirect"); + +The name in Python defaults to ``ostream_redirect`` if no name is passed. This +creates the following context manager in Python: + +.. code-block:: python + + with ostream_redirect(stdout=True, stderr=True): + noisy_function() + +It defaults to redirecting both streams, though you can use the keyword +arguments to disable one of the streams if needed. + +.. note:: + + The above methods will not redirect C-level output to file descriptors, such + as ``fprintf``. For those cases, you'll need to redirect the file + descriptors either directly in C or with Python's ``os.dup2`` function + in an operating-system dependent way. + +.. _eval: + Evaluating Python expressions from strings and files ==================================================== -pybind11 provides the :func:`eval` and :func:`eval_file` functions to evaluate +pybind11 provides the `eval`, `exec` and `eval_file` functions to evaluate Python expressions and statements. The following example illustrates how they can be used. -Both functions accept a template parameter that describes how the argument -should be interpreted. Possible choices include ``eval_expr`` (isolated -expression), ``eval_single_statement`` (a single statement, return value is -always ``none``), and ``eval_statements`` (sequence of statements, return value -is always ``none``). - .. code-block:: cpp // At beginning of file @@ -48,10 +110,35 @@ is always ``none``). int result = py::eval("my_variable + 10", scope).cast<int>(); // Evaluate a sequence of statements - py::eval<py::eval_statements>( + py::exec( "print('Hello')\n" "print('world!');", scope); // Evaluate the statements in an separate Python file on disk py::eval_file("script.py", scope); + +C++11 raw string literals are also supported and quite handy for this purpose. +The only requirement is that the first statement must be on a new line following +the raw string delimiter ``R"(``, ensuring all lines have common leading indent: + +.. code-block:: cpp + + py::exec(R"( + x = get_answer() + if x == 42: + print('Hello World!') + else: + print('Bye!') + )", scope + ); + +.. note:: + + `eval` and `eval_file` accept a template parameter that describes how the + string/file should be interpreted. Possible choices include ``eval_expr`` + (isolated expression), ``eval_single_statement`` (a single statement, return + value is always ``none``), and ``eval_statements`` (sequence of statements, + return value is always ``none``). `eval` defaults to ``eval_expr``, + `eval_file` defaults to ``eval_statements`` and `exec` is just a shortcut + for ``eval<eval_statements>``. diff --git a/ext/pybind11/docs/advanced/smart_ptrs.rst b/ext/pybind11/docs/advanced/smart_ptrs.rst index e4a238603..da57748ca 100644 --- a/ext/pybind11/docs/advanced/smart_ptrs.rst +++ b/ext/pybind11/docs/advanced/smart_ptrs.rst @@ -63,16 +63,12 @@ code? std::shared_ptr<Child> child; }; - PYBIND11_PLUGIN(example) { - py::module m("example"); - + PYBIND11_MODULE(example, m) { py::class_<Child, std::shared_ptr<Child>>(m, "Child"); py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent") .def(py::init<>()) .def("get_child", &Parent::get_child); - - return m.ptr(); } The following Python code will cause undefined behavior (and likely a diff --git a/ext/pybind11/docs/basics.rst b/ext/pybind11/docs/basics.rst index 33c60049d..447250ed9 100644 --- a/ext/pybind11/docs/basics.rst +++ b/ext/pybind11/docs/basics.rst @@ -73,6 +73,8 @@ For brevity, all code examples assume that the following two lines are present: Some features may require additional headers, but those will be specified as needed. +.. _simple_example: + Creating bindings for a simple function ======================================= @@ -96,25 +98,21 @@ a file named :file:`example.cpp` with the following contents: return i + j; } - namespace py = pybind11; - - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); + PYBIND11_MODULE(example, m) { + m.doc() = "pybind11 example plugin"; // optional module docstring m.def("add", &add, "A function which adds two numbers"); - - return m.ptr(); } .. [#f1] In practice, implementation and binding code will generally be located in separate files. -The :func:`PYBIND11_PLUGIN` macro creates a function that will be called when an -``import`` statement is issued from within Python. The next line creates a -module named ``example`` (with the supplied docstring). The method -:func:`module::def` generates binding code that exposes the -``add()`` function to Python. The last line returns the internal Python object -associated with ``m`` to the Python interpreter. +The :func:`PYBIND11_MODULE` macro creates a function that will be called when an +``import`` statement is issued from within Python. The module name (``example``) +is given as the first macro argument (it should not be in quotes). The second +argument (``m``) defines a variable of type :class:`py::module <module>` which +is the main interface for creating bindings. The method :func:`module::def` +generates binding code that exposes the ``add()`` function to Python. .. note:: @@ -124,23 +122,31 @@ associated with ``m`` to the Python interpreter. approach and the used syntax are borrowed from Boost.Python, though the underlying implementation is very different. -pybind11 is a header-only-library, hence it is not necessary to link against -any special libraries (other than Python itself). On Windows, use the CMake -build file discussed in section :ref:`cmake`. On Linux and Mac OS, the above -example can be compiled using the following command +pybind11 is a header-only library, hence it is not necessary to link against +any special libraries and there are no intermediate (magic) translation steps. +On Linux, the above example can be compiled using the following command: .. code-block:: bash - $ c++ -O3 -shared -std=c++11 -I <path-to-pybind11>/include `python-config --cflags --ldflags` example.cpp -o example.so + $ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` + +For more details on the required compiler flags on Linux and MacOS, see +:ref:`building_manually`. For complete cross-platform compilation instructions, +refer to the :ref:`compiling` page. + +The `python_example`_ and `cmake_example`_ repositories are also a good place +to start. They are both complete project examples with cross-platform build +systems. The only difference between the two is that `python_example`_ uses +Python's ``setuptools`` to build the module, while `cmake_example`_ uses CMake +(which may be preferable for existing C++ projects). -In general, it is advisable to include several additional build parameters -that can considerably reduce the size of the created binary. Refer to section -:ref:`cmake` for a detailed example of a suitable cross-platform CMake-based -build system. +.. _python_example: https://github.com/pybind/python_example +.. _cmake_example: https://github.com/pybind/cmake_example -Assuming that the created file :file:`example.so` (:file:`example.pyd` on Windows) -is located in the current directory, the following interactive Python session -shows how to load and execute the example. +Building the above C++ code will produce a binary module file that can be +imported to Python. Assuming that the compiled module is located in the +current directory, the following interactive Python session shows how to +load and execute the example: .. code-block:: pycon @@ -261,12 +267,10 @@ converted using the function ``py::cast``. .. code-block:: cpp - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); + PYBIND11_MODULE(example, m) { m.attr("the_answer") = 42; py::object world = py::cast("World"); m.attr("what") = world; - return m.ptr(); } These are then accessible from Python: diff --git a/ext/pybind11/docs/benchmark.py b/ext/pybind11/docs/benchmark.py index 6f02e92ff..6dc0604ea 100644 --- a/ext/pybind11/docs/benchmark.py +++ b/ext/pybind11/docs/benchmark.py @@ -33,10 +33,8 @@ def generate_dummy_code_pybind11(nclasses=10): result = "#include <pybind11/pybind11.h>\n\n" result += "namespace py = pybind11;\n\n" result += decl + '\n' - result += "PYBIND11_PLUGIN(example) {\n" - result += " py::module m(\"example\");" + result += "PYBIND11_MODULE(example, m) {\n" result += bindings - result += " return m.ptr();" result += "}" return result diff --git a/ext/pybind11/docs/benchmark.rst b/ext/pybind11/docs/benchmark.rst index 8babaa319..59d533df9 100644 --- a/ext/pybind11/docs/benchmark.rst +++ b/ext/pybind11/docs/benchmark.rst @@ -31,8 +31,7 @@ Here is an example of the binding code for one class: }; ... - PYBIND11_PLUGIN(example) { - py::module m("example"); + PYBIND11_MODULE(example, m) { ... py::class_<cl034>(m, "cl034") .def("fn_000", &cl034::fn_000) @@ -40,7 +39,6 @@ Here is an example of the binding code for one class: .def("fn_002", &cl034::fn_002) .def("fn_003", &cl034::fn_003) ... - return m.ptr(); } The Boost.Python version looks almost identical except that a return value diff --git a/ext/pybind11/docs/changelog.rst b/ext/pybind11/docs/changelog.rst index aba8a2009..1ca501d15 100644 --- a/ext/pybind11/docs/changelog.rst +++ b/ext/pybind11/docs/changelog.rst @@ -6,6 +6,365 @@ Changelog Starting with version 1.8.0, pybind11 releases use a `semantic versioning <http://semver.org>`_ policy. +v2.3.0 (Not yet released) +----------------------------------------------------- + +* TBD + +v2.2.1 (September 14, 2017) +----------------------------------------------------- + +* Added ``py::module::reload()`` member function for reloading a module. + `#1040 <https://github.com/pybind/pybind11/pull/1040>`_. + +* Fixed a reference leak in the number converter. + `#1078 <https://github.com/pybind/pybind11/pull/1078>`_. + +* Fixed compilation with Clang on host GCC < 5 (old libstdc++ which isn't fully + C++11 compliant). `#1062 <https://github.com/pybind/pybind11/pull/1062>`_. + +* Fixed a regression where the automatic ``std::vector<bool>`` caster would + fail to compile. The same fix also applies to any container which returns + element proxies instead of references. + `#1053 <https://github.com/pybind/pybind11/pull/1053>`_. + +* Fixed a regression where the ``py::keep_alive`` policy could not be applied + to constructors. `#1065 <https://github.com/pybind/pybind11/pull/1065>`_. + +* Fixed a nullptr dereference when loading a ``py::module_local`` type + that's only registered in an external module. + `#1058 <https://github.com/pybind/pybind11/pull/1058>`_. + +* Fixed implicit conversion of accessors to types derived from ``py::object``. + `#1076 <https://github.com/pybind/pybind11/pull/1076>`_. + +* The ``name`` in ``PYBIND11_MODULE(name, variable)`` can now be a macro. + `#1082 <https://github.com/pybind/pybind11/pull/1082>`_. + +* Relaxed overly strict ``py::pickle()`` check for matching get and set types. + `#1064 <https://github.com/pybind/pybind11/pull/1064>`_. + +* Conversion errors now try to be more informative when it's likely that + a missing header is the cause (e.g. forgetting ``<pybind11/stl.h>``). + `#1077 <https://github.com/pybind/pybind11/pull/1077>`_. + +v2.2.0 (August 31, 2017) +----------------------------------------------------- + +* Support for embedding the Python interpreter. See the + :doc:`documentation page </advanced/embedding>` for a + full overview of the new features. + `#774 <https://github.com/pybind/pybind11/pull/774>`_, + `#889 <https://github.com/pybind/pybind11/pull/889>`_, + `#892 <https://github.com/pybind/pybind11/pull/892>`_, + `#920 <https://github.com/pybind/pybind11/pull/920>`_. + + .. code-block:: cpp + + #include <pybind11/embed.h> + namespace py = pybind11; + + int main() { + py::scoped_interpreter guard{}; // start the interpreter and keep it alive + + py::print("Hello, World!"); // use the Python API + } + +* Support for inheriting from multiple C++ bases in Python. + `#693 <https://github.com/pybind/pybind11/pull/693>`_. + + .. code-block:: python + + from cpp_module import CppBase1, CppBase2 + + class PyDerived(CppBase1, CppBase2): + def __init__(self): + CppBase1.__init__(self) # C++ bases must be initialized explicitly + CppBase2.__init__(self) + +* ``PYBIND11_MODULE`` is now the preferred way to create module entry points. + ``PYBIND11_PLUGIN`` is deprecated. See :ref:`macros` for details. + `#879 <https://github.com/pybind/pybind11/pull/879>`_. + + .. code-block:: cpp + + // new + PYBIND11_MODULE(example, m) { + m.def("add", [](int a, int b) { return a + b; }); + } + + // old + PYBIND11_PLUGIN(example) { + py::module m("example"); + m.def("add", [](int a, int b) { return a + b; }); + return m.ptr(); + } + +* pybind11's headers and build system now more strictly enforce hidden symbol + visibility for extension modules. This should be seamless for most users, + but see the :doc:`upgrade` if you use a custom build system. + `#995 <https://github.com/pybind/pybind11/pull/995>`_. + +* Support for ``py::module_local`` types which allow multiple modules to + export the same C++ types without conflicts. This is useful for opaque + types like ``std::vector<int>``. ``py::bind_vector`` and ``py::bind_map`` + now default to ``py::module_local`` if their elements are builtins or + local types. See :ref:`module_local` for details. + `#949 <https://github.com/pybind/pybind11/pull/949>`_, + `#981 <https://github.com/pybind/pybind11/pull/981>`_, + `#995 <https://github.com/pybind/pybind11/pull/995>`_, + `#997 <https://github.com/pybind/pybind11/pull/997>`_. + +* Custom constructors can now be added very easily using lambdas or factory + functions which return a class instance by value, pointer or holder. This + supersedes the old placement-new ``__init__`` technique. + See :ref:`custom_constructors` for details. + `#805 <https://github.com/pybind/pybind11/pull/805>`_, + `#1014 <https://github.com/pybind/pybind11/pull/1014>`_. + + .. code-block:: cpp + + struct Example { + Example(std::string); + }; + + py::class_<Example>(m, "Example") + .def(py::init<std::string>()) // existing constructor + .def(py::init([](int n) { // custom constructor + return std::make_unique<Example>(std::to_string(n)); + })); + +* Similarly to custom constructors, pickling support functions are now bound + using the ``py::pickle()`` adaptor which improves type safety. See the + :doc:`upgrade` and :ref:`pickling` for details. + `#1038 <https://github.com/pybind/pybind11/pull/1038>`_. + +* Builtin support for converting C++17 standard library types and general + conversion improvements: + + 1. C++17 ``std::variant`` is supported right out of the box. C++11/14 + equivalents (e.g. ``boost::variant``) can also be added with a simple + user-defined specialization. See :ref:`cpp17_container_casters` for details. + `#811 <https://github.com/pybind/pybind11/pull/811>`_, + `#845 <https://github.com/pybind/pybind11/pull/845>`_, + `#989 <https://github.com/pybind/pybind11/pull/989>`_. + + 2. Out-of-the-box support for C++17 ``std::string_view``. + `#906 <https://github.com/pybind/pybind11/pull/906>`_. + + 3. Improved compatibility of the builtin ``optional`` converter. + `#874 <https://github.com/pybind/pybind11/pull/874>`_. + + 4. The ``bool`` converter now accepts ``numpy.bool_`` and types which + define ``__bool__`` (Python 3.x) or ``__nonzero__`` (Python 2.7). + `#925 <https://github.com/pybind/pybind11/pull/925>`_. + + 5. C++-to-Python casters are now more efficient and move elements out + of rvalue containers whenever possible. + `#851 <https://github.com/pybind/pybind11/pull/851>`_, + `#936 <https://github.com/pybind/pybind11/pull/936>`_, + `#938 <https://github.com/pybind/pybind11/pull/938>`_. + + 6. Fixed ``bytes`` to ``std::string/char*`` conversion on Python 3. + `#817 <https://github.com/pybind/pybind11/pull/817>`_. + + 7. Fixed lifetime of temporary C++ objects created in Python-to-C++ conversions. + `#924 <https://github.com/pybind/pybind11/pull/924>`_. + +* Scope guard call policy for RAII types, e.g. ``py::call_guard<py::gil_scoped_release>()``, + ``py::call_guard<py::scoped_ostream_redirect>()``. See :ref:`call_policies` for details. + `#740 <https://github.com/pybind/pybind11/pull/740>`_. + +* Utility for redirecting C++ streams to Python (e.g. ``std::cout`` -> + ``sys.stdout``). Scope guard ``py::scoped_ostream_redirect`` in C++ and + a context manager in Python. See :ref:`ostream_redirect`. + `#1009 <https://github.com/pybind/pybind11/pull/1009>`_. + +* Improved handling of types and exceptions across module boundaries. + `#915 <https://github.com/pybind/pybind11/pull/915>`_, + `#951 <https://github.com/pybind/pybind11/pull/951>`_, + `#995 <https://github.com/pybind/pybind11/pull/995>`_. + +* Fixed destruction order of ``py::keep_alive`` nurse/patient objects + in reference cycles. + `#856 <https://github.com/pybind/pybind11/pull/856>`_. + +* Numpy and buffer protocol related improvements: + + 1. Support for negative strides in Python buffer objects/numpy arrays. This + required changing integers from unsigned to signed for the related C++ APIs. + Note: If you have compiler warnings enabled, you may notice some new conversion + warnings after upgrading. These can be resolved with ``static_cast``. + `#782 <https://github.com/pybind/pybind11/pull/782>`_. + + 2. Support ``std::complex`` and arrays inside ``PYBIND11_NUMPY_DTYPE``. + `#831 <https://github.com/pybind/pybind11/pull/831>`_, + `#832 <https://github.com/pybind/pybind11/pull/832>`_. + + 3. Support for constructing ``py::buffer_info`` and ``py::arrays`` using + arbitrary containers or iterators instead of requiring a ``std::vector``. + `#788 <https://github.com/pybind/pybind11/pull/788>`_, + `#822 <https://github.com/pybind/pybind11/pull/822>`_, + `#860 <https://github.com/pybind/pybind11/pull/860>`_. + + 4. Explicitly check numpy version and require >= 1.7.0. + `#819 <https://github.com/pybind/pybind11/pull/819>`_. + +* Support for allowing/prohibiting ``None`` for specific arguments and improved + ``None`` overload resolution order. See :ref:`none_arguments` for details. + `#843 <https://github.com/pybind/pybind11/pull/843>`_. + `#859 <https://github.com/pybind/pybind11/pull/859>`_. + +* Added ``py::exec()`` as a shortcut for ``py::eval<py::eval_statements>()`` + and support for C++11 raw string literals as input. See :ref:`eval`. + `#766 <https://github.com/pybind/pybind11/pull/766>`_, + `#827 <https://github.com/pybind/pybind11/pull/827>`_. + +* ``py::vectorize()`` ignores non-vectorizable arguments and supports + member functions. + `#762 <https://github.com/pybind/pybind11/pull/762>`_. + +* Support for bound methods as callbacks (``pybind11/functional.h``). + `#815 <https://github.com/pybind/pybind11/pull/815>`_. + +* Allow aliasing pybind11 methods: ``cls.attr("foo") = cls.attr("bar")``. + `#802 <https://github.com/pybind/pybind11/pull/802>`_. + +* Don't allow mixed static/non-static overloads. + `#804 <https://github.com/pybind/pybind11/pull/804>`_. + +* Fixed overriding static properties in derived classes. + `#784 <https://github.com/pybind/pybind11/pull/784>`_. + +* Improved deduction of member functions of a derived class when its bases + aren't registered with pybind11. + `#855 <https://github.com/pybind/pybind11/pull/855>`_. + + .. code-block:: cpp + + struct Base { + int foo() { return 42; } + } + + struct Derived : Base {} + + // Now works, but previously required also binding `Base` + py::class_<Derived>(m, "Derived") + .def("foo", &Derived::foo); // function is actually from `Base` + +* The implementation of ``py::init<>`` now uses C++11 brace initialization + syntax to construct instances, which permits binding implicit constructors of + aggregate types. `#1015 <https://github.com/pybind/pybind11/pull/1015>`_. + + .. code-block:: cpp + + struct Aggregate { + int a; + std::string b; + }; + + py::class_<Aggregate>(m, "Aggregate") + .def(py::init<int, const std::string &>()); + +* Fixed issues with multiple inheritance with offset base/derived pointers. + `#812 <https://github.com/pybind/pybind11/pull/812>`_, + `#866 <https://github.com/pybind/pybind11/pull/866>`_, + `#960 <https://github.com/pybind/pybind11/pull/960>`_. + +* Fixed reference leak of type objects. + `#1030 <https://github.com/pybind/pybind11/pull/1030>`_. + +* Improved support for the ``/std:c++14`` and ``/std:c++latest`` modes + on MSVC 2017. + `#841 <https://github.com/pybind/pybind11/pull/841>`_, + `#999 <https://github.com/pybind/pybind11/pull/999>`_. + +* Fixed detection of private operator new on MSVC. + `#893 <https://github.com/pybind/pybind11/pull/893>`_, + `#918 <https://github.com/pybind/pybind11/pull/918>`_. + +* Intel C++ compiler compatibility fixes. + `#937 <https://github.com/pybind/pybind11/pull/937>`_. + +* Fixed implicit conversion of `py::enum_` to integer types on Python 2.7. + `#821 <https://github.com/pybind/pybind11/pull/821>`_. + +* Added ``py::hash`` to fetch the hash value of Python objects, and + ``.def(hash(py::self))`` to provide the C++ ``std::hash`` as the Python + ``__hash__`` method. + `#1034 <https://github.com/pybind/pybind11/pull/1034>`_. + +* Fixed ``__truediv__`` on Python 2 and ``__itruediv__`` on Python 3. + `#867 <https://github.com/pybind/pybind11/pull/867>`_. + +* ``py::capsule`` objects now support the ``name`` attribute. This is useful + for interfacing with ``scipy.LowLevelCallable``. + `#902 <https://github.com/pybind/pybind11/pull/902>`_. + +* Fixed ``py::make_iterator``'s ``__next__()`` for past-the-end calls. + `#897 <https://github.com/pybind/pybind11/pull/897>`_. + +* Added ``error_already_set::matches()`` for checking Python exceptions. + `#772 <https://github.com/pybind/pybind11/pull/772>`_. + +* Deprecated ``py::error_already_set::clear()``. It's no longer needed + following a simplification of the ``py::error_already_set`` class. + `#954 <https://github.com/pybind/pybind11/pull/954>`_. + +* Deprecated ``py::handle::operator==()`` in favor of ``py::handle::is()`` + `#825 <https://github.com/pybind/pybind11/pull/825>`_. + +* Deprecated ``py::object::borrowed``/``py::object::stolen``. + Use ``py::object::borrowed_t{}``/``py::object::stolen_t{}`` instead. + `#771 <https://github.com/pybind/pybind11/pull/771>`_. + +* Changed internal data structure versioning to avoid conflicts between + modules compiled with different revisions of pybind11. + `#1012 <https://github.com/pybind/pybind11/pull/1012>`_. + +* Additional compile-time and run-time error checking and more informative messages. + `#786 <https://github.com/pybind/pybind11/pull/786>`_, + `#794 <https://github.com/pybind/pybind11/pull/794>`_, + `#803 <https://github.com/pybind/pybind11/pull/803>`_. + +* Various minor improvements and fixes. + `#764 <https://github.com/pybind/pybind11/pull/764>`_, + `#791 <https://github.com/pybind/pybind11/pull/791>`_, + `#795 <https://github.com/pybind/pybind11/pull/795>`_, + `#840 <https://github.com/pybind/pybind11/pull/840>`_, + `#844 <https://github.com/pybind/pybind11/pull/844>`_, + `#846 <https://github.com/pybind/pybind11/pull/846>`_, + `#849 <https://github.com/pybind/pybind11/pull/849>`_, + `#858 <https://github.com/pybind/pybind11/pull/858>`_, + `#862 <https://github.com/pybind/pybind11/pull/862>`_, + `#871 <https://github.com/pybind/pybind11/pull/871>`_, + `#872 <https://github.com/pybind/pybind11/pull/872>`_, + `#881 <https://github.com/pybind/pybind11/pull/881>`_, + `#888 <https://github.com/pybind/pybind11/pull/888>`_, + `#899 <https://github.com/pybind/pybind11/pull/899>`_, + `#928 <https://github.com/pybind/pybind11/pull/928>`_, + `#931 <https://github.com/pybind/pybind11/pull/931>`_, + `#944 <https://github.com/pybind/pybind11/pull/944>`_, + `#950 <https://github.com/pybind/pybind11/pull/950>`_, + `#952 <https://github.com/pybind/pybind11/pull/952>`_, + `#962 <https://github.com/pybind/pybind11/pull/962>`_, + `#965 <https://github.com/pybind/pybind11/pull/965>`_, + `#970 <https://github.com/pybind/pybind11/pull/970>`_, + `#978 <https://github.com/pybind/pybind11/pull/978>`_, + `#979 <https://github.com/pybind/pybind11/pull/979>`_, + `#986 <https://github.com/pybind/pybind11/pull/986>`_, + `#1020 <https://github.com/pybind/pybind11/pull/1020>`_, + `#1027 <https://github.com/pybind/pybind11/pull/1027>`_, + `#1037 <https://github.com/pybind/pybind11/pull/1037>`_. + +* Testing improvements. + `#798 <https://github.com/pybind/pybind11/pull/798>`_, + `#882 <https://github.com/pybind/pybind11/pull/882>`_, + `#898 <https://github.com/pybind/pybind11/pull/898>`_, + `#900 <https://github.com/pybind/pybind11/pull/900>`_, + `#921 <https://github.com/pybind/pybind11/pull/921>`_, + `#923 <https://github.com/pybind/pybind11/pull/923>`_, + `#963 <https://github.com/pybind/pybind11/pull/963>`_. v2.1.1 (April 7, 2017) ----------------------------------------------------- diff --git a/ext/pybind11/docs/classes.rst b/ext/pybind11/docs/classes.rst index 30fb2a2d5..ca2477e83 100644 --- a/ext/pybind11/docs/classes.rst +++ b/ext/pybind11/docs/classes.rst @@ -27,15 +27,11 @@ The binding code for ``Pet`` looks as follows: namespace py = pybind11; - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind11 example plugin"); - + PYBIND11_MODULE(example, m) { py::class_<Pet>(m, "Pet") .def(py::init<const std::string &>()) .def("setName", &Pet::setName) .def("getName", &Pet::getName); - - return m.ptr(); } :class:`class_` creates bindings for a C++ *class* or *struct*-style data @@ -229,8 +225,8 @@ just brings them on par. .. _inheritance: -Inheritance -=========== +Inheritance and automatic upcasting +=================================== Suppose now that the example consists of two data structures with an inheritance relationship: @@ -287,6 +283,65 @@ expose fields and methods of both types: >>> p.bark() u'woof!' +The C++ classes defined above are regular non-polymorphic types with an +inheritance relationship. This is reflected in Python: + +.. code-block:: cpp + + // Return a base pointer to a derived instance + m.def("pet_store", []() { return std::unique_ptr<Pet>(new Dog("Molly")); }); + +.. code-block:: pycon + + >>> p = example.pet_store() + >>> type(p) # `Dog` instance behind `Pet` pointer + Pet # no pointer upcasting for regular non-polymorphic types + >>> p.bark() + AttributeError: 'Pet' object has no attribute 'bark' + +The function returned a ``Dog`` instance, but because it's a non-polymorphic +type behind a base pointer, Python only sees a ``Pet``. In C++, a type is only +considered polymorphic if it has at least one virtual function and pybind11 +will automatically recognize this: + +.. code-block:: cpp + + struct PolymorphicPet { + virtual ~PolymorphicPet() = default; + }; + + struct PolymorphicDog : PolymorphicPet { + std::string bark() const { return "woof!"; } + }; + + // Same binding code + py::class_<PolymorphicPet>(m, "PolymorphicPet"); + py::class_<PolymorphicDog, PolymorphicPet>(m, "PolymorphicDog") + .def(py::init<>()) + .def("bark", &PolymorphicDog::bark); + + // Again, return a base pointer to a derived instance + m.def("pet_store2", []() { return std::unique_ptr<PolymorphicPet>(new PolymorphicDog); }); + +.. code-block:: pycon + + >>> p = example.pet_store2() + >>> type(p) + PolymorphicDog # automatically upcast + >>> p.bark() + u'woof!' + +Given a pointer to a polymorphic base, pybind11 performs automatic upcasting +to the actual derived type. Note that this goes beyond the usual situation in +C++: we don't just get access to the virtual functions of the base, we get the +concrete derived type including functions and attributes that the base type may +not even be aware of. + +.. seealso:: + + For more information about polymorphic behavior see :ref:`overriding_virtuals`. + + Overloaded methods ================== diff --git a/ext/pybind11/docs/compiling.rst b/ext/pybind11/docs/compiling.rst index c7053dbf9..b5d6ce948 100644 --- a/ext/pybind11/docs/compiling.rst +++ b/ext/pybind11/docs/compiling.rst @@ -1,3 +1,5 @@ +.. _compiling: + Build systems ############# @@ -14,10 +16,10 @@ the [python_example]_ repository. Building with cppimport ======================== - cppimport is a small Python import hook that determines whether there is a C++ - source file whose name matches the requested module. If there is, the file is - compiled as a Python extension using pybind11 and placed in the same folder as - the C++ source file. Python is then able to find the module and load it. +[cppimport]_ is a small Python import hook that determines whether there is a C++ +source file whose name matches the requested module. If there is, the file is +compiled as a Python extension using pybind11 and placed in the same folder as +the C++ source file. Python is then able to find the module and load it. .. [cppimport] https://github.com/tbenthompson/cppimport @@ -74,13 +76,15 @@ removes this target from the default build (see CMake docs for details). Since pybind11 is a template library, ``pybind11_add_module`` adds compiler flags to ensure high quality code generation without bloat arising from long -symbol names and duplication of code in different translation units. The -additional flags enable LTO (Link Time Optimization), set default visibility -to *hidden* and strip unneeded symbols. See the :ref:`FAQ entry <faq:symhidden>` -for a more detailed explanation. These optimizations are never applied in -``Debug`` mode. If ``NO_EXTRAS`` is given, they will always be disabled, even -in ``Release`` mode. However, this will result in code bloat and is generally -not recommended. +symbol names and duplication of code in different translation units. It +sets default visibility to *hidden*, which is required for some pybind11 +features and functionality when attempting to load multiple pybind11 modules +compiled under different pybind11 versions. It also adds additional flags +enabling LTO (Link Time Optimization) and strip unneeded symbols. See the +:ref:`FAQ entry <faq:symhidden>` for a more detailed explanation. These +latter optimizations are never applied in ``Debug`` mode. If ``NO_EXTRAS`` is +given, they will always be disabled, even in ``Release`` mode. However, this +will result in code bloat and is generally not recommended. As stated above, LTO is enabled by default. Some newer compilers also support different flavors of LTO such as `ThinLTO`_. Setting ``THIN_LTO`` will cause @@ -92,17 +96,28 @@ regular LTO if ``-flto=thin`` is not available. Configuration variables ----------------------- -By default, pybind11 will compile modules with the latest C++ standard -available on the target compiler. To override this, the standard flag can -be given explicitly in ``PYBIND11_CPP_STANDARD``: +By default, pybind11 will compile modules with the C++14 standard, if available +on the target compiler, falling back to C++11 if C++14 support is not +available. Note, however, that this default is subject to change: future +pybind11 releases are expected to migrate to newer C++ standards as they become +available. To override this, the standard flag can be given explicitly in +``PYBIND11_CPP_STANDARD``: .. code-block:: cmake + # Use just one of these: + # GCC/clang: set(PYBIND11_CPP_STANDARD -std=c++11) + set(PYBIND11_CPP_STANDARD -std=c++14) + set(PYBIND11_CPP_STANDARD -std=c++1z) # Experimental C++17 support + # MSVC: + set(PYBIND11_CPP_STANDARD /std:c++14) + set(PYBIND11_CPP_STANDARD /std:c++latest) # Enables some MSVC C++17 features + add_subdirectory(pybind11) # or find_package(pybind11) Note that this and all other configuration variables must be set **before** the -call to ``add_subdiretory`` or ``find_package``. The variables can also be set +call to ``add_subdirectory`` or ``find_package``. The variables can also be set when calling CMake from the command line using the ``-D<variable>=<value>`` flag. The target Python version can be selected by setting ``PYBIND11_PYTHON_VERSION`` @@ -170,11 +185,84 @@ to an independently constructed (through ``add_library``, not flags (i.e. this is up to you). These include Link Time Optimization (``-flto`` on GCC/Clang/ICPC, ``/GL`` - and ``/LTCG`` on Visual Studio). Default-hidden symbols on GCC/Clang/ICPC - (``-fvisibility=hidden``) and .OBJ files with many sections on Visual Studio - (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an + and ``/LTCG`` on Visual Studio) and .OBJ files with many sections on Visual + Studio (``/bigobj``). The :ref:`FAQ <faq:symhidden>` contains an explanation on why these are needed. +Embedding the Python interpreter +-------------------------------- + +In addition to extension modules, pybind11 also supports embedding Python into +a C++ executable or library. In CMake, simply link with the ``pybind11::embed`` +target. It provides everything needed to get the interpreter running. The Python +headers and libraries are attached to the target. Unlike ``pybind11::module``, +there is no need to manually set any additional properties here. For more +information about usage in C++, see :doc:`/advanced/embedding`. + +.. code-block:: cmake + + cmake_minimum_required(VERSION 3.0) + project(example) + + find_package(pybind11 REQUIRED) # or add_subdirectory(pybind11) + + add_executable(example main.cpp) + target_link_libraries(example PRIVATE pybind11::embed) + +.. _building_manually: + +Building manually +================= + +pybind11 is a header-only library, hence it is not necessary to link against +any special libraries and there are no intermediate (magic) translation steps. + +On Linux, you can compile an example such as the one given in +:ref:`simple_example` using the following command: + +.. code-block:: bash + + $ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` + +The flags given here assume that you're using Python 3. For Python 2, just +change the executable appropriately (to ``python`` or ``python2``). + +The ``python3 -m pybind11 --includes`` command fetches the include paths for +both pybind11 and Python headers. This assumes that pybind11 has been installed +using ``pip`` or ``conda``. If it hasn't, you can also manually specify +``-I <path-to-pybind11>/include`` together with the Python includes path +``python3-config --includes``. + +Note that Python 2.7 modules don't use a special suffix, so you should simply +use ``example.so`` instead of ``example`python3-config --extension-suffix```. +Besides, the ``--extension-suffix`` option may or may not be available, depending +on the distribution; in the latter case, the module extension can be manually +set to ``.so``. + +On Mac OS: the build command is almost the same but it also requires passing +the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when +building the module: + +.. code-block:: bash + + $ c++ -O3 -Wall -shared -std=c++11 -undefined dynamic_lookup `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix` + +In general, it is advisable to include several additional build parameters +that can considerably reduce the size of the created binary. Refer to section +:ref:`cmake` for a detailed example of a suitable cross-platform CMake-based +build system that works on all platforms including Windows. + +.. note:: + + On Linux and macOS, it's better to (intentionally) not link against + ``libpython``. The symbols will be resolved when the extension library + is loaded into a Python binary. This is preferable because you might + have several different installations of a given Python version (e.g. the + system-provided Python, and one that ships with a piece of commercial + software). In this way, the plugin will work with both versions, instead + of possibly importing a second Python library into a process that already + contains one (which will lead to a segfault). + Generating binding code automatically ===================================== diff --git a/ext/pybind11/docs/conf.py b/ext/pybind11/docs/conf.py index 09604cfeb..cd0e17eb7 100644 --- a/ext/pybind11/docs/conf.py +++ b/ext/pybind11/docs/conf.py @@ -53,7 +53,7 @@ master_doc = 'index' # General information about the project. project = 'pybind11' -copyright = '2016, Wenzel Jakob' +copyright = '2017, Wenzel Jakob' author = 'Wenzel Jakob' # The version info for the project you're documenting, acts as replacement for @@ -61,9 +61,9 @@ author = 'Wenzel Jakob' # built documents. # # The short X.Y version. -version = '2.1' +version = '2.2' # The full version, including alpha/beta/rc tags. -release = '2.1.1' +release = '2.2.1' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. @@ -314,13 +314,13 @@ highlight_language = 'cpp' def generate_doxygen_xml(app): - build_dir = '.build' + build_dir = os.path.join(app.confdir, '.build') if not os.path.exists(build_dir): os.mkdir(build_dir) try: subprocess.call(['doxygen', '--version']) - retcode = subprocess.call(['doxygen']) + retcode = subprocess.call(['doxygen'], cwd=app.confdir) if retcode < 0: sys.stderr.write("doxygen error code: {}\n".format(-retcode)) except OSError as e: diff --git a/ext/pybind11/docs/faq.rst b/ext/pybind11/docs/faq.rst index 34002b42d..8f33eb014 100644 --- a/ext/pybind11/docs/faq.rst +++ b/ext/pybind11/docs/faq.rst @@ -4,30 +4,24 @@ Frequently asked questions "ImportError: dynamic module does not define init function" =========================================================== -1. Make sure that the name specified in ``pybind::module`` and - ``PYBIND11_PLUGIN`` is consistent and identical to the filename of the - extension library. The latter should not contain any extra prefixes (e.g. - ``test.so`` instead of ``libtest.so``). - -2. If the above did not fix your issue, then you are likely using an - incompatible version of Python (for instance, the extension library was - compiled against Python 2, while the interpreter is running on top of some - version of Python 3, or vice versa) +You are likely using an incompatible version of Python (for instance, the +extension library was compiled against Python 2, while the interpreter is +running on top of some version of Python 3, or vice versa). "Symbol not found: ``__Py_ZeroStruct`` / ``_PyInstanceMethod_Type``" ======================================================================== -See item 2 of the first answer. +See the first answer. "SystemError: dynamic module not initialized properly" ====================================================== -See item 2 of the first answer. +See the first answer. The Python interpreter immediately crashes when importing my module =================================================================== -See item 2 of the first answer. +See the first answer. CMake doesn't detect the right Python version ============================================= @@ -104,14 +98,10 @@ following example: void init_ex2(py::module &); /* ... */ - PYBIND11_PLUGIN(example) { - py::module m("example", "pybind example plugin"); - + PYBIND11_MODULE(example, m) { init_ex1(m); init_ex2(m); /* ... */ - - return m.ptr(); } :file:`ex1.cpp`: @@ -161,6 +151,33 @@ specifying a larger value, e.g. ``-ftemplate-depth=1024`` on GCC/Clang. The culprit is generally the generation of function signatures at compile time using C++14 template metaprogramming. +.. _`faq:hidden_visibility`: + +"‘SomeClass’ declared with greater visibility than the type of its field ‘SomeClass::member’ [-Wattributes]" +============================================================================================================ + +This error typically indicates that you are compiling without the required +``-fvisibility`` flag. pybind11 code internally forces hidden visibility on +all internal code, but if non-hidden (and thus *exported*) code attempts to +include a pybind type (for example, ``py::object`` or ``py::list``) you can run +into this warning. + +To avoid it, make sure you are specifying ``-fvisibility=hidden`` when +compiling pybind code. + +As to why ``-fvisibility=hidden`` is necessary, because pybind modules could +have been compiled under different versions of pybind itself, it is also +important that the symbols defined in one module do not clash with the +potentially-incompatible symbols defined in another. While Python extension +modules are usually loaded with localized symbols (under POSIX systems +typically using ``dlopen`` with the ``RTLD_LOCAL`` flag), this Python default +can be changed, but even if it isn't it is not always enough to guarantee +complete independence of the symbols involved when not using +``-fvisibility=hidden``. + +Additionally, ``-fvisiblity=hidden`` can deliver considerably binary size +savings. (See the following section for more details). + .. _`faq:symhidden`: @@ -202,11 +219,14 @@ world. So we'll generally only want to export symbols for those functions which are actually called from the outside. This can be achieved by specifying the parameter ``-fvisibility=hidden`` to GCC -and Clang, which sets the default symbol visibility to *hidden*. It's best to -do this only for release builds, since the symbol names can be helpful in -debugging sessions. On Visual Studio, symbols are already hidden by default, so -nothing needs to be done there. Needless to say, this has a tremendous impact -on the final binary size of the resulting extension library. +and Clang, which sets the default symbol visibility to *hidden*, which has a +tremendous impact on the final binary size of the resulting extension library. +(On Visual Studio, symbols are already hidden by default, so nothing needs to +be done there.) + +In addition to decreasing binary size, ``-fvisibility=hidden`` also avoids +potential serious issues when loading multiple modules and is required for +proper pybind operation. See the previous FAQ entry for more details. Another aspect that can require a fair bit of code are function signature descriptions. pybind11 automatically generates human-readable function diff --git a/ext/pybind11/docs/index.rst b/ext/pybind11/docs/index.rst index cedf65209..d236611b7 100644 --- a/ext/pybind11/docs/index.rst +++ b/ext/pybind11/docs/index.rst @@ -14,6 +14,7 @@ pybind11 --- Seamless operability between C++11 and Python intro changelog + upgrade .. toctree:: :caption: The Basics @@ -33,6 +34,7 @@ pybind11 --- Seamless operability between C++11 and Python advanced/smart_ptrs advanced/cast/index advanced/pycpp/index + advanced/embedding advanced/misc .. toctree:: diff --git a/ext/pybind11/docs/reference.rst b/ext/pybind11/docs/reference.rst index 3d211f7e9..e41141bd9 100644 --- a/ext/pybind11/docs/reference.rst +++ b/ext/pybind11/docs/reference.rst @@ -9,10 +9,12 @@ Reference ######### +.. _macros: + Macros ====== -.. doxygendefine:: PYBIND11_PLUGIN +.. doxygendefine:: PYBIND11_MODULE .. _core_types: @@ -58,6 +60,26 @@ Passing extra arguments to ``def`` or ``class_`` .. doxygengroup:: annotations :members: +Embedding the interpreter +========================= + +.. doxygendefine:: PYBIND11_EMBEDDED_MODULE + +.. doxygenfunction:: initialize_interpreter + +.. doxygenfunction:: finalize_interpreter + +.. doxygenclass:: scoped_interpreter + +Redirecting C++ streams +======================= + +.. doxygenclass:: scoped_ostream_redirect + +.. doxygenclass:: scoped_estream_redirect + +.. doxygenfunction:: add_ostream_redirect + Python build-in functions ========================= diff --git a/ext/pybind11/docs/release.rst b/ext/pybind11/docs/release.rst index 30d159a6f..b31bbe97e 100644 --- a/ext/pybind11/docs/release.rst +++ b/ext/pybind11/docs/release.rst @@ -2,7 +2,7 @@ To release a new version of pybind11: - Update the version number and push to pypi - Update ``pybind11/_version.py`` (set release version, remove 'dev'). - - Update ``PYBIND11_VERSION_MAJOR`` etc. in ``include/pybind11/common.h``. + - Update ``PYBIND11_VERSION_MAJOR`` etc. in ``include/pybind11/detail/common.h``. - Ensure that all the information in ``setup.py`` is up-to-date. - Update version in ``docs/conf.py``. - Tag release date in ``docs/changelog.rst``. @@ -14,8 +14,9 @@ To release a new version of pybind11: - ``python setup.py sdist upload``. - ``python setup.py bdist_wheel upload``. - Update conda-forge (https://github.com/conda-forge/pybind11-feedstock) via PR - - change version number in ``recipe/meta.yml`` - - update checksum to match the one computed by pypi + - download release package from Github: ``wget https://github.com/pybind/pybind11/archive/vX.Y.Z.tar.gz`` + - compute checksum: ``shasum -a 256 vX.Y.Z.tar.gz`` + - change version number and checksum in ``recipe/meta.yml`` - Get back to work - Update ``_version.py`` (add 'dev' and increment minor). - Update version in ``docs/conf.py`` diff --git a/ext/pybind11/docs/upgrade.rst b/ext/pybind11/docs/upgrade.rst new file mode 100644 index 000000000..3f5697391 --- /dev/null +++ b/ext/pybind11/docs/upgrade.rst @@ -0,0 +1,404 @@ +Upgrade guide +############# + +This is a companion guide to the :doc:`changelog`. While the changelog briefly +lists all of the new features, improvements and bug fixes, this upgrade guide +focuses only the subset which directly impacts your experience when upgrading +to a new version. But it goes into more detail. This includes things like +deprecated APIs and their replacements, build system changes, general code +modernization and other useful information. + + +v2.2 +==== + +Deprecation of the ``PYBIND11_PLUGIN`` macro +-------------------------------------------- + +``PYBIND11_MODULE`` is now the preferred way to create module entry points. +The old macro emits a compile-time deprecation warning. + +.. code-block:: cpp + + // old + PYBIND11_PLUGIN(example) { + py::module m("example", "documentation string"); + + m.def("add", [](int a, int b) { return a + b; }); + + return m.ptr(); + } + + // new + PYBIND11_MODULE(example, m) { + m.doc() = "documentation string"; // optional + + m.def("add", [](int a, int b) { return a + b; }); + } + + +New API for defining custom constructors and pickling functions +--------------------------------------------------------------- + +The old placement-new custom constructors have been deprecated. The new approach +uses ``py::init()`` and factory functions to greatly improve type safety. + +Placement-new can be called accidentally with an incompatible type (without any +compiler errors or warnings), or it can initialize the same object multiple times +if not careful with the Python-side ``__init__`` calls. The new-style custom +constructors prevent such mistakes. See :ref:`custom_constructors` for details. + +.. code-block:: cpp + + // old -- deprecated (runtime warning shown only in debug mode) + py::class<Foo>(m, "Foo") + .def("__init__", [](Foo &self, ...) { + new (&self) Foo(...); // uses placement-new + }); + + // new + py::class<Foo>(m, "Foo") + .def(py::init([](...) { // Note: no `self` argument + return new Foo(...); // return by raw pointer + // or: return std::make_unique<Foo>(...); // return by holder + // or: return Foo(...); // return by value (move constructor) + })); + +Mirroring the custom constructor changes, ``py::pickle()`` is now the preferred +way to get and set object state. See :ref:`pickling` for details. + +.. code-block:: cpp + + // old -- deprecated (runtime warning shown only in debug mode) + py::class<Foo>(m, "Foo") + ... + .def("__getstate__", [](const Foo &self) { + return py::make_tuple(self.value1(), self.value2(), ...); + }) + .def("__setstate__", [](Foo &self, py::tuple t) { + new (&self) Foo(t[0].cast<std::string>(), ...); + }); + + // new + py::class<Foo>(m, "Foo") + ... + .def(py::pickle( + [](const Foo &self) { // __getstate__ + return py::make_tuple(f.value1(), f.value2(), ...); // unchanged + }, + [](py::tuple t) { // __setstate__, note: no `self` argument + return new Foo(t[0].cast<std::string>(), ...); + // or: return std::make_unique<Foo>(...); // return by holder + // or: return Foo(...); // return by value (move constructor) + } + )); + +For both the constructors and pickling, warnings are shown at module +initialization time (on import, not when the functions are called). +They're only visible when compiled in debug mode. Sample warning: + +.. code-block:: none + + pybind11-bound class 'mymodule.Foo' is using an old-style placement-new '__init__' + which has been deprecated. See the upgrade guide in pybind11's docs. + + +Stricter enforcement of hidden symbol visibility for pybind11 modules +--------------------------------------------------------------------- + +pybind11 now tries to actively enforce hidden symbol visibility for modules. +If you're using either one of pybind11's :doc:`CMake or Python build systems +<compiling>` (the two example repositories) and you haven't been exporting any +symbols, there's nothing to be concerned about. All the changes have been done +transparently in the background. If you were building manually or relied on +specific default visibility, read on. + +Setting default symbol visibility to *hidden* has always been recommended for +pybind11 (see :ref:`faq:symhidden`). On Linux and macOS, hidden symbol +visibility (in conjunction with the ``strip`` utility) yields much smaller +module binaries. `CPython's extension docs`_ also recommend hiding symbols +by default, with the goal of avoiding symbol name clashes between modules. +Starting with v2.2, pybind11 enforces this more strictly: (1) by declaring +all symbols inside the ``pybind11`` namespace as hidden and (2) by including +the ``-fvisibility=hidden`` flag on Linux and macOS (only for extension +modules, not for embedding the interpreter). + +.. _CPython's extension docs: https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module + +The namespace-scope hidden visibility is done automatically in pybind11's +headers and it's generally transparent to users. It ensures that: + +* Modules compiled with different pybind11 versions don't clash with each other. + +* Some new features, like ``py::module_local`` bindings, can work as intended. + +The ``-fvisibility=hidden`` flag applies the same visibility to user bindings +outside of the ``pybind11`` namespace. It's now set automatic by pybind11's +CMake and Python build systems, but this needs to be done manually by users +of other build systems. Adding this flag: + +* Minimizes the chances of symbol conflicts between modules. E.g. if two + unrelated modules were statically linked to different (ABI-incompatible) + versions of the same third-party library, a symbol clash would be likely + (and would end with unpredictable results). + +* Produces smaller binaries on Linux and macOS, as pointed out previously. + +Within pybind11's CMake build system, ``pybind11_add_module`` has always been +setting the ``-fvisibility=hidden`` flag in release mode. From now on, it's +being applied unconditionally, even in debug mode and it can no longer be opted +out of with the ``NO_EXTRAS`` option. The ``pybind11::module`` target now also +adds this flag to it's interface. The ``pybind11::embed`` target is unchanged. + +The most significant change here is for the ``pybind11::module`` target. If you +were previously relying on default visibility, i.e. if your Python module was +doubling as a shared library with dependents, you'll need to either export +symbols manually (recommended for cross-platform libraries) or factor out the +shared library (and have the Python module link to it like the other +dependents). As a temporary workaround, you can also restore default visibility +using the CMake code below, but this is not recommended in the long run: + +.. code-block:: cmake + + target_link_libraries(mymodule PRIVATE pybind11::module) + + add_library(restore_default_visibility INTERFACE) + target_compile_options(restore_default_visibility INTERFACE -fvisibility=default) + target_link_libraries(mymodule PRIVATE restore_default_visibility) + + +Local STL container bindings +---------------------------- + +Previous pybind11 versions could only bind types globally -- all pybind11 +modules, even unrelated ones, would have access to the same exported types. +However, this would also result in a conflict if two modules exported the +same C++ type, which is especially problematic for very common types, e.g. +``std::vector<int>``. :ref:`module_local` were added to resolve this (see +that section for a complete usage guide). + +``py::class_`` still defaults to global bindings (because these types are +usually unique across modules), however in order to avoid clashes of opaque +types, ``py::bind_vector`` and ``py::bind_map`` will now bind STL containers +as ``py::module_local`` if their elements are: builtins (``int``, ``float``, +etc.), not bound using ``py::class_``, or bound as ``py::module_local``. For +example, this change allows multiple modules to bind ``std::vector<int>`` +without causing conflicts. See :ref:`stl_bind` for more details. + +When upgrading to this version, if you have multiple modules which depend on +a single global binding of an STL container, note that all modules can still +accept foreign ``py::module_local`` types in the direction of Python-to-C++. +The locality only affects the C++-to-Python direction. If this is needed in +multiple modules, you'll need to either: + +* Add a copy of the same STL binding to all of the modules which need it. + +* Restore the global status of that single binding by marking it + ``py::module_local(false)``. + +The latter is an easy workaround, but in the long run it would be best to +localize all common type bindings in order to avoid conflicts with +third-party modules. + + +Negative strides for Python buffer objects and numpy arrays +----------------------------------------------------------- + +Support for negative strides required changing the integer type from unsigned +to signed in the interfaces of ``py::buffer_info`` and ``py::array``. If you +have compiler warnings enabled, you may notice some new conversion warnings +after upgrading. These can be resolved using ``static_cast``. + + +Deprecation of some ``py::object`` APIs +--------------------------------------- + +To compare ``py::object`` instances by pointer, you should now use +``obj1.is(obj2)`` which is equivalent to ``obj1 is obj2`` in Python. +Previously, pybind11 used ``operator==`` for this (``obj1 == obj2``), but +that could be confusing and is now deprecated (so that it can eventually +be replaced with proper rich object comparison in a future release). + +For classes which inherit from ``py::object``, ``borrowed`` and ``stolen`` +were previously available as protected constructor tags. Now the types +should be used directly instead: ``borrowed_t{}`` and ``stolen_t{}`` +(`#771 <https://github.com/pybind/pybind11/pull/771>`_). + + +Stricter compile-time error checking +------------------------------------ + +Some error checks have been moved from run time to compile time. Notably, +automatic conversion of ``std::shared_ptr<T>`` is not possible when ``T`` is +not directly registered with ``py::class_<T>`` (e.g. ``std::shared_ptr<int>`` +or ``std::shared_ptr<std::vector<T>>`` are not automatically convertible). +Attempting to bind a function with such arguments now results in a compile-time +error instead of waiting to fail at run time. + +``py::init<...>()`` constructor definitions are also stricter and now prevent +bindings which could cause unexpected behavior: + +.. code-block:: cpp + + struct Example { + Example(int &); + }; + + py::class_<Example>(m, "Example") + .def(py::init<int &>()); // OK, exact match + // .def(py::init<int>()); // compile-time error, mismatch + +A non-``const`` lvalue reference is not allowed to bind to an rvalue. However, +note that a constructor taking ``const T &`` can still be registered using +``py::init<T>()`` because a ``const`` lvalue reference can bind to an rvalue. + +v2.1 +==== + +Minimum compiler versions are enforced at compile time +------------------------------------------------------ + +The minimums also apply to v2.0 but the check is now explicit and a compile-time +error is raised if the compiler does not meet the requirements: + +* GCC >= 4.8 +* clang >= 3.3 (appleclang >= 5.0) +* MSVC >= 2015u3 +* Intel C++ >= 15.0 + + +The ``py::metaclass`` attribute is not required for static properties +--------------------------------------------------------------------- + +Binding classes with static properties is now possible by default. The +zero-parameter version of ``py::metaclass()`` is deprecated. However, a new +one-parameter ``py::metaclass(python_type)`` version was added for rare +cases when a custom metaclass is needed to override pybind11's default. + +.. code-block:: cpp + + // old -- emits a deprecation warning + py::class_<Foo>(m, "Foo", py::metaclass()) + .def_property_readonly_static("foo", ...); + + // new -- static properties work without the attribute + py::class_<Foo>(m, "Foo") + .def_property_readonly_static("foo", ...); + + // new -- advanced feature, override pybind11's default metaclass + py::class_<Bar>(m, "Bar", py::metaclass(custom_python_type)) + ... + + +v2.0 +==== + +Breaking changes in ``py::class_`` +---------------------------------- + +These changes were necessary to make type definitions in pybind11 +future-proof, to support PyPy via its ``cpyext`` mechanism (`#527 +<https://github.com/pybind/pybind11/pull/527>`_), and to improve efficiency +(`rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_). + +1. Declarations of types that provide access via the buffer protocol must + now include the ``py::buffer_protocol()`` annotation as an argument to + the ``py::class_`` constructor. + + .. code-block:: cpp + + py::class_<Matrix>("Matrix", py::buffer_protocol()) + .def(py::init<...>()) + .def_buffer(...); + +2. Classes which include static properties (e.g. ``def_readwrite_static()``) + must now include the ``py::metaclass()`` attribute. Note: this requirement + has since been removed in v2.1. If you're upgrading from 1.x, it's + recommended to skip directly to v2.1 or newer. + +3. This version of pybind11 uses a redesigned mechanism for instantiating + trampoline classes that are used to override virtual methods from within + Python. This led to the following user-visible syntax change: + + .. code-block:: cpp + + // old v1.x syntax + py::class_<TrampolineClass>("MyClass") + .alias<MyClass>() + ... + + // new v2.x syntax + py::class_<MyClass, TrampolineClass>("MyClass") + ... + + Importantly, both the original and the trampoline class are now specified + as arguments to the ``py::class_`` template, and the ``alias<..>()`` call + is gone. The new scheme has zero overhead in cases when Python doesn't + override any functions of the underlying C++ class. + `rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_. + + The class type must be the first template argument given to ``py::class_`` + while the trampoline can be mixed in arbitrary order with other arguments + (see the following section). + + +Deprecation of the ``py::base<T>()`` attribute +---------------------------------------------- + +``py::base<T>()`` was deprecated in favor of specifying ``T`` as a template +argument to ``py::class_``. This new syntax also supports multiple inheritance. +Note that, while the type being exported must be the first argument in the +``py::class_<Class, ...>`` template, the order of the following types (bases, +holder and/or trampoline) is not important. + +.. code-block:: cpp + + // old v1.x + py::class_<Derived>("Derived", py::base<Base>()); + + // new v2.x + py::class_<Derived, Base>("Derived"); + + // new -- multiple inheritance + py::class_<Derived, Base1, Base2>("Derived"); + + // new -- apart from `Derived` the argument order can be arbitrary + py::class_<Derived, Base1, Holder, Base2, Trampoline>("Derived"); + + +Out-of-the-box support for ``std::shared_ptr`` +---------------------------------------------- + +The relevant type caster is now built in, so it's no longer necessary to +include a declaration of the form: + +.. code-block:: cpp + + PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>) + +Continuing to do so won’t cause an error or even a deprecation warning, +but it's completely redundant. + + +Deprecation of a few ``py::object`` APIs +---------------------------------------- + +All of the old-style calls emit deprecation warnings. + ++---------------------------------------+---------------------------------------------+ +| Old syntax | New syntax | ++=======================================+=============================================+ +| ``obj.call(args...)`` | ``obj(args...)`` | ++---------------------------------------+---------------------------------------------+ +| ``obj.str()`` | ``py::str(obj)`` | ++---------------------------------------+---------------------------------------------+ +| ``auto l = py::list(obj); l.check()`` | ``py::isinstance<py::list>(obj)`` | ++---------------------------------------+---------------------------------------------+ +| ``py::object(ptr, true)`` | ``py::reinterpret_borrow<py::object>(ptr)`` | ++---------------------------------------+---------------------------------------------+ +| ``py::object(ptr, false)`` | ``py::reinterpret_steal<py::object>(ptr)`` | ++---------------------------------------+---------------------------------------------+ +| ``if (obj.attr("foo"))`` | ``if (py::hasattr(obj, "foo"))`` | ++---------------------------------------+---------------------------------------------+ +| ``if (obj["bar"])`` | ``if (obj.contains("bar"))`` | ++---------------------------------------+---------------------------------------------+ |