summaryrefslogtreecommitdiff
path: root/ext/pybind11/docs/benchmark.rst
diff options
context:
space:
mode:
authorAndreas Sandberg <andreas.sandberg@arm.com>2017-02-27 13:17:51 +0000
committerAndreas Sandberg <andreas.sandberg@arm.com>2017-05-02 12:37:32 +0000
commitc79706ff4ce591df2151db5504d3c224f3c9965f (patch)
treeb56cd2bfe704a40575a71075e78194a4c516c98d /ext/pybind11/docs/benchmark.rst
parent359cb08623324b62d7c34973ae54d5bc7f23f9fd (diff)
downloadgem5-c79706ff4ce591df2151db5504d3c224f3c9965f.tar.xz
ext: Add pybind rev f4b81b3
Change-Id: I52e4fc9ebf2f59da57d8cf8f3e37cc79598c2f5f Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Andreas Hansson <andreas.hansson@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com> Reviewed-on: https://gem5-review.googlesource.com/2229 Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com> Reviewed-by: Jason Lowe-Power <jason@lowepower.com> Reviewed-by: Pierre-Yves PĂ©neau <pierre-yves.peneau@lirmm.fr>
Diffstat (limited to 'ext/pybind11/docs/benchmark.rst')
-rw-r--r--ext/pybind11/docs/benchmark.rst99
1 files changed, 99 insertions, 0 deletions
diff --git a/ext/pybind11/docs/benchmark.rst b/ext/pybind11/docs/benchmark.rst
new file mode 100644
index 000000000..8babaa319
--- /dev/null
+++ b/ext/pybind11/docs/benchmark.rst
@@ -0,0 +1,99 @@
+Benchmark
+=========
+
+The following is the result of a synthetic benchmark comparing both compilation
+time and module size of pybind11 against Boost.Python. A detailed report about a
+Boost.Python to pybind11 conversion of a real project is available here: [#f1]_.
+
+.. [#f1] http://graylab.jhu.edu/RosettaCon2016/PyRosetta-4.pdf
+
+Setup
+-----
+
+A python script (see the ``docs/benchmark.py`` file) was used to generate a set
+of files with dummy classes whose count increases for each successive benchmark
+(between 1 and 2048 classes in powers of two). Each class has four methods with
+a randomly generated signature with a return value and four arguments. (There
+was no particular reason for this setup other than the desire to generate many
+unique function signatures whose count could be controlled in a simple way.)
+
+Here is an example of the binding code for one class:
+
+.. code-block:: cpp
+
+ ...
+ class cl034 {
+ public:
+ cl279 *fn_000(cl084 *, cl057 *, cl065 *, cl042 *);
+ cl025 *fn_001(cl098 *, cl262 *, cl414 *, cl121 *);
+ cl085 *fn_002(cl445 *, cl297 *, cl145 *, cl421 *);
+ cl470 *fn_003(cl200 *, cl323 *, cl332 *, cl492 *);
+ };
+ ...
+
+ PYBIND11_PLUGIN(example) {
+ py::module m("example");
+ ...
+ py::class_<cl034>(m, "cl034")
+ .def("fn_000", &cl034::fn_000)
+ .def("fn_001", &cl034::fn_001)
+ .def("fn_002", &cl034::fn_002)
+ .def("fn_003", &cl034::fn_003)
+ ...
+ return m.ptr();
+ }
+
+The Boost.Python version looks almost identical except that a return value
+policy had to be specified as an argument to ``def()``. For both libraries,
+compilation was done with
+
+.. code-block:: bash
+
+ Apple LLVM version 7.0.2 (clang-700.1.81)
+
+and the following compilation flags
+
+.. code-block:: bash
+
+ g++ -Os -shared -rdynamic -undefined dynamic_lookup -fvisibility=hidden -std=c++14
+
+Compilation time
+----------------
+
+The following log-log plot shows how the compilation time grows for an
+increasing number of class and function declarations. pybind11 includes many
+fewer headers, which initially leads to shorter compilation times, but the
+performance is ultimately fairly similar (pybind11 is 19.8 seconds faster for
+the largest largest file with 2048 classes and a total of 8192 methods -- a
+modest **1.2x** speedup relative to Boost.Python, which required 116.35
+seconds).
+
+.. only:: not latex
+
+ .. image:: pybind11_vs_boost_python1.svg
+
+.. only:: latex
+
+ .. image:: pybind11_vs_boost_python1.png
+
+Module size
+-----------
+
+Differences between the two libraries become much more pronounced when
+considering the file size of the generated Python plugin: for the largest file,
+the binary generated by Boost.Python required 16.8 MiB, which was **2.17
+times** / **9.1 megabytes** larger than the output generated by pybind11. For
+very small inputs, Boost.Python has an edge in the plot below -- however, note
+that it stores many definitions in an external library, whose size was not
+included here, hence the comparison is slightly shifted in Boost.Python's
+favor.
+
+.. only:: not latex
+
+ .. image:: pybind11_vs_boost_python2.svg
+
+.. only:: latex
+
+ .. image:: pybind11_vs_boost_python2.png
+
+