It's been a while since I wrote this and Poetry has become my Python package manager of choice due to pip missing several features I need.
I've added a section below on how the build process can be integegrated into a the Poetry project structure.
PyBind11 is a fantastic header only library that allows you to integrate C++11 into Python. For example, lets say you have the following C++ source file:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 | #include <iostream>
#include <vector>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
namespace py = pybind11;
std::vector<float> my_wrapper_function(std::vector<float> my_vector) {
for (int i = 0; i < my_vector; ++i) {
std::cout << my_vector[i] << ' ' << std::endl;
}
return my_vector;
}
PYBIND11_MODULE(my_wrapper, m) {
m.def("my_wrapper_function", &my_wrapper_function);
}
|
You can can then compile it with the following flags:
| > c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` my_wrapper.cpp -o example`python3-config --extension-suffix`
|
This produces a my_wrapper.cpython-36m-x86_64-linux-gnu.so, shared object
file, that you can import from a Python shell or script.
| > python3.6
Python 3.6.3 (default, Oct 3 2017, 21:45:48)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import my_wrapper
>>> result = my_wrapper.my_wrapper_function([0, 1, 2, 3, 4])
0 1 2 3 4
>>> result
[0, 1, 2, 3, 4]
|
When I began using PyBind11, I initially struggled to find a good way to import the shared object. One solution was to place make the wrapper it's own package and install it to the global or local site-packages.
However, in most cases I wanted to keep my project self-contained and have the wrapper in the same package as the rest of my sub-modules. The following is my solution to not using weird path manipulating import magic.
Lets say we have the following package layout:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 | my_project_package/
my_python_module/
__init__.py
my_module_script_0.py
my_module_script_1.py
my_python_submodule/
__init__.py
my_submodule_script_0.py
my_submodule_script_1.py
my_wrapper/
__init__.py
my_wrapper_source.cpp
tests/
test_0.py
test_1.py
images/
image_0.png
setup.py
|
To build the Python/C++ binding we can make our setup.py reflect the following:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64 | #!/usr/bin/env python3
from setuptools import setup, Extension, find_packages
import sysconfig
import glob
import os
import shutil
OUTPUT_FILE_NAME = 'my_wrapper'
PATH_TEMPLATE = '{}/{}{}'
SUFFIX = sysconfig.get_config_var('EXT_SUFFIX')
class get_pybind_include(object):
"""Helper class to determine the pybind11 include path
The purpose of this class is to postpone importing pybind11
until it is actually installed, so that the ``get_include()``
method can be invoked. """
def __init__(self, user=False):
self.user = user
def __str__(self):
import pybind11
return pybind11.get_include(self.user)
my_wrapper_extension = Extension(
OUTPUT_FILE_NAME,
sources=['my_wrapper/my_wrapper_source.cpp'],
include_dirs=[
get_pybind_include(),
get_pybind_include(user=True)
],
libraries=[],
extra_compile_args=['-std=c++11'],
language='c++'
)
setup(
name='My Package',
ext_modules=[my_wrapper_extension],
packages=find_packages(),
install_requires=[
'pybind11'
]
)
def move_to_submodule():
if os.path.isdir('build'):
build_directory = glob.glob('build/lib*')[0]
build_path = PATH_TEMPLATE.format(build_directory, OUTPUT_FILE_NAME, SUFFIX)
if os.path.isfile(build_path):
shutil.copy(build_path, 'my_python_module/my_wrapper/')
else:
print('Could not find shared object.')
move_to_submodule()
|
Without the move_to_submodule() method when you run the command python setup.py build, your wrapper will be placed in a directory called build in the top level of your package. So in my example, at this path: my_project_package/build.
This makes importing the shared object from within your package difficult because it is above the top level module. My solution to this is to copy the shared object to the same directory as the source and to make the wrapper a module by creating an __init__.py file. Both of the actions are reflected in the above directory listing and by the contents of setup.py.
The pyproject.toml file:
1
2
3
4
5
6
7
8
9
10
11
12
13
14 | [tool.poetry]
name = "my_python_module"
version = "0.1.0"
description = ""
authors = ["drew_gunderson <drewgun@icloud.com>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.11"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
|
Public Key 
DockerHub