Bundling C++ extension headers with a Python package source distribution
<p>I'm writing a Cython wrapper to a C++ library that I would like to distribute as a Python package. I've come up with a dummy version of my package that looks like this (full source <a href="https://github.com/standage/packagetest" rel="nofollow">here</a>).</p>

<pre><code>$ tree
.
âââ bogus.pyx
âââ inc
âÂ Â  âââ bogus.hpp
âââ setup.py
âââ src
    âââ bogus.cpp
$
$ cat inc/bogus.hpp 
#ifndef BOGUS
#define BOGUS

class bogus
{
protected:
    int data;

public:
    bogus();
    int get_double(int value);
};

#endif
$
$ cat src/bogus.cpp 
#include "bogus.hpp"

bogus::bogus() : data(0)
{

}

int bogus::get_double(int value)
{
    data = value * 2;
    return data;
}
$ cat bogus.pyx 
# distutils: language = c++
# distutils: sources = src/bogus.cpp
# cython: c_string_type=str, c_string_encoding=ascii

cdef extern from 'bogus.hpp':
    cdef cppclass bogus:
        bogus() except +
        int get_double(int value)

cdef class Bogus:
    cdef bogus b
    def get_double(self, int value):
        return self.b.get_double(value)
</code></pre>

<p>With the following <code>setup.py</code> file, I can can confirm that the library installs correctly with <code>python setup.py install</code> and that it works correctly.</p>

<pre><code>from setuptools import setup, Extension
import glob

headers = list(glob.glob('inc/*.hpp'))

bogus = Extension(
    'bogus',
    sources=['bogus.pyx', 'src/bogus.cpp'],
    include_dirs=['inc/'],
    language='c++',
    extra_compile_args=['--std=c++11', '-Wno-unused-function'],
    extra_link_args=['--std=c++11'],
)

setup(
    name='bogus',
    description='Troubleshooting Python packaging and distribution',
    author='Daniel Standage',
    ext_modules=[bogus],
    install_requires=['cython'],
    version='0.1.0'
)
</code></pre>

<p>However, when I build a source distribution using <code>python setup.py sdist build</code>, the C++ header files are not included and the C++ extension cannot be compiled.</p>

<p><strong>How can I make sure the C++ header files get bundled with the source distribution?!?!</strong></p>

<p>&lt;rant&gt;</p>

<p>Troubleshooting this has uncovered a tremendously convoluted and inconsistent mess of documentation, suggestions, and hacks, none of which have worked for me. Put a <code>graft</code> line in <code>MANIFEST.in</code>? Nope. The <code>package_data</code> or <code>data_files</code> options? Nope. Python packaging seems to have improved a lot in the last few years, but it is still nigh impenetrable for those of us that don't live and breathe Python packaging!</p>

<p>&lt;/rant&gt;</p>

<h2>Short answer</h2>

<p>Put <code>include inc/*.hpp</code> in the <code>MANIFEST.in</code> file.</p>

<h2>Long answer</h2>

<p>Based on various blog posts and SO threads, I had tried the suggestion of declaring the files in a <code>MANIFEST.in</code> file. Following <a href="https://docs.python.org/2/distutils/sourcedist.html" rel="nofollow">these instructions</a>, I added a <code>graft inc/</code> line to <code>MANIFEST.in</code> to include the entire directory. This did not work.</p>

<p>However, replacing this line with <code>include inc/*.hpp</code> did work. Arguably this should have been the first thing I tried, but being unfamiliar with the intricacies and warts of setuptools and distutils, I had no reason to expect that <code>graft</code> wouldn't work.</p>

python packaging python-extensions python-packaging