CMake and distutils

Imagine you are writing a nice C library, together with some nice Python bindings and build this with CMake. Now, you would also like to deploy pure Python packages that use the nice bindings. You could just use the distutils module, hack a setup.py together and let the poor soul using your software figure out how to install that. However, there are some good reasons to let CMake build and install the package:

  • CMake knows if a Python interpreter is installed.
  • The setup.py script can be configured with central information like version numbers.
  • Each time the library changes, your package can be run and tested against it.
  • The poor soul can type sudo make install and forget everything else.

To avoid any misunderstandings, let’s assume the following project layout

.
|-- CMakeLists.txt
|-- src
|   |-- CMakeLists.txt
|   |-- foo.c
|   `-- foo.h
`-- python
    |-- CMakeLists.txt
    |-- foo
    |   `-- __init__.py
    `-- setup.py.in

and that ${PACKAGE_VERSION} is already defined in the top-level CMakeLists.txt. setup.py.in contains the usual mumbo jumbo spiced with variables that are replaced during the configure step:

from distutils.core import setup

setup(name='foo',
      version='${PACKAGE_VERSION}',
      package_dir={ '': '${CMAKE_CURRENT_SOURCE_DIR}' },
      packages=['foo'])

Because, CMake can build from any directory, setup.py will be generated nowhere near the package root directory. For this, we set the general package root dir via the package_dir parameter to ${CMAKE_CURRENT_SOURCE_DIR} (which is python/ in our example).

To build and install the Python package with CMake, python/CMakeLists.txt looks something like this:

find_program(PYTHON "python")

if (PYTHON)
    set(SETUP_PY_IN "${CMAKE_CURRENT_SOURCE_DIR}/setup.py.in")
    set(SETUP_PY    "${CMAKE_CURRENT_BINARY_DIR}/setup.py")
    set(DEPS        "${CMAKE_CURRENT_SOURCE_DIR}/module/__init__.py")
    set(OUTPUT      "${CMAKE_CURRENT_BINARY_DIR}/build/timestamp")

    configure_file(${SETUP_PY_IN} ${SETUP_PY})

    add_custom_command(OUTPUT ${OUTPUT}
                       COMMAND ${PYTHON} ${SETUP_PY} build
                       COMMAND ${CMAKE_COMMAND} -E touch ${OUTPUT}
                       DEPENDS ${DEPS})

    add_custom_target(target ALL DEPENDS ${OUTPUT})

    install(CODE "execute_process(COMMAND ${PYTHON} ${SETUP_PY} install)")
endif()

What’s so special about that? Well, the target that we add to the build system needs a way to tell when it’s time to run. For this, the add_custom_command specifies one or more outputs on which the target depends. But the output from python setup.py build is not a good candidate because it is placed in architecture- and platform-specific directories like lib.linux-x86_64-2.6. And you don’t want to test on that in a sane and portable way.

What we can do is to generate a file in the build root, each time our source dependencies change. The simplest way to do this, is to touch a stamp file and thus update its modification time. The dependency graph then looks like this:

{% ditaa %} +––––+ depends +—––+ depends +––––+ | Target |———>| Stamp |———>| Source | +––––+ +—––+ +––––+ {% endditaa %}

Last but not least, we install the package with the CODE form of the install command. install(CODE) is tied to the install target and expects a CMake command. For our purposes this is execute_process that in turn runs python setup.py install.