Literate programming is a programming paradigm that intermingles natural language comments with short snippets of source code from which then both compiled code and documentation can be generated. Knuth’s TeX is probably the most famous project written in a literate programming style.

Although, I am not a big fan of literate programming myself, I appreciate the general idea of interweaving natural language with code snippets. For example, I am always intrigued by explorative documents for research, in which you develop an idea in words and code. IPython notebooks come pretty close, but they are not what I would call plain text.

I was then looking for alternatives that mix and match Markdown with Python code and output static Markdown or HTML. I found Stijn Debrouwere’s python-literate, but it is too focused too much on the literate programming part and tries to invent basically anything from scratch.

So, I sat down and wrote a tiny script called pyl that does exactly one job, execute Python code and output results. It uses Pandoc to parse Markdown¹ into Pandoc’s JSON syntax tree representation, traverses the tree, executes all code blocks within Python and replaces inline code with defined variables. It then outputs the altered syntax tree from which a second Pandoc instance can generate the final output in any desired target format.

With that 30-odd line long Python script, you can turn

Here is some Python code

    x = 2 * 3 + 4
    z = 'A fine string'

Now, is `x` = 10? and `z` a fine string?

into this

Here is some Python code

    x = 2 * 3 + 4
    z = 'A fine string'

Now, is 10 = 10? and A fine string a fine string?

Because there are no restrictions, you can do all kinds of crazy and potentially dangerous things. Just have a look at the example gallery.

Again, one word of warning: pyl executes arbitrary code in a non-sandboxed Python environment. This gives a lot of opportunities but can be a major security problem if building documents from untrusted sources.

As a follow-up to my cherry pick UI post, here’s a quick tutorial how you can extend Git with new sub-commands. If you used the cherry pick tool, you noticed that it is not very well integrated and you needed to remember the exact name of the script. I suggested to add an alias to your .gitconfig but there is more “natural” way to do it.

Luckily¹, Git uses any program prefixed with git- and located in $PATH as a sub-command. That means, moving the pick-from to /usr/local/bin/git-pick, allows us to use it with

$ git pick 

Calling git pick --help spawns man for a git-pick.7 manpage. Hence, you could for example generate one from Markdown with Pandoc and copy that into your local man search directory ~/.local/share/man/man7.

Git’s sub-command completion is implemented in a similar pragmatic way: simply provide a bash function _git_command_name() that completes to whatever you like. Git already provides helper functions for typical situations, which is why we can complete branches for git pick simply with

function _git_pick () {
  __gitcomp_nl "$(__git_refs)"
}

Compiling software in a restricted environment requires changing essential environment variables and finding a place where to install the final product. This is a tedious task that can be mitigated by a simple shell script that sets up the appropriate directory and overrides the path variables:

$ cenv foo
$ (foo) ./configure --prefix=$CENV && make && make install

Because this script only knows about a limited set of environment variables, you can add more by running

$ cenv --edit foo

This will open a local file that is sourced right before starting the new subshell and in which you can define more variables to be exported.

To make it work even more like a Python virtualenv, you could alias common build tools in your .bashrc to include the prefix implicitly:

alias cmake="cmake -DCMAKE_INSTALL_PREFIX=$CENV"
alias configure="configure --prefix=$CENV"

This little script works for my use case but there are of course alternatives: lightweight containers (either raw or using Docker) or virtual machines being some of them.

I had to give many talks, both to internal and external audiences. As you could imagine, I use LaTeX and Beamer to compile my slides. But until now, I had no good approach how to organize them. This lead to a chaos of copy-pasted Makefiles, assets such as images and TikZ sources, TeX preludes etc. Everything was scattered around several Git repositories, some talks have their own repositories, some were collected in a single one. That was quite a mess, to be honest.

So, I sat down and thought how to lay out things. Eventually, I set up one repository for all Beamer sources which are simply structured in a year/title hierarchy. I set up one repository for all assets and one repository for my BibTeX databases. At the moment I don’t have a good reason to split them that way other than that it doesn’t feel “semantically” right. To avoid repeating myself in the prelude, I pulled out the most common settings – including packages, fonts, colors, environments and commands – into its own top-level beamercommon.sty.

Each talk subdirectory as its own Makefile that consists of one of:

include ../../common.mk
include ../../common.pandoc.mk

The first Makefile is based on rubber and simply tries to compile all .tex files found in the current directory. The second Makefile uses Pandoc and a slightly modified template to build Beamer slides from Markdown input:

SRC=$(wildcard *.md)
PDF=$(subst .md,.pdf,$(SRC))
TEMPLATE=../../pandoc-template.tex
THEME_OPTIONS="english, titlepage0"
OPTS=-t beamer --latex-engine=xelatex --template=$(TEMPLATE) \
	 --variable theme=KIT \
	 --variable sansfont="Source Sans Pro" \
	 --variable monofont="Inconsolata" \
	 --variable fontsize=18pt \
	 --variable graphicspath=../../common/images/ \
	 --variable themeoptions=$(THEME_OPTIONS) \

.PHONY: clean

all: $(PDF)

%.pdf: %.md $(TEMPLATE)
	@pandoc $(OPTS) -o $@ $<

clean:
	@rm -f $(PDF)

In the talk-specific Makefile, I can then simply add or override the $(OPTS) variable to customize the outcome.

There was one thing, I wasn’t sure about: keeping the common and BibTeX repos outside, inside as a submodule or inside as a subtree. I decided for the submodule approach, although I know, it could hurt me in the long run. In any case, this is what the final, version-controlled directory structure looks like: