The smallest literate Python implementation

Literate programming is a programming paradigm that intermingles natural language comments with short snippets of source code from which then both compiled code and documentation can be generated. Knuth’s TeX is probably the most famous project written in a literate programming style.

Although, I am not a big fan of literate programming myself, I appreciate the general idea of interweaving natural language with code snippets. For example, I am always intrigued by explorative documents for research, in which you develop an idea in words and code. IPython notebooks come pretty close, but they are not what I would call plain text.

I was then looking for alternatives that mix and match Markdown with Python code and output static Markdown or HTML. I found Stijn Debrouwere’s python-literate, but it is too focused too much on the literate programming part and tries to invent basically anything from scratch.

So, I sat down and wrote a tiny script called pyl that does exactly one job, execute Python code and output results. It uses Pandoc to parse Markdown1 into Pandoc’s JSON syntax tree representation, traverses the tree, executes all code blocks within Python and replaces inline code with defined variables. It then outputs the altered syntax tree from which a second Pandoc instance can generate the final output in any desired target format.

With that 30-odd line long Python script, you can turn

Here is some Python code

    x = 2 * 3 + 4
    z = 'A fine string'

Now, is `x` = 10? and `z` a fine string?

into this

Here is some Python code

    x = 2 * 3 + 4
    z = 'A fine string'

Now, is 10 = 10? and A fine string a fine string?

Because there are no restrictions, you can do all kinds of crazy and potentially dangerous things. Just have a look at the example gallery.

Again, one word of warning: pyl executes arbitrary code in a non-sandboxed Python environment. This gives a lot of opportunities but can be a major security problem if building documents from untrusted sources.

  1. Actually anything that Pandoc can parse should work with pyl, but I haven’t checked that yet.