Bloerg
       

Disclaimer: The main idea of this post might only be interesting for ten other people in the entire world but the general notion of keeping a double-entry booking journal might also appeal to you. Note though that I am not a finance expert, so your mileage may vary.

Double-entry bookkeeping with ledger

For a very long time, I have not taken particular interest in my finances and my main strategy has been to spend less than I earned. This has worked quite well, however, with regular income and a large variety of expenses such as insurances, retirement funds, daily things etc., I got overwhelmed a bit and felt losing control. To get rid of this uneasy feeling, I had to record whatever I am gaining or spending. As a layperson one has two options to record this: keep running totals in a spreadsheet – also known as single-entry bookkeeping – or use a double-entry bookkeeping software to model the flow of money between different accounts representing assets, liabilities and equities. Although the former is probably easier to grasp at the beginning, you will run into limitations such as difficult error detection soon enough. As a CLI person, I was thus soon attracted to the ledger-like family of double-entry bookkeeping systems.

The idea of ledger is simple: you record transactions between accounts in a simple journal text file and then let ledger report balances. Unlike GUI programs such as GnuCash, ledger never touches your data. This separation of concerns eases editing (e.g. Vim), revision control (e.g. Git) and greatly improves interoperability. The latter is also the main reason for the many different ledger clones of which I want to highlight hledger. Contrary to ledger that is written in C++, hledger is based on Haskell and is in my experience a lot more stable. To get more information about accounting with ledger-likes, please refer to the very extensive documentation of both ledger and hledger.

Data import with hledger

Now, here’s the actual piece covering the title of this post. I have a checking account with Deutsche Bank1 that provides export of all transactions in CSV format. As with probably all corporate data formats, the CSV transaction format has many quirks and pecularities that need to be fixed before appending everything to a ledger journal. I used to use a homegrown Python script to fix up the CSV format and output ledger format, however hledger provides a fantastic csv command that already does the last step perfectly fine. So if you are in the same boat and want to import Deutsche Bank data into your ledger journal here is what you need to do:

tail -n +6 $1 | head -n -1 | \
iconv -f ISO8859-1 -t utf8 | \
sed -E 's/([0-9]+)\.([0-9]+),([0-9]+)/\1\2\.\3/' | \
sed -E 's/([0-9]+),([0-9]+)/\1\.\2/' | \
sed -E 's/"//g' | \
sed -E 's/([A-Z][a-z]+), ([A-Z][a-z]+)/\2 \1/' | \
sed -E 's/,/./g' | \
sed -E 's/;/,/g' | \
hledger -f - --rules-file=db.csv.rules print | \
sed -E '/^$/d' | \
sed -E 's/(.*) ; (.*)/\1\n    ; \2/' | \
sed -E 's/^(20[0-9][0-9]\/[0-9][0-9]\/[0-9][0-9]) (.*)/\1  \2/' | \
sed -E 's/ *$//'

Let’s go through this line-by-line in case you want to add your own adaptions. First we skip the funny six line header and remove the totaling sum at the bottom. Then we need to convert the encoding from ISO 8859 to UTF-8, otherwise hledger would give up reading the data. In the next lines, we convert the amount format from German 1234.00,50 EUR to 1234.50 EUR, remove quotes, replace “Doe, John” with “John Doe”, commas with dots and semicolons with commas. Then we push the cleaned data to hledger which uses a db.csv.rules file to interpret the CSV file and determine account transformations based on some heuristics. Here is the shortened db.csv.rules file that should get you running:

account1 Assets:Checking
account2 ReplaceMe

fields date,,,description,type,,,,,,,,,,out,in,
date-format %d.%m.%Y

comment %type
amount %in%out EUR

# ... patterns to determine accounts

Because I like to specify amounts as “1234 EUR” rather than “EUR1234”, I have to construct them manually with the amount statement. After letting hledger do its job the shell script takes over for final post-processing, including removal of empty lines between transactions2, date formatting and trailing white space removal. Quite a bit of upfront work, but tremendously helpful for day-to-day operations.

  1. Yes, yes, I know …

  2. They just look plain wrong with vim-ledger’s collapsed folds.

According to the stars and forks1, the Beamer theme that I introduced a year ago has filled a longstanding desire for a theme that has a simple and clean default look. It started as a one man effort but over the year I received a lot of contributions from many people, first and foremost from Benjamin Weiss – author of the HSRM theme that was a huge inspiration for this theme – and Ross Churchley. With their help, the theme transformed from a stupid hack into a polished and highly customizable Beamer theme called Metropolis that is now available on CTAN2. In this post, I will quickly go through the most important changes.

Customization

Many aspects of the layout that were formerly hardcoded can now be modified using pgfkey-based package options. For example, to change the per-frame progress bar and disable the frame numbering, you can simply pass the following options when loading the theme

\usetheme[
  outer/progressbar=foot,
  outer/numbering=none
]{metropolis}
2015-12-10/default.png

The color theme is also normalized and derives the entire look from only a few color specifications. For instance, to modify the general look you would change the normal text and alerted text colors:

\definecolor{Purple}{HTML}{911146}
\definecolor{Orange}{HTML}{CF4A30}

% Theme colors are derived from these two elements
\setbeamercolor{alerted text}{fg=Orange}

% ... however you can of course override styles of all elements
\setbeamercolor{frametitle}{bg=Purple}
2015-12-10/colors.png

Although, Metropolis is mostly recognized as a Fira Sans theme, the hard dependency quickly became a major pain point for some folks unwilling or unable to install Fira Sans. With the current version it is finally possible to compile a theme with PdfTeX or override the font as usual with

\setsansfont[BoldFont={Source Sans Pro Semibold},
              Numbers={OldStyle}]{Source Sans Pro}
2015-12-10/fonts.png

You may have noticed that the frame title does not sport small caps by default anymore. We disabled them because there were too many problems lowercasing the title and typesetting the title with \textsc{}. However, if you are like me and prefer the small caps styling, you can get them back easily with

\metroset{titleformat=allsmallcaps}
2015-12-10/allsmallcaps.png

As you can see, \metroset{} can be used within the document to change options on the fly. Moreover, there is a second option for a semi small caps style, i.e. all uppercased letters have a slightly larger x-height:

\metroset{titleformat=smallcaps}
2015-12-10/smallcaps.png

Documentation

Originally, the entire theme was documented using the top-level README. With a growing number of options this approach became a bit tedious and the README totally overcrowded. Moreover, distribution on CTAN and the likes recommends a proper manual in PDF format. Thus, we moved all documentation to a real manual and limit the README to installation and basic usage information. The manual itself is now derived from the documented TeX source.

The future

This is the first version in a series of stable releases, so you can expect a better theme – but still adhering to its core principles – soon. Nevertheless, the theme lives from outside contributions, be it code or bug reports, so do not hesitate to contact us on GitHub.

  1. As of now the repository has been starred more than 800 times and cloned by more than 150 people.

  2. … and TeXlive and MikTeX.

It’s time for the forth installment of updates concerning my Vim configuration. Unlike previous years, I made a lot less changes to my .vimrc over the year. Although, my Vim configuration converges towards a stable setup, I am still not one hundred percent satisfied with. My biggest gripe is a change in NeoComplete that sometimes completes gibberish at inappropriate times, for example when I hit the return key for a new line, NeoComplete it completes whatever it suggested on the previous line.

You will be surprised to hear that I sticked with Plug as my go-to plugin manager. There is no new one on the scene or at least none that would make any difference. However, I started to lazy-load all plugins as much as possible in order to improve startup latencies.

The behind-the-scences plugin highlight of the year is Gutentags. It’s a non-obtrusive way to generate tags for a project. Before that I updated the Ctags file with a keymap, however, that obviously causes out-of-sync problems from time to time. But these times are long forgotten.

Concerning newly acquired builtin Vim mechanics: I fully embraced the f, t, ; and , keys that I wrote about, resulting in huge productivity gains!

That’s it for this year. If you have any Vim secret sauce: let me know!

Even though I could reduce the pain of using git-annex on the command line with a Ranger plugin, the weird concept and the workflow concerning files which are not read-only let me use it less and less. Moreover, when collaborating with others, I could feel that Dropbox does something right with their system which was not possible to with git-annex in any way. However, since Dropbox is a US-based, commercial system, I was looking for alternatives …

Peer-to-peer synchronization with Syncthing

Syncthing is a dead simple tool to organize arbitrary synchronization networks between devices. Install the syncthing binary and point your browser to localhost:8384. Each device that you run Syncthing on gets a device ID – a secure certificate really – for identification. Devices who know each others device IDs form your personal cluster. You can then setup shared folders between subsets of that cluster that are then synchronized securely using TLS.

2015-11-18/syncthing.png

As you can see, I run Syncthing on three different machines, one is currently off, while the one I currently configure and another are able to synchronize. Note, though that Titan and ipemv1 cannot reliably see each other1, I use an intermediate device to distribute data from these four folders. This device’s rescan interval is set to a relatively large value because I usually won’t change data from there.

Running Syncthing on a VPS

To have a real Dropbox replacement you will need at least one server that is always on – for example a cheap VPS – which will act as the intermediate file server. Be aware though that in case you want to configure this server via the internet, you will either have to secure the connection with HTTPS2 and set up a user and a password or tunnel the HTTP port via SSH to your local machine.

To enable HTTPS, you replace the https-cert.pem and https-key.pem in ~/.config/syncthing or use the default created by Syncthing and modify

<gui enabled="true" tls="false">

in ~/.config/syncthing/config.xml accordingly. To enable SSH tunneling, just forward the remote port (in this case 8384) to some local port which you then use to access the configuration page:

$ ssh -L 9876:127.0.0.1:8384 whereever
$ firefox http://localhost:9876

Ignoring these measures could seriously harm your setup!

Long-term use

Because I only set up this system recently, I do not have any long-term experiences to share with you. Most importantly, I have no clue how often and severe conflicts happen or if the delta sync algorithm is good enough for my use cases. Only time will tell, but so far I am more than happy to retire git-annex for a much simpler setup.

  1. One is behind the home router, the other only accessible via VPN. Moreover, either one is off while the other is on.

  2. One more good reason to get a free certificate.