# Recent Publications

. Physically Embedded Planning Problems: New Challenges for Reinforcement Learning. Preprint, 2020.

. The Option Keyboard: Combining Skills in Reinforcement Learning. NeurIPS, 2019.

. Composing Entropic Policies using Divergence Correction. ICML, 2019.

. Transfer in Reinforcement Learning with Successor Features and Generalised Policy Improvement. NeurIPS Infer2Control Workshop, 2018.

. Successor Features for Transfer in Reinforcement Learning. NIPS, 2017.

. Transfer in Reinforcement Learning with Successor Features and Generalised Policy Improvement. ICLR Lifelong Learning Workshop, 2017.

. Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. NIPS, 2016.

. Deep Reinforcement Learning in Large Discrete Action Spaces. Preprint, 2016.

. Continuous control with deep reinforcement learning. ICLR, 2016.

. Memory-based control with recurrent neural networks. Neural Information Processing Systems Deep Reinforcement Learning Workshop, 2015.

# Posts

### Removing Type 3 Fonts in a Hurry with Adobe Acrobat DC

One of the things they don’t show in the AI movies is the part where the protagonist is fixing their camera ready1 just before the deadline and discovers (unexpected plot twist) that their pdf contains “Type 3” bitmap fonts which are apparently forbidden. Frantically, they google for a solution. This was me recently and I relay my findings below. The most likely cause of this error, when using pdflatex, is that one or more of your figures contains type 3 fonts.

### github pages secure with cloudflare

[ Update October 2018: I no longer use this setup. I migrated to gitlab (mostly because gitlab can easily build my website with hugo. Gitlab provides straightforward instructions on securing your site with Cloudflare. This would be my recommended setup now. ] Do you see a little green icon in your browser bar? This website is now served using TLS (a secure connection also known as SSL) thanks to Cloudflare’s new free Universal SSL1.

### btrees are the new black

Self-balancing trees are an important class of data structures. In most textbooks, red/black binary trees are introduced first, and then btrees come later, usually mentioned as a data structure used in file systems. Both data structures have asymptotically identical performance ($O(\log n)$ time for insertion, lookup and deletion). Here, I will show why, despite your textbook’s ordering, btrees may often be a better choice. Btrees (or variants) are often used in file systems because they make better use of devices which read and write blocks of data (such as hard drives or flash).

### atoms slash symbols in python

I’ve been learning Erlang and one thing I like is the idea of atoms (aka symbols in Ruby I think) 1. Atoms in Erlang are written by just using lowercase. They are pointers to a global string table. This means comparison operations are very fast (you are just comparing two pointers) but for debugging and pattern matching the atom has a meaning (the string). Atoms are particularly useful in pattern-matching languages like Erlang, but they’re a useful idea even without pattern matching.

### you look ridiculous

Conservative New Zealand politician Colin Craig threatened satirical website The Civilian with defamation proceedings because they “quoted” him in satirical news article news link. Outrageously, Colin’s lawyer even demanded \$500 cash as well as a correction. Luckily, the author of The Civilian responded with a tongue-in-cheek correction and Colin appears to have realized how silly the whole thing makes him look and dropped the whole thing. Unfortunately, I’m not as a clever as some, so I will respond by writing a boring policy recommendation instead.