What does Hegel mean by “Reason”?
Published:
I finally finished reading the big and baffling Phenomenology of Spirit by Georg Wilhelm Friedrich Hegel.
Blog posts
2021
First PC build
Published:
My everyday work computer is a MacBook Pro that I’ve had since 2013. It’s a great machine and continues to serve me well, but I was moved in a moment of pandemic malaise to treat myself to a little upgrade.
2020
End-to-end models falling short on SLURP
Published:
There’s a hot new dataset for spoken language understanding: SLURP. I’m excited about SLURP for a couple reasons:
Siri from scratch! (Not really.)
Published:
I make fairly heavy use of the voice assistant on my phone for things like setting timers while cooking. As a result, when I spent some time this summer at my in-laws’ place—where there was no cell signal and not-very-good Wi-Fi—I often tried using Siri only to get a sad little “sorry, no Internet :(“ response. (#FirstWorldProblems.)
Sequence-to-sequence learning with Transducers
Published:
The Transducer (sometimes called the “RNN Transducer” or “RNN-T”, though it need not use RNNs) is a sequence-to-sequence model proposed by Alex Graves in “Sequence Transduction with Recurrent Neural Networks”. The paper was published at the ICML 2012 Workshop on Representation Learning. Graves showed that the Transducer was a sensible model to use for speech recognition, achieving good results on a small dataset (TIMIT).
My research goals
Published:
I wanted to clarify to myself and others what some of my research goals are, and why I’m working on certain problems. The hope is that putting this online for the world to see will help challenge me to keep focused and working towards those goals—sort of like telling your friends that you’re going to quit smoking, or something like that.
Predictive coding in machines and brains
Published:
The name “predictive coding” has been applied to a number of engineering techniques and scientific theories. All these techniques and theories involve predicting future observations from past observations, but what exactly is meant by “coding” differs in each case. Here is a quick tour of some flavors of “predictive coding” and how they’re related.
A contemplation of $\text{logsumexp}$
Published:
$\text{logsumexp}$ is an interesting little function that shows up surprisingly often in machine learning. Join me in this post to shed some light on $\text{logsumexp}$: where it lives, how it behaves, and how to interpret it.
Notebook: Fun with Hidden Markov Models
Published:
I’ve written a notebook introducing Hidden Markov Models (HMMs) with a PyTorch implementation of the forward algorithm, the Viterbi algorithm, and training a model on a text dataset—check it out here!
2019
An introduction to sequence-to-sequence learning
Published:
Many interesting problems in artificial intelligence can be described in the following way:
Map a sequence of inputs $\mathbf{x}$ to the correct sequence of outputs $\mathbf{y}$.