Skip to content

greydanus/scribe

Repository files navigation

Scribe: Realistic Handwriting in Tensorflow

See blog post

Samples

"A project by Sam Greydanus" Sample output 1 "You know nothing Jon Snow" (print) Sample output 2 "You know nothing Jon Snow" (cursive) Sample output 3

"lowering the bias" Sample output 4 "makes the writing messier" Sample output 5 "but more random" Sample output 6

Jupyter Notebooks

For an easy intro to the code (along with equations and explanations) check out these Jupyter notebooks:

Getting started

  • install dependencies (see below).
  • download the repo
  • navigate to the repo in bash
  • download and unzip folder containing pretrained models: Google Drive link
    • place in this directory

Now you have two options:

  1. Run the sampler in bash: mkdir -p ./logs/figures && python run.py --sample --tsteps 700
  2. Open the sample.ipynb jupyter notebook and run cell-by-cell (it includes equations and text to explain how the model works)

About

This model is trained on the IAM handwriting dataset and was inspired by the model described by the famous 2014 Alex Graves paper. It consists of a three-layer recurrent neural network (LSTM cells) with a Gaussian Mixture Density Network (MDN) cap on top. I have also implemented the attention mechanism from the paper which allows the network to 'focus' on character at a time in a sequence as it draws them.

The model at one time step looks like this

Rolled model

Unrolling in time, we get Unrolled model

I've implemented the attention mechanism from the paper: Attention mechanism

Dependencies

  • All code is written in python 2.7. You will need:
  • Numpy
  • Matplotlib
  • TensorFlow 1.0
  • OPTIONAL: Jupyter (if you want to run sample.ipynb and dataloader.ipynb)