If you don’t already know, style transfer is the cool, hip thing that has been taking the recreational AI community by storm. It’s so cool that even Kristen Stewart co-authored a paper about it. To quote one researcher who has done extensive work in style transfer that I’ve got a chance to talk to, “it is an utterly unremarkable paper that wouldn’t have been published otherwise [if Kristen Stewart’s name is not on it]. That’s a publicity stunt.”

Some background on why I’m doing this: I’m teaching the course CS 20SI: “TensorFlow for Deep Learning Research” and for the assignment about convolution neural networks, I thought it’d be fun for students to do style transfer as their exercise at home. They, after all, showed a lot of enthusiasm when we did Deep Dream in class.

There have been several implementations of style transfer in TensorFlow made available to the public, for example, this by log0 and this by Anish Athalye. I consulted both repositories when I wrote the assignment and they were very helpful when I needed to figure out some tricky details. However, both implementations had some parts that I thought I could improve on. I also wrote a very detailed instruction explaining some of the math of the model and how to do certain implementation details in TensorFlow. I also published the starter code with comments so you can implement it in your own free time. The model works pretty well, but it’s really not my merit. The hard part of figuring out how to do style transfer has been done by other people. I just reimplemented it in a way that I thought would be easier to understand.

Basically, for style transfer, we build a system that can take in two images (one content image and one style image), and output another image whose content is closest to the content of the content image while style is closest to the style of the style image. For example, take this image of Deadpool in the Civil War as the content image and combine it with Guernica by Picasso (1937) as the style image.

+

=

Some more Deadpool in arts:

The style images, from left to right, top to bottom are:

‘The Starry Night’ by Vincent van Gogh (1889)

Pattern by Olga Drozdova (201x?)

‘The Harlequin’s Carnival’ by Joan Miró (1924-1925)

‘Metamorphosis of Narcissus’ by Salvador Dalí (1937)

And because I’m self-indulgent, I used the model on my own image. I took this image:

And did a really sloppy job to produce the several images below. I said sloppy because I didn’t spend as much time tuning those images as I did with Deadpool.

Again, if you’re interested, you can follow the instruction.

Have fun!