[Day 276] Detailed instruction on how to do Style Transfer

If you don’t already know, style transfer is the cool, hip thing that has been taking the recreational AI community by storm. It’s so cool that even Kristen Stewart co-authored a paper about it. To quote one researcher who has done extensive work in style transfer that I’ve got a chance to talk to, “it is an utterly unremarkable paper that wouldn’t have been published otherwise [if Kristen Stewart’s name is not on it]. That’s a publicity stunt.”

kristen stewart style transfer.jpgSome background on why I’m doing this: I’m teaching the course CS 20SI: “TensorFlow for Deep Learning Research” and for the assignment about convolution neural networks, I thought it’d be fun for students to do style transfer as their exercise at home. They, after all, showed a lot of enthusiasm when we did Deep Dream in class.

There have been several implementations of style transfer in TensorFlow made available to public, for example, this by log0 and this by Anish Athalye. I consulted both repositories when I wrote the assignment and they were very helpful when I needed to figure out some tricky details. However, both implementations had some parts that I thought I could improve on. I also wrote a very detailed instruction explaining some of the math of the model and how to do certain implementation details in TensorFlow. I also published the starter code with comments so you can implement it in your own free time. The model works pretty well, but it’s really not my merit. The hard part of figuring out how to do style transfer has been done by other people. I just reimplemented it in a way that I thought would be easier to understand.

Basically, for style transfer, we build a system that can take in two images (one content image and one style image), and output another image whose content is closest to the content of the content image while style is closest to the style of the style image. For example, take this image of Deadpool in Civil War as the content image and combine it with Guernica by Picasso (1937) as the style image.

big_deadpool.jpg

+

big_guernica.jpg

=

159.png

Some more Deadpool in arts:

deadpool collage.jpg

The style images, from left to right, top to bottom are:

‘The Starry Night’ by Vincent van Gogh (1889)

Pattern by Olga Drozdova (201x?)

‘The Harlequin’s Carnival’ by Joan Miró (1924-1925)

‘Metamorphosis of Narcissus’ by Salvador Dalí (1937)

And because I’m self-indulgent, I used the model on my own image. I took this image:

big_chip

And did a really sloppy job to produce the several images below. I said sloppy because I didn’t spend as much time tuning those images as I did with Deadpool.

chip collage.jpg

Again, if you’re interested, you can follow the instruction.

Have fun!

 

 

Advertisements
[Day 276] Detailed instruction on how to do Style Transfer

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s