Editing Faces with StyleGAN

3 minute read

Introduction

I recently set out to manipulate faces with StyleGAN and this is the end result. My goal was not to start from scratch, but to find the quickest path to producing results. Unfortunately, StyleGAN has been around for several years, and the best resource I found had fallen into disrepair, so I’ll talk through my fixes here, as well as the results.

Source

This YouTube video was the launching point I decided to work from. The producer provides two Colab Jupyter notebooks that are saved here. The first notebook accepts images provided by the user and encodes them into StyleGAN’s latent space. The second notebook begins with this encoding and generates new images, morphing the face in each image based on age, gender, pose, smile and whether the person is wearing glasses.

Problems

The first notebook relies on a repository called StyleGAN-Encoder. The notebook could not run to the end without an error occurring. Colab would complain that some quota had been exceeded when trying to download a file required by encode_images.py. So I modified it to longer download the file. Instead, the new notebook I’ve created (described below) will download the file to its temporary workspace and encode_images.py will find it. I’ve stored this modification on Github by forking the original repo here.

The second notebook relies on a repository called InterfaceGAN. No changes to this source code were needed in order for the notebook to work, but small changes were needed to the notebook itself.

I have distilled these two notebooks into a single notebook to do everything I wanted. The only trouble with my notebook is that it almost always encounters memory problems when trying to generate new images. This is easily fixed by restarting the notebook’s environment, then running 2 of the later cells in the notebook again. See the notebook for detailed instructions. This new notebook has been added to my StyleGAN-Encoder repo as Face_Editing_with_StyleGAN.ipynb. The notebook is also accessible from my Google Drive here.

Results

First, I’ll show some results when an image from StyleGAN2 is used as the seed. These images can be obtained from thispersondoesnotexist.com. Here’s my seed image:

The encoded image (shown below) should be very similar since StyleGAN2 was used to produce the original. But the new images should have some degradation because the code in these notebooks is based on the original StyleGAN, whereas the seed has been taken from its successor. And indeed there are some noticable differences: (1) the frames of the eyeglasses are more square-shaped and they don’t look quite as natural because the frames don’t have any highlighting from lighting reflections, and (2) the skin is largely free of blemishes.

The next two images correspond with making the person younger or older respectively. We can see that latent direction in the model seems to be confounded with eyeglass frame thickness: the older someone appears, the thicker their frames will be.

The next two images correspond with altering the pose of the person’s head. The model appears to do a good job in this case. It is also interesting that the profile angle elects to use a common convention of photographers: leading the subject. Instead of keeping the subject centered, the person’s head has been moved to the right edge of the image and they are looking toward the empty side of the image.

The next two images correspond with decreasing and increasing the person’s smile respectively. The reduced smile appears fine, but the stronger smile seems to have make the face more like a female.

Lastly, the next two images correspond with decreasing or increasing the presence of eyeglasses respectively. The former looks fine, but the eyeglasses in the latter image look cartoonish and there’s some bad artifacts where the hair and ear meet the frame. However, these two don’t seem to alter the age of the person, which was expected based on the age adjustments shown above.