Ever wanted to create a picturesque masterpiece but only ever seem to manage a few abstract shapes on a page? Well, your dreams may soon come true.
NVIDIA Research have created a deep learning model, known as GauGAN, which can turn basic shapes into photorealistic images in real time. This tool makes use of generative adversarial networks, known as GANs, to transform the basic ‘segmentation maps’ into lifelike images.
The user draws a basic block image in the different colours provided, and the programme uses its learned behaviour to adjust the scene to add the new features in real time. Each element is associated with a colour which can be chosen from the options at the bottom of the screen.
The software features three basic tools; a brush; a pencil; and a fill bucket. These three tools work in the same way as in any other editing software. The brush can be used to draw rough, broad strokes of the chosen colour, whereas the pencil is used for more intricate detailing. The fill bucket can then be used to fill a selected area with a block of colour. The AI can then simultaneously take the minimalistic design to synthesize a photorealistic image.
A typical AI neural network learns to recognise photos of trees, for example, by being shown thousands of images of trees. However, these photos are only useful after each one has been individually labelled to describe what is in each image by a human.
However, GANs transcend this issue by training two competing networks to label the images themselves, with a greatly reduced input from the researchers. Generator and discriminator networks compete, labelling the images as they develop. This in turn allows them to learn from each other much more quickly than with manual input.
The generator machine tries to create a fake photograph of a tree which looks realistic, and then the discriminator machine examines the image to try and decipher whether it is a real or fake photograph. This diagram from Lyrn explains the GAN system well:
This pushes the generator to create more realistic images, and then the discriminator must work harder to tell the difference between the actual photograph and those which have been fabricated.
The name GauGAN is an affectionate nod to post-impressionist artist Paul Gauguin, who was a pioneer in synthetism. He often used contrasting colours in his paintings, and had a great influence on the works of Pablo Picasso.
Not only will this inevitably be used to impress the friends of those with a lack of artistic prowess, it also so has many real-world applications. The research carried out to create this programme could be developed for use in fields from architecture to self-driving car training to easily create realistic scenes.
GANs have a been used for a wide range of applications, from transforming a horse into a zebra in real-time, to creating a whole library of AI-generated faces of people who don’t exist.
The ‘This Person Does Not Exist’ study saw a GAN create hundreds of synthesised faces of people who do not exist. The face shown below is not a real person, rather a face which has been generated by this AI network. Click the refresh button below to see another image.
This is the future of technology in the workplace, but if your current IT systems are in need of renovation, contact us to find out how we can help.