It’s not a lot of fun to play, but it’s fairly remarkable considering it was created entirely by an AI.
A couple of programmers wanted to test whether they could teach a generative adversarial network (GAN) to develop its own Grand Theft Auto 5 version. The end result is a somewhat hazy acid trip that is readily recognised as GTA5.
Harrison Kinsley, who goes by Sentdex on YouTube, and his partner trained a fork of Nvidia’s GameGAN neural network using a black car on a short section of highway in Grand Theft Auto 5. Nvidia loaned them its DGX Station, which is equipped with four A180 gigabyte cards to help with the processing. Kinsley explains what they did and demos the results in the video above.
Initial models were very pixelated, but Kinsley improved this with AI-assisted supersampling. While it’s not pretty, it’s important to keep in mind that this is not GAN-generated footage. It is a real-time interactive demo. Kinsley is driving an AI-created car in a fully AI-created environment.
Before and after upsampling.
Kinsley explained that his training time was limited since the DGX Station was on loan. Despite the fact that the model attempts to compute obstacle clipping in certain cases, he would have wanted to run additional collision samples. Kinsley was also curious as to how much of the GTA5 map the GAN could handle. That, however, would have needed hours of additional training as he gradually increased the driving distance, something he did not have time to accomplish.
The movie reveals that the neural network performed a good job of replicating some surprising elements. One would assume that the GAN would disregard shadows and the sun shining off the automobile for the sake of efficiency. It didn’t, much to Kinsley’s astonishment. Shadows, lighting, and reflections move in the anticipated manner. The GAN also developed its own rudimentary physics system after being trained by colliding with objects. Another example is how another automobile skews to the right when the left rear quarter panel is tapped. Head-on accidents are not as adequately managed.
Kinsley has uploaded a playable demo called “GAN Theft Auto” on GitHub for those who want to try it out. Again, it’s only a little section of the GTA map, and it’s not the same as playing a genuine game. It’s more of a tech demo than anything more, although it’s fascinating to see how the model performs in untrained scenarios. That’s when things start to turn trippy.