Training GANs: A Convex Optimization Perspective

Generative adversarial networks (GANs) comprise a potent class of deep generative models that are extensively used in many fields of AI and machine learning. The potential of GANs is enormous because they can be trained to mimic any distribution of data. They can learn to create environments in domains such as images, music, dialogue, and text. However, the process of training GANs is known to present many problems and complexities. This critical aspect is the focus of a research proposal by EPFL’s Professor Volkan Cevher. The proposed work (A Convex Optimization Perspective for GANs) has been selected by Google AI for its Faculty Research Award for 2018.

The proposed research builds on Prof. Cevher’s earlier work on Langevin dynamics to boost the overall efficiency of the new training procedures through novel sampling algorithms, and to help obtain optimal solutions numerically via convex optimization. It not only aims to provide effective solutions to known GAN problems that inhibit efficiency–such as non-convergence, mode collapse, and diminished gradient—but also proposes a framework for attaining ‘mixed Nash Equilibrium’ (mixed NE) for the selected formulations.

While there is a vast body of literature on training GANs, only a handful of papers discuss the mixed NE perspective, and that too by advancing heuristic algorithms that assume the discriminator to be a single-layered neural network. On the other hand, Prof. Cevher’s study is applicable to arbitrary architectures. That increases the novelty of his study.

The research will also establish how to generate robust GAN formulations by controlling the discriminator network’s Lipschitz constant with new scalable semidefinite programming approaches for concrete applications.

The Google Faculty Research Award, which provides funding for research at institutions around the world, is a highly coveted distinction conferred on world-class technical research in various fields. Prof. Cevher is an awardee under the ‘machine learning and data mining’ category.

Related Publications