PI: Pontus Giselsson (LU).
Generative adversarial networks (GANs) are generative networks designed to learn probability distributions of training data. They consist of two deep neural networks with opposite objectives. One network, the generator, generates new fake data instances, while the other, the discriminator, evaluates them for authenticity. This adversarial structure gives rise to training problems with saddle-point structure. These are inherently different from standard deep neural network training (minimization) problems and are considered very difficult to train. A reason is that stochastic gradient descent (SGD), that works well for standard deep learning training, is often blindly applied to also train GANs. SGD, and its deterministic counterpart gradient descent, may fail to converge even on very simple convex-concave saddle-point problems. This research program will provide a strong mathematical foundation for training of GANs. A starting point will be a recently submitted paper [CV:paper[S1]] by the project proposer. It introduces the novel mathematical concept of nonlinear resolvents, and a nonlinear forward-backward method. Two, among many, special cases of the proposed method are forward-backward-forward splitting (FBF) and the extra-gradient method (EG) that both can solve saddle-point problems. Our framework opens up a completely new research direction in large-scale optimization and it sheds new light on how FBF and EG work. This project will take the first steps into this research direction with one long term objective being to devise algorithms that efficently and realiably can train GANs.
Project number: A20
