Creative GANs is a research project on training language models to generate creative text with either a Maximum Likelihood Estimation (MLE) or GAN objective. It contains the code for running the experiments as described in the paper.
Some curated outputs can be found here
- Gutenberg Novels
- English poetry
- Song lyrics
- Metaphors
- AWD LSTM paper link
- Transformer XL paper link
The encoder models are from the fastai library
- Preprocess data
python preprocess.py [gutenberg/metaphors/poems/lyrics]and save preprocessed file - Train language model
lang_model.py PATH FILENAME MODEL [PRETRAINED_FNAMES] - Train gan model
textgan.py PATH FILENAME PRETRAINED MODEL [CRIT] [PREDS] [EPOCHS]
PATH - folder with data
FILENAME - name of preprocessed file
PRETRAINED - fastai model saved with learn.save()
PRETRAINED_FNAMES - pretrained weight file and vocab file (comma seperated)
MODEL - architecture to use {'AWD': AWD_LSTM, 'XL':TransformerXL}
CRIT - loss function: gumbel softmax/reinforce (only for gan)
PREDS - generate output from validation set
EPOCHS - number of epochs to train