I would like to do text generation in french using a language model such as GPT-2.
I want to fine-tune it on a specific corpus. My problems are:
(1) There is no available good GPT-2 for french trained from scratch like the english one, hence I cannot use it as a base to fine tune it.
(2) Some people try to use the english base and fine tune it in french, it seems to work technically but results are quite poor in my opinion.
I only have a 2080ti for the moment, I would really appreciate any advice to make it work.
P.S. I am not using [MASK] based language models like BERT since they perform poorly in text generation.