I have a memory leak in my application which uses CTranslate. I’ve been able to narrow the problem down to occurring when I create a CTranslate Translator inside of a PyQt QThread that is created from a QWidget. If I remove the CTranslate Translator and do something else that allocates a large amount of memory there is no leak. If I create the CTranslate Translator from the QWidget with no QThread there is also no leak. If I run the QThread outside of a QWidget there is no leak. The leak only happens with the combo of all three.
My best guess is that there is some bug/me misusing in the combination of Python/Qt/CTranslate memory management. Python uses automatic reference counting memory management, while Qt in native C++ use C++ parent based memory management. On top of that CTranslate uses C++ extensions to Python so it seems like there are a lot of places where the problem could be appearing.
I made an example script demonstrating the leak. To run it you need a CTranslate model and need to provide a path to it in the script. Here’s a Google Drive link where you can download a package for my project that if extracted (its just a renamed .zip archive) has a CTranslate model at /model. When this script runs it leaks ~5GB of memory.
Interesting that you can’t reproduce it. For me on Ubuntu 20.04.1 running the linked script with just updating the path to a CTranslate model leaks memory. I’ll try reproducing it on a VM, maybe it has more to do with my specific system then I think.
Not using a new Translator instance for every translation makes sense but it seems like it could use excessive memory if multiple different translations are used and they all stay in memory.
@guillaumekln tested on two separate Ubuntu 20.04 machines with the script above and I get the leak. The leak should also appear if you install the application from the Snap Store or PyPI and do a large number of translations (this is easiest done by just typing something long and having it translated as you go).
I can reproduce with Ubuntu 20.04. However, this is not a CTranslate2 issue so I can’t investigate more. It’s probably related to the error message:
QThread::wait: Thread tried to wait on itself
If all models are used at the same time, they are all in memory anyway. If a single model is used at a time, you can unload the others. For example, if the user changes the language pair you can unload the previous model and lazy load the new one on the first translation.
Interesting. In one of my own set-ups where all my models are loaded at start-up I eventually get a memory leak (with 32 GB RAM). In another set-up on my tiny Asus Zenbook (with 8 GB RAM) the models are loaded on demand and I NEVER get any memory issues.
I’m not sure your implementation of QThread is solid. You could try avoid subclassing QThread and instead create worker objects and use moveToThread and signals and slots to keep synchronization.
But why do you need threading in the GUI? My understanding is that the user cannot do other work in the GUI while a translation is in progress, so there’s no problem anyways if the GUI is blocked.