Help applying changes made to the c++ model

Support
1

Hello, I am making some changes such as adding attention to the results of the whisper model. I made some modifications, such as adding arguments to the generate function, however when I run the model using fast whisper it does not detect the change made to the c++ whisper model.

Do I have to build it every time I push a change to a dev fork of ctranslate2 or is there something else I am missing?

2

Hi,

Yes you should rebuild the package each time you make a change to the C++ code.

3

Hello, after compiling the C++ library and compiling the python wrapper in google colab and trying to run fast whisper with the change I get this error

/usr/local/lib/python3.8/dist-packages/faster_whisper/__init__.py in <module>
----> 1 from faster_whisper.transcribe import WhisperModel

/usr/local/lib/python3.8/dist-packages/faster_whisper/transcribe.py in <module>
      2 import zlib
      3 
----> 4 import ctranslate2
      5 import numpy as np
      6 import tokenizers

/usr/local/lib/python3.8/dist-packages/ctranslate2/__init__.py in <module>
     18 
     19 try:
---> 20     from ctranslate2._ext import (
     21         AsyncGenerationResult,
     22         AsyncScoringResult,

ImportError: libctranslate2.so: cannot open shared object file: No such file or directory

I followed the documentation, is there something I’m missing in the building process? I’m not sure that libctranslate2.so is even being added to any file directory i.e. it doesn’t show up when uninstalling it unlike when I uninstall the pip-installed version.

5

You should read the “Attention” section in the documentation: Installation — CTranslate2 3.5.0 documentation

By default, libctranslate2.so is not embedded in the Python wheel. You need to use auditwheel if you want to do that but it is not necessary during development.

6

I am already setting the env variable based on the documentation

%env CTRANSLATE2_ROOT=/opt/ctranslate2
%env LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CTRANSLATE2_ROOT/lib

the build code segment looks like this

%cd CTranslate2
!mkdir build
%cd build
!cmake .. -DWITH_MKL=OFF -DWITH_CUDA=ON -DWITH_CUDNN=ON
!make -j4
!make install

# compile python wrapper
%cd ../python
!pip install -r install_requirements.txt
!python setup.py bdist_wheel
!pip install dist/*.whl

when I then run fast-whisper I get the error above, is there anything I am doing wrong, I am also running this on google colab if that changes anything.

7

Check that LD_LIBRARY_PATH is still set correctly when running faster-whisper.

8

I’m not sure what to set LD_LIBRARY_PATH because I cannot find where libctranslate2.so is installed. I am installing the c++ build in /opt/ctranslate2

my mistake I put the wrong build code segment, the updated one has -DCMAKE_INSTALL_PREFIX in the build

the build code segment should be

# set the env variables
%env CTRANSLATE2_ROOT=/opt/ctranslate2
%env LD_LIBRARY_PATH=$CTRANSLATE2_ROOT/lib

# compile the c++
%cd CTranslate2
!mkdir build
%cd build
!cmake .. -DWITH_MKL=OFF -DWITH_CUDA=ON -DWITH_CUDNN=ON -DCMAKE_INSTALL_PREFIX=${CTRANSLATE2_ROOT}
!make -j4
!make install

# compile python wrapper
%cd ../python
!pip install -r install_requirements.txt
!python setup.py bdist_wheel
!pip install dist/*.whl

I’m not sure what LD_LIBRARY_PATH should point to.