ValueError: Cannot use sin/cos positional encoding with odd dim (got dim=515)

Support
1

Hi,
I’m getting the following error when training a transformer with source features. I only have one feature per token.

ValueError: Cannot use sin/cos positional encoding with odd dim (got dim=515)

2

You probably want to set both --word_vec_size and --feat_vec_size to make the total embedding size an even size.

3

So, if --word_vec_size is 512 and I have 3 possible values for the feature it sums up to 515. Therefore, I should set --feat_vec_size for example to 8, in order to have a 520 dimension. Am I right?

4

Yes, this should work.

5

Thanks @guillaumekln !