[funboarder13920]

openai whisper large-v3 introduces change from 80 to 128 in mel input feature.
exposing n_mels is required to propagate the input size to the audio feature extractor

we also need to add the large-v3 alignment heads
a fix is required in the computation of _is_multilingual

[chiiyeh]

Hi could you also add to here: https://github.com/OpenNMT/CTranslate2/blob/master/python/ctranslate2/converters/transformers.py#L1929-L2042

    "openai/whisper-large-v3": [
        (7, 0),
        (10, 17),
        (12, 18),
        (13, 12),
        (16, 1),
        (17, 14),
        (19, 11),
        (21, 4),
        (24, 1),
        (25, 6),
    ],

obtained from here: https://github.com/openai/whisper/blob/fcfeaf1b61994c071bba62da47d7846933576ac9/whisper/__init__.py#L45

[RafaRed]

The current check for multilingual support seems to be hardcoded with a specific vocabulary size:
_is_multilingual = vocabulary.size() == 51865;
Link to code

For instance, I believe the whisper-latest-v3 model has a vocabulary size of 51866, which is one more than the hardcoded value. This discrepancy could lead to the multilingual feature being incorrectly disabled for this model.

Probably a more dynamic check need to be implemented to ensure compatibility with future models.

edit: oh sorry, did not notice its already fixed on PR.

[funboarder13920]

The current check for multilingual support seems to be hardcoded with a specific vocabulary size: _is_multilingual = vocabulary.size() == 51865; Link to code

For instance, I believe the whisper-latest-v3 model has a vocabulary size of 51866, which is one more than the hardcoded value. This discrepancy could lead to the multilingual feature being incorrectly disabled for this model.

Probably a more dynamic check need to be implemented to ensure compatibility with future models.

A fix is already in this PR :

CTranslate2/src/models/whisper.cc

Line 73 in 7615e41

_is_multilingual = vocabulary.size() >= 51865;

[Purfview]

@vince62s Can you merge this?

[ostegm]

Possible worth adding a comment on the source of these since its different from the source on L1928.