1 reference to Bpe
Microsoft.ML.Tokenizers (1)
Model\LlamaTokenizer.cs (1)
44
if (modelProto.TrainerSpec.ModelType != TrainerSpec.Types.ModelType.
Bpe
)