Types

6 references to Types

Microsoft.ML.Tokenizers (6)

Model\LlamaTokenizer.cs (1)

47if (modelProto.TrainerSpec.ModelType != TrainerSpec.Types.ModelType.Bpe)

Model\SentencePieceTokenizer.cs (2)

27TrainerSpec.Types.ModelType.Bpe => new SentencePieceBpeModel(modelProto, addBos, addEos, specialTokens), 28TrainerSpec.Types.ModelType.Unigram => new SentencePieceUnigramModel(modelProto, addBos, addEos, specialTokens),

SentencepieceModel.cs (3)

325internal Types.ModelType ModelType { get; private set; } = Types.ModelType.Unigram; 350result.ModelType = (Types.ModelType)SentencePieceProtobufReader.ReadRawVarint32(data, end, ref pos);