1 write to TreatWhitespaceAsSuffix
Microsoft.ML.Tokenizers (1)
SentencepieceModel.cs (1)
354result.TreatWhitespaceAsSuffix = SentencePieceProtobufReader.ReadRawVarint32(data, end, ref pos) != 0;
2 references to TreatWhitespaceAsSuffix
Microsoft.ML.Tokenizers (2)
Model\SentencePieceBaseModel.cs (2)
35TreatWhitespaceAsSuffix = modelProto.TrainerSpec.TreatWhitespaceAsSuffix; 58modelProto.TrainerSpec.TreatWhitespaceAsSuffix,