2 writes to EscapeWhiteSpaces
Microsoft.ML.Tokenizers (2)
Model\SentencePieceBaseModel.cs (2)
34EscapeWhiteSpaces = modelProto.NormalizerSpec.EscapeWhitespaces; 95EscapeWhiteSpaces = options.EscapeWhiteSpaces;
5 references to EscapeWhiteSpaces
Microsoft.ML.Tokenizers (5)
Model\SentencePieceBaseModel.cs (4)
57AddDummyPrefix, EscapeWhiteSpaces, 226char prefixSuffixChar = EscapeWhiteSpaces ? SentencePieceNormalizer.DummyPrefix : ' '; 334return EscapeWhiteSpaces ? sb.ToString(SentencePieceNormalizer.DummyPrefix, ' ') : sb.ToString(); 451char prefixSuffixChar = EscapeWhiteSpaces ? SentencePieceNormalizer.DummyPrefix : ' ';
Model\SentencePieceTokenizer.cs (1)
61public bool EscapeWhiteSpaces => _model.EscapeWhiteSpaces;