2 writes to EscapeWhiteSpaces
Microsoft.ML.Tokenizers (2)
Model\SentencePieceBaseModel.cs (2)
34EscapeWhiteSpaces = modelProto.NormalizerSpec.EscapeWhitespaces; 81EscapeWhiteSpaces = escapeWhiteSpaces;
5 references to EscapeWhiteSpaces
Microsoft.ML.Tokenizers (5)
Model\SentencePieceBaseModel.cs (4)
57AddDummyPrefix, EscapeWhiteSpaces, 209char prefixSuffixChar = EscapeWhiteSpaces ? SentencePieceNormalizer.DummyPrefix : ' '; 317return EscapeWhiteSpaces ? sb.ToString(SentencePieceNormalizer.DummyPrefix, ' ') : sb.ToString(); 434char prefixSuffixChar = EscapeWhiteSpaces ? SentencePieceNormalizer.DummyPrefix : ' ';
Model\SentencePieceTokenizer.cs (1)
57public bool EscapeWhiteSpaces => _model.EscapeWhiteSpaces;