2 references to Replace
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
826
sb.
Replace
(EndOfWordSuffix, " ");
831
sb.
Replace
(ContinuingSubwordPrefix, string.Empty);