2 references to Replace
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
665
sb.
Replace
(EndOfWordSuffix, " ");
670
sb.
Replace
(ContinuingSubwordPrefix, string.Empty);