2 references to Replace
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
822
sb.
Replace
(EndOfWordSuffix, " ");
827
sb.
Replace
(ContinuingSubwordPrefix, string.Empty);