1 reference to SplitText
Microsoft.ML.Tokenizers (1)
PreTokenizer\RegexPreTokenizer.cs (1)
55
return
SplitText
(text, _regex, _specialTokensRegex);