1 reference to LastIndexOf
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
466
return
LastIndexOf
(text, textSpan, settings.MaxTokenCount, settings.ConsiderPreTokenization, settings.ConsiderNormalization, out normalizedString, out tokenCount);