1 reference to LastIndexOf
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
481
return
LastIndexOf
(text, textSpan, settings.MaxTokenCount, settings.ConsiderPreTokenization, settings.ConsiderNormalization, out normalizedText, out tokenCount);