1 reference to EncodeToIds
Microsoft.ML.Tokenizers (1)
Model\EnglishRobertaTokenizer.cs (1)
408
=>
EncodeToIds
(text, textSpan, settings.ConsiderPreTokenization, settings.ConsiderNormalization, settings.MaxTokenCount);