2 references to EncodeToIdsResult
Microsoft.ML.Tokenizers (2)
Model\EnglishRobertaTokenizer.cs (2)
661
return
EncodeToIdsResult
(hit, accumulatedIds, maxTokens, text.Length, out charsConsumed);
693
return
EncodeToIdsResult
(result, accumulatedIds, maxTokens, text.Length, out charsConsumed);