2 references to EncodeToIdsResult
Microsoft.ML.Tokenizers (2)
Model\EnglishRobertaTokenizer.cs (2)
672
return
EncodeToIdsResult
(hit, accumulatedIds, maxTokens, text.Length, out charsConsumed);
704
return
EncodeToIdsResult
(result, accumulatedIds, maxTokens, text.Length, out charsConsumed);