2 references to EncodeToIdsResult
Microsoft.ML.Tokenizers (2)
Model\CodeGenTokenizer.cs (2)
1125
return
EncodeToIdsResult
(hit, accumulatedIds, maxTokens, textSpan.Length, out charsConsumed);
1155
return
EncodeToIdsResult
(result, accumulatedIds, maxTokens, textSpan.Length, out charsConsumed);