1 reference to CountIdsUpToMaxFromEnd
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
1002
return word.
CountIdsUpToMaxFromEnd
(maxTokens, fullTextLength, out textIndex);