1 reference to CountIdsUpToMax
Microsoft.ML.Tokenizers (1)
Model\BpeTokenizer.cs (1)
1408return word.CountIdsUpToMax(maxTokens, out charsConsumed);