3 references to MaxWordLengthToCache
Microsoft.ML.Tokenizers (3)
Model\BPETokenizer.cs (3)
1350
if (textSpan.Length <=
MaxWordLengthToCache
)
1466
if (textSpan.Length <=
MaxWordLengthToCache
)
1547
if (textSpan.Length <=
MaxWordLengthToCache
)