3 references to MaxWordLengthToCache
Microsoft.ML.Tokenizers (3)
Model\BPETokenizer.cs (3)
1354
if (textSpan.Length <=
MaxWordLengthToCache
)
1470
if (textSpan.Length <=
MaxWordLengthToCache
)
1551
if (textSpan.Length <=
MaxWordLengthToCache
)