3 references to MaxWordLengthToCache
Microsoft.ML.Tokenizers (3)
Model\BPETokenizer.cs (3)
1000
if (text.Length <=
MaxWordLengthToCache
)
1075
if (text.Length <=
MaxWordLengthToCache
)
1108
if (text.Length <=
MaxWordLengthToCache
)