3 references to MaxWordLengthToCache
Microsoft.ML.Tokenizers (3)
Model\BPETokenizer.cs (3)
1354if (textSpan.Length <= MaxWordLengthToCache) 1470if (textSpan.Length <= MaxWordLengthToCache) 1551if (textSpan.Length <= MaxWordLengthToCache)