2 references to WordToTokens
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
1344WordToTokens(ref word, tokens, offset, mapping); 1360WordToTokens(ref word, tokens, offset, mapping);