2 references to WordToTokens
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
1344
WordToTokens
(ref word, tokens, offset, mapping);
1360
WordToTokens
(ref word, tokens, offset, mapping);