6 references to MergeWord
Microsoft.ML.Tokenizers (6)
Model\BPETokenizer.cs (6)
1348word = MergeWord(textSpan, ref priorityQueue); 1357word = MergeWord(text, ref priorityQueue); 1464word = MergeWord(textSpan, ref priorityQueue); 1473word = MergeWord(textSpan, ref priorityQueue); 1545word = MergeWord(textSpan, ref priorityQueue); 1554word = MergeWord(textSpan, ref priorityQueue);