6 references to MergeWord
Microsoft.ML.Tokenizers (6)
Model\BPETokenizer.cs (6)
1352word = MergeWord(textSpan, ref priorityQueue); 1361word = MergeWord(text, ref priorityQueue); 1468word = MergeWord(textSpan, ref priorityQueue); 1477word = MergeWord(textSpan, ref priorityQueue); 1549word = MergeWord(textSpan, ref priorityQueue); 1558word = MergeWord(textSpan, ref priorityQueue);