3 references to SegmentFromEnd
Microsoft.ML.Tokenizers (3)
Model\SentencePieceTokenizer.cs (3)
1422if (!SegmentFromEnd(symbols[index].pieceSpan, text, ref textIndex)) 1521return SegmentFromEnd((merge.RightIndex, merge.RightLen), text, ref textIndex) && SegmentFromEnd((merge.LeftIndex, merge.LeftLen), text, ref textIndex);