1 reference to GetTokenString
Microsoft.ML.Tokenizers (1)
Model\SentencePieceTokenizer.cs (1)
354
GetTokenString
(id, symbols[index].pieceSpan.Index, symbols[index].pieceSpan.Length, text),