1 reference to GetTokenString
Microsoft.ML.Tokenizers (1)
Model\SentencePieceBpeModel.cs (1)
236
GetTokenString
(id, symbols[index].pieceSpan.Index, symbols[index].pieceSpan.Length, text),