_vocabReverse

10 references to _vocabReverse

Microsoft.ML.Tokenizers (10)

Model\SentencePieceTokenizer.cs (10)

52_vocabReverse.Add(i, piece.Piece); 382if (_vocabReverse.TryGetValue(id, out string? token)) 406if (_vocabReverse.TryGetValue(id, out string? token)) 1579else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1588else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1639if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1835else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1853else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1928if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 2169=> _vocabReverse.TryGetValue(id, out string? token) ? token : text.Slice(index, length).ToString();