_vocabReverse

10 references to _vocabReverse

Microsoft.ML.Tokenizers (10)

Model\SentencePieceTokenizer.cs (10)

52_vocabReverse.Add(i, piece.Piece); 383if (_vocabReverse.TryGetValue(id, out string? token)) 407if (_vocabReverse.TryGetValue(id, out string? token)) 1580else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1589else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1640if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1836else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1854else if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 1929if (_vocabReverse.TryGetValue(enumerator.Current, out string? token)) 2170=> _vocabReverse.TryGetValue(id, out string? token) ? token : text.Slice(index, length).ToString();