1 write to _specialTokensReverse
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
351
_specialTokensReverse
= specialTokens.ToDictionary(kvp => kvp.Value, kvp => kvp.Key);
2 references to _specialTokensReverse
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
840
if (
_specialTokensReverse
?.TryGetValue(id, out string? token) is true)
998
if (
_specialTokensReverse
?.TryGetValue(id, out string? specialToken) is true)