1 reference to ParseAddedTokens
Microsoft.ML.Tokenizers (1)
Model\SentencePieceTokenizer.cs (1)
719
Dictionary<string, int> mergedSpecialTokens =
ParseAddedTokens
(root);