1 reference to GetSpecialTokens
Microsoft.ML.Tokenizers.Tests (1)
BpeTests.cs (1)
916
bpeOptions.SpecialTokens =
GetSpecialTokens
(root);