1 write to SpecialTokens
Microsoft.ML.Tokenizers (1)
Model\WordPieceTokenizer.cs (1)
49
SpecialTokens
= specialTokens;
2 references to SpecialTokens
Microsoft.ML.Tokenizers.Tests (2)
WordPieceTests.cs (2)
60
Assert.Null(tokenizer.
SpecialTokens
);
190
Assert.Equal(specialTokens, tokenizer.
SpecialTokens
);