1 reference to CreateUnigramTokenizerWithSpecialTokens
Microsoft.ML.Tokenizers.Tests (1)
UnigramTests.cs (1)
21
private static SentencePieceTokenizer _unigramTokenizerWithSpecialTokens =
CreateUnigramTokenizerWithSpecialTokens
();