2 references to CreateEmptyBpe
Microsoft.ML.Tokenizers.Tests (2)
NormalizerTests.cs (1)
64
Tokenizer tokenizer = BpeTests.
CreateEmptyBpe
(preTokenizer: null, normalizer);
PreTokenizerTests.cs (1)
57
Tokenizer tokenizer = BpeTests.
CreateEmptyBpe
(normalizer: null, preTokenizer: preTokenizer);