2 references to Create
Microsoft.ML.Tokenizers.Tests (2)
CodeGenTests.cs (2)
31
return CodeGenTokenizer.
Create
(vocabStream, mergesStream, addPrefixSpace, bos, eos);
42
return CodeGenTokenizer.
Create
(vocabStream, mergesStream);