2 references to EnglishRobertaTokenizer
Microsoft.ML.Tokenizers (2)
Model\EnglishRobertaTokenizer.cs (2)
46
=> new
EnglishRobertaTokenizer
(vocabularyPath, mergePath, highestOccurrenceMappingPath);
64
=> new
EnglishRobertaTokenizer
(vocabularyPath, mergePath, highestOccurrenceMappingPath, preTokenizer, normalizer, filterUnsupportedChars);