2 references to EnglishRobertaTokenizer
Microsoft.ML.Tokenizers (2)
Model\EnglishRobertaTokenizer.cs (2)
49
=> new
EnglishRobertaTokenizer
(vocabularyPath, mergePath, highestOccurrenceMappingPath);
70
=> new
EnglishRobertaTokenizer
(vocabularyPath, mergePath, highestOccurrenceMappingPath, preTokenizer, normalizer, filterUnsupportedChars);