3 references to WordPieceTokenizer
Microsoft.ML.Tokenizers (3)
Model\BertTokenizer.cs (1)
30
BertOptions options) :
base
(vocab, vocabReverse, options)
Model\WordPieceTokenizer.cs (2)
165
return new
WordPieceTokenizer
(vocab, vocabReverse, options);
229
return new
WordPieceTokenizer
(vocab, vocabReverse, options);