1 instantiation of BertNormalizer
Microsoft.ML.Tokenizers (1)
Model\BertTokenizer.cs (1)
763
options.Normalizer ??= options.ApplyBasicTokenization ? new
BertNormalizer
(options.LowerCaseBeforeTokenization, options.IndividuallyTokenizeCjk, options.RemoveNonSpacingMarks) : null;
1 reference to BertNormalizer
Microsoft.ML.Tokenizers (1)
Normalizer\BertNormalizer.cs (1)
137
/// Initializes a new instance of the <see cref="
BertNormalizer
"/> class.