1 instantiation of BertNormalizer
Microsoft.ML.Tokenizers (1)
Model\BertTokenizer.cs (1)
763options.Normalizer ??= options.ApplyBasicTokenization ? new BertNormalizer(options.LowerCaseBeforeTokenization, options.IndividuallyTokenizeCjk, options.RemoveNonSpacingMarks) : null;
1 reference to BertNormalizer
Microsoft.ML.Tokenizers (1)
Normalizer\BertNormalizer.cs (1)
137/// Initializes a new instance of the <see cref="BertNormalizer"/> class.