2 references to NormalizeCore
Microsoft.ML.Tokenizers (2)
Normalizer\SentencePieceNormalizer.cs (2)
146return NormalizeCore(NormalizationChain.Normalize(original.ToString()).AsSpan()); 149return NormalizeCore(original);