2 references to NormalizeCore
Microsoft.ML.Tokenizers (2)
Normalizer\SentencePieceNormalizer.cs (2)
405return NormalizeCore(chainedBytes.AsSpan(0, chainedLength), ref normalized, ref poolArray); 413return NormalizeCore(input, ref normalized, ref poolArray);