1 reference to Normalize
Microsoft.ML.Tokenizers (1)
Model\SentencePieceUnigramModel.cs (1)
357
int normalizationCount = Normalizer!.
Normalize
(byteSpan.Slice(0, byteCount), ref normalizationSpan, ref normalizedArrayPool);