StoreNormalizedText

5 references to StoreNormalizedText

Microsoft.ML.Tokenizers (5)

Model\SentencePieceUnigramModel.cs (5)

398StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex); 624StoreNormalizedText(normalizationSpan, ref normalizedString!, ref normalizedStringIndex); 926StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex); 1152StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex); 1201StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex);