StoreNormalizedText

5 references to StoreNormalizedText

Microsoft.ML.Tokenizers (5)

Model\SentencePieceUnigramModel.cs (5)

673StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex); 910StoreNormalizedText(normalizationSpan, ref normalizedString!, ref normalizedStringIndex); 1212StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex); 1448StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex); 1497StoreNormalizedText(normalizationSpan, ref normalizedString, ref normalizedStringIndex);