2 references to NormalizePrefix
Microsoft.ML.Tokenizers (2)
Normalizer\SentencePieceNormalizer.cs (2)
430int p = NormalizePrefix(input, out Memory<byte> normalizedPrefix); 469int p = NormalizePrefix(input, out Memory<byte> normalizedPrefix);