2 references to NormalizePrefix
Microsoft.ML.Tokenizers (2)
Normalizer\SentencePieceNormalizer.cs (2)
349int p = NormalizePrefix(input, out Memory<byte> normalizedPrefix); 388int p = NormalizePrefix(input, out Memory<byte> normalizedPrefix);