1 reference to
Microsoft.ML.Tokenizers (1)
Model\SentencePieceBpeTokenizer.cs (1)
1657
Debug.Assert(sb
[
suffixIndex] == SentencePieceNormalizer.DummyPrefix);