1 reference to EncodeCodePointToUtf8
Microsoft.ML.Tokenizers (1)
Model\CodeGenTokenizer.cs (1)
1581i += Helpers.EncodeCodePointToUtf8(text, i, ref bytes, ref bytesIndex) - 1;