EncodeByte

2 references to EncodeByte

Microsoft.ML.Tokenizers (2)

Model\SentencePieceBaseModel.cs (2)

232EncodeByte(current, OneByteUtf8EncodingMaxId, ByteCodeToIdOffset, ref bytesCount, ref bytesPoolArray, ref sb); 277EncodeByte(current, OneByteUtf8EncodingMaxId, ByteCodeToIdOffset, ref bytesCount, ref bytesPoolArray, ref sb);