EncodeByte

2 references to EncodeByte

Microsoft.ML.Tokenizers (2)

Model\SentencePieceBaseModel.cs (2)

188EncodeByte(current, OneByteUtf8EncodingMaxId, ByteCodeToIdOffset, ref bytesCount, ref bytesPoolArray, ref sb); 233EncodeByte(current, OneByteUtf8EncodingMaxId, ByteCodeToIdOffset, ref bytesCount, ref bytesPoolArray, ref sb);