EncodeByte

2 references to EncodeByte

Microsoft.ML.Tokenizers (2)

Model\SentencePieceBaseModel.cs (2)

249EncodeByte(current, OneByteUtf8EncodingMaxId, ByteCodeToIdOffset, ref bytesCount, ref bytesPoolArray, ref sb); 294EncodeByte(current, OneByteUtf8EncodingMaxId, ByteCodeToIdOffset, ref bytesCount, ref bytesPoolArray, ref sb);