1 reference to None
Microsoft.ML.Tokenizers (1)
Model\TiktokenTokenizer.cs (1)
1109
if (encoder == ModelEncoding.
None
)