1 write to ByteLevel
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
315ByteLevel = byteLevel;
9 references to ByteLevel
Microsoft.ML.Tokenizers (9)
Model\BPETokenizer.cs (9)
788if (ByteLevel) 906if (ByteLevel) 1321if (ByteLevel) 1434if (ByteLevel) 1460if (ByteLevel) 1481if (ByteLevel) 1514if (ByteLevel) 1541if (ByteLevel) 1563if (ByteLevel)