1 write to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
312
FuseUnknownTokens
= fuseUnknownTokens;
1 reference to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
1267
if (
FuseUnknownTokens
)