1 write to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
307
FuseUnknownTokens
= fuseUnknownTokens;
1 reference to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
1263
if (
FuseUnknownTokens
)