1 write to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
240
FuseUnknownTokens
= fuseUnknownTokens;
1 reference to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
941
if (
FuseUnknownTokens
)