1 write to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
225
FuseUnknownTokens
= fuseUnknownTokens;
1 reference to FuseUnknownTokens
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
926
if (
FuseUnknownTokens
)