2 writes to ContinuingSubwordPrefix
Microsoft.ML.Tokenizers.Tests (2)
BpeTests.cs (2)
276
ContinuingSubwordPrefix
= continuingSubwordPrefix,
903
bpeOptions.
ContinuingSubwordPrefix
= continuingSubwordPrefixElement.GetString();
1 reference to ContinuingSubwordPrefix
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
187
options.
ContinuingSubwordPrefix
,