1 write to BeginningOfSentenceToken
Microsoft.ML.Tokenizers.Tests (1)
BpeTests.cs (1)
926
bpeOptions.
BeginningOfSentenceToken
= "<｜begin▁of▁sentence｜>";
1 reference to BeginningOfSentenceToken
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
191
options.
BeginningOfSentenceToken
,