1 write to BeginningOfSentenceId
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
328BeginningOfSentenceId = aId;
2 references to BeginningOfSentenceId
Microsoft.ML.Tokenizers (2)
Model\BPETokenizer.cs (2)
457tokens.Add(new EncodedToken(BeginningOfSentenceId, BeginningOfSentenceToken, new Range(0, 0))); 517ids.Add(BeginningOfSentenceId);