2 references to Decode
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
616
public override string Decode(IEnumerable<int> ids) =>
Decode
(ids, considerSpecialTokens: true);
Microsoft.ML.Tokenizers.Tests (1)
BpeTests.cs (1)
267
Assert.Equal(decodedTokensWithoutUnknownToken, bpe.
Decode
(encodingIds, considerSpecialTokens: false));