3 references to Decode
Microsoft.ML.Tokenizers (1)
Model\BPETokenizer.cs (1)
640
=>
Decode
(ids, destination, considerSpecialTokens: true, out idsConsumed, out charsWritten);
Microsoft.ML.Tokenizers.Tests (2)
BpeTests.cs (2)
306
status = bpe.
Decode
(ids, destinationBuffer.AsSpan().Slice(0, i), considerSpecialTokens, out idsConsumed, out charactersWritten);
316
status = bpe.
Decode
(ids, destinationBuffer.AsSpan(), considerSpecialTokens, out idsConsumed, out charactersWritten);