Decode

5 references to Decode

Microsoft.ML.GenAI.Mistral.Tests (1)

Mistral_7B_Instruct_V0_3Tests.cs (1)

129var decodeToString = tokenizer.Decode(tokenizeIds, considerSpecialTokens: true);

Microsoft.ML.GenAI.Phi.Tests (1)

Phi3Tests.cs (1)

143var decodeToString = tokenizer.Decode(tokenizeIds, considerSpecialTokens: true);

Microsoft.ML.Tokenizers (1)

Model\SentencePieceTokenizer.cs (1)

1531=> Decode(ids, considerSpecialTokens: false);

Microsoft.ML.Tokenizers.Tests (2)

LlamaTests.cs (2)

839Assert.Equal(textWithSpecialTokens, tokenizer.Decode(expectedIds, considerSpecialTokens: true)); 877Assert.Equal(textWithSpecialTokens, tokenizerWithSuffix.Decode(expectedIdsWithSuffix, considerSpecialTokens: true));