BuildInputsWithSpecialTokens

8 references to BuildInputsWithSpecialTokens

Microsoft.ML.Tokenizers.Tests (8)

BertTokenizerTests.cs (8)

379OperationStatus status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out int written); 384status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written); 394status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2); 399status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2); 417status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written); 422status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written); 434status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2); 439status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2);