BuildInputsWithSpecialTokens

8 references to BuildInputsWithSpecialTokens

Microsoft.ML.Tokenizers.Tests (8)

BertTokenizerTests.cs (8)

290OperationStatus status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out int written); 295status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written); 305status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2); 310status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2); 328status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written); 333status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written); 345status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2); 350status = bertTokenizer.BuildInputsWithSpecialTokens(ids1, ids1Span, out written, ids2);