CreateTokenTypeIdsFromSequences

4 references to CreateTokenTypeIdsFromSequences

Microsoft.ML.Tokenizers.Tests (4)

BertTokenizerTests.cs (4)

572OperationStatus status = bertTokenizer.CreateTokenTypeIdsFromSequences(ids1, ids1Span, out int written); 577status = bertTokenizer.CreateTokenTypeIdsFromSequences(ids1, ids1Span, out written); 587status = bertTokenizer.CreateTokenTypeIdsFromSequences(ids1, ids1Span, out written, ids2); 592status = bertTokenizer.CreateTokenTypeIdsFromSequences(ids1, ids1Span, out written, ids2);