1 reference to CreateForModelAsync
Microsoft.ML.Tokenizers.Tests (1)
TiktokenTests.cs (1)
85
tokenizer = await TiktokenTokenizer.
CreateForModelAsync
("gpt-3.5-turbo", stream);