VocabSize

1 write to VocabSize

Microsoft.ML.GenAI.LLaMA (1)

LlamaConfig.cs (1)

37this.VocabSize = 128256;

5 references to VocabSize

Microsoft.ML.GenAI.LLaMA (5)

LlamaForCausalLM.cs (3)

31_vocabSize = config.VocabSize; 41lm_head = nn.Linear(config.HiddenSize, config.VocabSize, hasBias: false, dtype: config.DType); 45lm_head = nn.Linear(config.HiddenSize, config.VocabSize, hasBias: false, dtype: config.DType);

Module\LlamaModel.cs (2)

31this._vocabSize = config.VocabSize; 33this.embed_tokens = nn.Embedding(config.VocabSize, config.HiddenSize, padding_idx: this._paddingIdx, dtype: config.DType);