1 write to q_layernorm
Microsoft.ML.GenAI.Phi (1)
Module\Phi2Attention.cs (1)
79
this.
q_layernorm
= nn.LayerNorm(this._hiddenSize / this._numAttentionHeads, eps: config.LayerNormEps, elementwise_affine: true, dtype: config.Dtype);
1 reference to q_layernorm
Microsoft.ML.GenAI.Phi (1)
Module\Phi2Attention.cs (1)
117
queryStates = this.
q_layernorm
!.forward(queryStates);