1 write to k_layernorm
Microsoft.ML.GenAI.Phi (1)
Module\Phi2Attention.cs (1)
80
this.
k_layernorm
= nn.LayerNorm(this._hiddenSize / this._numAttentionHeads, eps: config.LayerNormEps, elementwise_affine: true, dtype: config.Dtype);
1 reference to k_layernorm
Microsoft.ML.GenAI.Phi (1)
Module\Phi2Attention.cs (1)
118
keyStates = this.
k_layernorm
!.forward(keyStates);