1 write to scale
Microsoft.ML.TorchSharp (1)
AutoFormerV2\Attention.cs (1)
51
this.
scale
= System.Math.Pow(keyChannels, -0.5);
1 reference to scale
Microsoft.ML.TorchSharp (1)
AutoFormerV2\Attention.cs (1)
116
var attn = (torch.matmul(q, k.transpose(-2, -1)) * this.
scale
) + this.attention_biases[RangeUtil.ToTensorIndex(..), this.attention_bias_idxs];