1 instantiation of ScenarioRun
Microsoft.Extensions.AI.Evaluation.Reporting (1)
ReportingConfiguration.cs (1)
253return new ScenarioRun(
138 references to ScenarioRun
Microsoft.Extensions.AI.Evaluation.Integration.Tests (35)
AgentQualityEvaluatorTests.cs (4)
89await using ScenarioRun scenarioRun = 112await using ScenarioRun scenarioRun = 145await using ScenarioRun scenarioRun = 169await using ScenarioRun scenarioRun =
NLPEvaluatorTests.cs (4)
52await using ScenarioRun scenarioRun = 78await using ScenarioRun scenarioRun = 105await using ScenarioRun scenarioRun = 131await using ScenarioRun scenarioRun =
QualityEvaluatorTests.cs (4)
85await using ScenarioRun scenarioRun = 124await using ScenarioRun scenarioRun = 162await using ScenarioRun scenarioRun = 197await using ScenarioRun scenarioRun =
ResultsTests.cs (10)
166await using ScenarioRun scenarioRun = 194await using ScenarioRun scenarioRun = 247await using ScenarioRun scenarioRun = 303await using ScenarioRun scenarioRun = 353await using ScenarioRun scenarioRun = 389await using ScenarioRun scenarioRun = 453await using ScenarioRun scenarioRun = 506await using ScenarioRun scenarioRun = 565await using ScenarioRun scenarioRun = 598await using ScenarioRun scenarioRun =
SafetyEvaluatorTests.cs (13)
162await using ScenarioRun scenarioRun = 174await using ScenarioRun scenarioRun = 181private static async Task EvaluateConversationWithSingleTurn(ScenarioRun scenarioRun) 240await using ScenarioRun scenarioRun = 252await using ScenarioRun scenarioRun = 259private static async Task EvaluateConversationWithMultipleTurns(ScenarioRun scenarioRun) 331await using ScenarioRun scenarioRun = 367await using ScenarioRun scenarioRun = 403await using ScenarioRun scenarioRun = 452await using ScenarioRun scenarioRun = 514await using ScenarioRun scenarioRun = 543await using ScenarioRun scenarioRun = 584await using ScenarioRun scenarioRun =
Microsoft.Extensions.AI.Evaluation.Reporting (100)
ChatDetails.cs (1)
12/// <see cref="ScenarioRun"/>.
ChatTurnDetails.cs (1)
11/// particular <see cref="ScenarioRun"/>.
Defaults.cs (1)
23/// <see cref="ScenarioRun"/> via
IEvaluationResponseCacheProvider.cs (9)
12/// <see cref="ScenarioRun"/>. 17/// the AI responses associated with each <see cref="ScenarioRun"/> are stored in the <see cref="IDistributedCache"/> 20/// same <see cref="ScenarioRun"/> use the cached responses instead of invoking the AI model to generate new ones. 27/// <see cref="ScenarioRun"/> with the supplied <paramref name="scenarioName"/> and 30/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 31/// <param name="iterationName">The <see cref="ScenarioRun.IterationName"/>.</param> 35/// <see cref="ScenarioRun"/> with the supplied <paramref name="scenarioName"/> and 44/// Deletes cached AI responses for all <see cref="ScenarioRun"/>s. 51/// Deletes expired cache entries for all <see cref="ScenarioRun"/>s.
IEvaluationResultStore.cs (24)
16/// Returns <see cref="ScenarioRunResult"/>s for <see cref="ScenarioRun"/>s filtered by the specified 25/// The <see cref="ScenarioRun.ExecutionName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 26/// If omitted, all <see cref="ScenarioRun.ExecutionName"/>s are considered. 29/// The <see cref="ScenarioRun.ScenarioName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 30/// If omitted, all <see cref="ScenarioRun.ScenarioName"/>s that are in scope based on the specified 34/// The <see cref="ScenarioRun.IterationName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 35/// If omitted, all <see cref="ScenarioRun.IterationName"/>s that are in scope based on the specified 55/// Deletes <see cref="ScenarioRunResult"/>s for <see cref="ScenarioRun"/>s filtered by the specified 64/// The <see cref="ScenarioRun.ExecutionName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 65/// If omitted, all <see cref="ScenarioRun.ExecutionName"/>s are considered. 68/// The <see cref="ScenarioRun.ScenarioName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 69/// If omitted, all <see cref="ScenarioRun.ScenarioName"/>s that are in scope based on the specified 73/// The <see cref="ScenarioRun.IterationName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 74/// If omitted, all <see cref="ScenarioRun.IterationName"/>s that are in scope based on the specified 86/// Gets the <see cref="ScenarioRun.ExecutionName"/>s of the most recent <paramref name="count"/> executions from 89/// <param name="count">The number of <see cref="ScenarioRun.ExecutionName"/>s to retrieve.</param> 92/// The <see cref="ScenarioRun.ExecutionName"/>s of the most recent <paramref name="count"/> executions from the 100/// Gets the <see cref="ScenarioRun.ScenarioName"/>s present in the execution with the specified 103/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 106/// The <see cref="ScenarioRun.ScenarioName"/>s present in the execution with the specified 114/// Gets the <see cref="ScenarioRun.IterationName"/>s present in the scenario with the specified 117/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 118/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 121/// The <see cref="ScenarioRun.IterationName"/>s present in the scenario with the specified
ReportingConfiguration.cs (11)
14/// Represents the configuration for a set of <see cref="ScenarioRun"/>s that defines the set of 77/// See <see cref="ScenarioRun.ExecutionName"/> for more information about this concept. 93/// Gets an optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using this 121/// The name of the current execution. See <see cref="ScenarioRun.ExecutionName"/> for more information about this 132/// A optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using this 164/// Creates a new <see cref="ScenarioRun"/> with the specified <paramref name="scenarioName"/> and 167/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 169/// The <see cref="ScenarioRun.IterationName"/>. Uses default value <c>"1"</c> if omitted. 176/// A optional set of text tags applicable to this <see cref="ScenarioRun"/>. 180/// A new <see cref="ScenarioRun"/> with the specified <paramref name="scenarioName"/> and 186public async ValueTask<ScenarioRun> CreateScenarioRunAsync(
ScenarioRun.cs (10)
26/// Gets the name of the scenario that this <see cref="ScenarioRun"/> represents. 34/// part of an evaluation. In this case, the <see cref="ScenarioName"/> for each <see cref="ScenarioRun"/> in the 44/// Gets the name of the iteration that this <see cref="ScenarioRun"/> represents. 60/// Gets the name of the execution that this <see cref="ScenarioRun"/> represents. 68/// <see cref="ScenarioRun"/>s in a particular evaluation run share the same timestamp for their 91/// <see cref="ScenarioRun"/>. 136/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 152$"The {nameof(ScenarioRun)} with {nameof(ScenarioName)}: {ScenarioName}, {nameof(IterationName)}: {IterationName} and {nameof(ExecutionName)}: {ExecutionName} has already been evaluated. Do not call {nameof(EvaluateAsync)} more than once on a given {nameof(ScenarioRun)}."); 187/// Disposes the <see cref="ScenarioRun"/> and writes the <see cref="ScenarioRunResult"/> to the configured
ScenarioRunExtensions.cs (19)
12/// Extension methods for <see cref="ScenarioRun"/>. 20/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 23/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 31this ScenarioRun scenarioRun, 44/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 51/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 59this ScenarioRun scenarioRun, 74/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 77/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 85this ScenarioRun scenarioRun, 98/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 101/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 109this ScenarioRun scenarioRun, 127/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 134/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 142this ScenarioRun scenarioRun, 157/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 164/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 172this ScenarioRun scenarioRun,
ScenarioRunResult.cs (21)
13/// In other words, <see cref="ScenarioRunResult"/> represents the results of evaluating a <see cref="ScenarioRun"/> 15/// <see cref="ScenarioRun.EvaluateAsync(IEnumerable{ChatMessage}, ChatResponse, IEnumerable{Microsoft.Extensions.AI.Evaluation.EvaluationContext}?, CancellationToken)"/> 23/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 24/// <param name="iterationName">The <see cref="ScenarioRun.IterationName"/>.</param> 25/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 32/// The <see cref="Evaluation.EvaluationResult"/> for the <see cref="ScenarioRun"/> corresponding to the 37/// turns involved in the execution of the the <see cref="ScenarioRun"/> corresponding to the 39/// <see cref="IEvaluator"/>s invoked during the execution of the <see cref="ScenarioRun"/> use an LLM. 61/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 62/// <param name="iterationName">The <see cref="ScenarioRun.IterationName"/>.</param> 63/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 71/// The <see cref="Evaluation.EvaluationResult"/> for the <see cref="ScenarioRun"/> corresponding to the 76/// conversation turns involved in the execution of the the <see cref="ScenarioRun"/> corresponding to the 78/// <see cref="IEvaluator"/>s invoked during the execution of the <see cref="ScenarioRun"/> use an LLM. 105/// Gets or sets the <see cref="ScenarioRun.ScenarioName"/>. 110/// Gets or sets the <see cref="ScenarioRun.IterationName"/>. 115/// Gets or sets the <see cref="ScenarioRun.ExecutionName"/>. 136/// Gets or sets the <see cref="Evaluation.EvaluationResult"/> for the <see cref="ScenarioRun"/> corresponding to 141/// <see cref="ScenarioRun.EvaluateAsync(IEnumerable{ChatMessage}, ChatResponse, IEnumerable{Microsoft.Extensions.AI.Evaluation.EvaluationContext}?, CancellationToken)"/> 148/// chat conversation turns involved in the execution of the <see cref="ScenarioRun"/> corresponding to this 153/// <see cref="ScenarioRun"/> use an LLM.
Storage\DiskBasedReportingConfiguration.cs (2)
44/// The name of the current execution. See <see cref="ScenarioRun.ExecutionName"/> for more information about this 55/// A optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using the returned
Storage\DiskBasedResponseCacheProvider.cs (1)
13/// AI responses for a particular <see cref="ScenarioRun"/> under the specified <paramref name="storageRootPath"/> on
Microsoft.Extensions.AI.Evaluation.Reporting.Azure (3)
Storage\AzureStorageReportingConfiguration.cs (2)
44/// The name of the current execution. See <see cref="ScenarioRun.ExecutionName"/> for more information about this 55/// A optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using the returned
Storage\AzureStorageResponseCacheProvider.cs (1)
14/// responses for a particular <see cref="ScenarioRun"/> under an Azure Storage container.