1 instantiation of ScenarioRun
Microsoft.Extensions.AI.Evaluation.Reporting (1)
ReportingConfiguration.cs (1)
252return new ScenarioRun(
125 references to ScenarioRun
Microsoft.Extensions.AI.Evaluation.Integration.Tests (22)
QualityEvaluatorTests.cs (4)
79await using ScenarioRun scenarioRun = 110await using ScenarioRun scenarioRun = 140await using ScenarioRun scenarioRun = 164await using ScenarioRun scenarioRun =
ResultsTests.cs (10)
165await using ScenarioRun scenarioRun = 193await using ScenarioRun scenarioRun = 246await using ScenarioRun scenarioRun = 302await using ScenarioRun scenarioRun = 352await using ScenarioRun scenarioRun = 388await using ScenarioRun scenarioRun = 452await using ScenarioRun scenarioRun = 505await using ScenarioRun scenarioRun = 564await using ScenarioRun scenarioRun = 597await using ScenarioRun scenarioRun =
SafetyEvaluatorTests.cs (8)
114await using ScenarioRun scenarioRun = 160await using ScenarioRun scenarioRun = 219await using ScenarioRun scenarioRun = 246await using ScenarioRun scenarioRun = 273await using ScenarioRun scenarioRun = 313await using ScenarioRun scenarioRun = 366await using ScenarioRun scenarioRun = 392await using ScenarioRun scenarioRun =
Microsoft.Extensions.AI.Evaluation.Reporting (100)
ChatDetails.cs (1)
12/// <see cref="ScenarioRun"/>.
ChatTurnDetails.cs (1)
15/// particular <see cref="ScenarioRun"/>.
Defaults.cs (1)
23/// <see cref="ScenarioRun"/> via
IResponseCacheProvider.cs (9)
12/// <see cref="ScenarioRun"/>. 17/// responses associated with each <see cref="ScenarioRun"/> are stored in the <see cref="IDistributedCache"/> that is 20/// <see cref="ScenarioRun"/> use the cached responses instead of invoking the AI model to generate new ones. Bypassing 27/// <see cref="ScenarioRun"/>. 29/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 30/// <param name="iterationName">The <see cref="ScenarioRun.IterationName"/>.</param> 34/// <see cref="ScenarioRun"/>. 42/// Deletes cached AI responses for all <see cref="ScenarioRun"/>s. 49/// Deletes expired cache entries for all <see cref="ScenarioRun"/>s.
IResultStore.cs (24)
16/// Returns <see cref="ScenarioRunResult"/>s for <see cref="ScenarioRun"/>s filtered by the specified 25/// The <see cref="ScenarioRun.ExecutionName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 26/// If omitted, all <see cref="ScenarioRun.ExecutionName"/>s are considered. 29/// The <see cref="ScenarioRun.ScenarioName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 30/// If omitted, all <see cref="ScenarioRun.ScenarioName"/>s that are in scope based on the specified 34/// The <see cref="ScenarioRun.IterationName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 35/// If omitted, all <see cref="ScenarioRun.IterationName"/>s that are in scope based on the specified 55/// Deletes <see cref="ScenarioRunResult"/>s for <see cref="ScenarioRun"/>s filtered by the specified 64/// The <see cref="ScenarioRun.ExecutionName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 65/// If omitted, all <see cref="ScenarioRun.ExecutionName"/>s are considered. 68/// The <see cref="ScenarioRun.ScenarioName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 69/// If omitted, all <see cref="ScenarioRun.ScenarioName"/>s that are in scope based on the specified 73/// The <see cref="ScenarioRun.IterationName"/> by which the <see cref="ScenarioRunResult"/>s should be filtered. 74/// If omitted, all <see cref="ScenarioRun.IterationName"/>s that are in scope based on the specified 86/// Gets the <see cref="ScenarioRun.ExecutionName"/>s of the most recent <paramref name="count"/> executions from 89/// <param name="count">The number of <see cref="ScenarioRun.ExecutionName"/>s to retrieve.</param> 92/// The <see cref="ScenarioRun.ExecutionName"/>s of the most recent <paramref name="count"/> executions from the 100/// Gets the <see cref="ScenarioRun.ScenarioName"/>s present in the execution with the specified 103/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 106/// The <see cref="ScenarioRun.ScenarioName"/>s present in the execution with the specified 114/// Gets the <see cref="ScenarioRun.IterationName"/>s present in the scenario with the specified 117/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 118/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 121/// The <see cref="ScenarioRun.IterationName"/>s present in the scenario with the specified
ReportingConfiguration.cs (11)
14/// Represents the configuration for a set of <see cref="ScenarioRun"/>s that defines the set of 75/// See <see cref="ScenarioRun.ExecutionName"/> for more information about this concept. 91/// Gets an optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using this 120/// The name of the current execution. See <see cref="ScenarioRun.ExecutionName"/> for more information about this 131/// A optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using this 165/// Creates a new <see cref="ScenarioRun"/> with the specified <paramref name="scenarioName"/> and 168/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 170/// The <see cref="ScenarioRun.IterationName"/>. Uses default value <c>"1"</c> if omitted. 177/// A optional set of text tags applicable to this <see cref="ScenarioRun"/>. 181/// A new <see cref="ScenarioRun"/> with the specified <paramref name="scenarioName"/> and 185public async ValueTask<ScenarioRun> CreateScenarioRunAsync(
ScenarioRun.cs (10)
24/// Gets the name of the scenario that this <see cref="ScenarioRun"/> represents. 32/// part of an evaluation. In this case, the <see cref="ScenarioName"/> for each <see cref="ScenarioRun"/> in the 40/// Gets the name of the iteration that this <see cref="ScenarioRun"/> represents. 56/// Gets the name of the execution that this <see cref="ScenarioRun"/> represents. 64/// <see cref="ScenarioRun"/>s in a particular evaluation run share the same timestamp for their 85/// part of the evaluation of this <see cref="ScenarioRun"/>. 132/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 149$"The {nameof(ScenarioRun)} with {nameof(ScenarioName)}: {ScenarioName}, {nameof(IterationName)}: {IterationName} and {nameof(ExecutionName)}: {ExecutionName} has already been evaluated. Do not call {nameof(EvaluateAsync)} more than once on a given {nameof(ScenarioRun)}."); 185/// Disposes the <see cref="ScenarioRun"/> and writes the <see cref="ScenarioRunResult"/> to the configured
ScenarioRunExtensions.cs (19)
12/// Extension methods for <see cref="ScenarioRun"/>. 20/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 23/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 31this ScenarioRun scenarioRun, 44/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 51/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 59this ScenarioRun scenarioRun, 74/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 77/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 85this ScenarioRun scenarioRun, 103/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 106/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 114this ScenarioRun scenarioRun, 132/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 139/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 147this ScenarioRun scenarioRun, 166/// <param name="scenarioRun">The <see cref="ScenarioRun"/> of which this evaluation is a part.</param> 173/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 181this ScenarioRun scenarioRun,
ScenarioRunResult.cs (21)
18/// In other words, <see cref="ScenarioRunResult"/> represents the results of evaluating a <see cref="ScenarioRun"/> 20/// <see cref="ScenarioRun.EvaluateAsync(IEnumerable{ChatMessage}, ChatResponse, IEnumerable{Microsoft.Extensions.AI.Evaluation.EvaluationContext}?, CancellationToken)"/> 28/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 29/// <param name="iterationName">The <see cref="ScenarioRun.IterationName"/>.</param> 30/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 37/// The <see cref="Evaluation.EvaluationResult"/> for the <see cref="ScenarioRun"/> corresponding to the 42/// turns involved in the execution of the the <see cref="ScenarioRun"/> corresponding to the 44/// <see cref="IEvaluator"/>s invoked during the execution of the <see cref="ScenarioRun"/> use an LLM. 66/// <param name="scenarioName">The <see cref="ScenarioRun.ScenarioName"/>.</param> 67/// <param name="iterationName">The <see cref="ScenarioRun.IterationName"/>.</param> 68/// <param name="executionName">The <see cref="ScenarioRun.ExecutionName"/>.</param> 76/// The <see cref="Evaluation.EvaluationResult"/> for the <see cref="ScenarioRun"/> corresponding to the 81/// conversation turns involved in the execution of the the <see cref="ScenarioRun"/> corresponding to the 83/// <see cref="IEvaluator"/>s invoked during the execution of the <see cref="ScenarioRun"/> use an LLM. 110/// Gets or sets the <see cref="ScenarioRun.ScenarioName"/>. 115/// Gets or sets the <see cref="ScenarioRun.IterationName"/>. 120/// Gets or sets the <see cref="ScenarioRun.ExecutionName"/>. 147/// Gets or sets the <see cref="Evaluation.EvaluationResult"/> for the <see cref="ScenarioRun"/> corresponding to 152/// <see cref="ScenarioRun.EvaluateAsync(IEnumerable{ChatMessage}, ChatResponse, IEnumerable{Microsoft.Extensions.AI.Evaluation.EvaluationContext}?, CancellationToken)"/> 159/// chat conversation turns involved in the execution of the <see cref="ScenarioRun"/> corresponding to this 164/// <see cref="ScenarioRun"/> use an LLM.
Storage\DiskBasedReportingConfiguration.cs (2)
41/// The name of the current execution. See <see cref="ScenarioRun.ExecutionName"/> for more information about this 52/// A optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using the returned
Storage\DiskBasedResponseCacheProvider.cs (1)
18/// for a particular <see cref="ScenarioRun"/> under the specified <paramref name="storageRootPath"/> on disk.
Microsoft.Extensions.AI.Evaluation.Reporting.Azure (3)
Storage\AzureStorageReportingConfiguration.cs (2)
45/// The name of the current execution. See <see cref="ScenarioRun.ExecutionName"/> for more information about this 56/// A optional set of text tags applicable to all <see cref="ScenarioRun"/>s created using the returned
Storage\AzureStorageResponseCacheProvider.cs (1)
19/// for a particular <see cref="ScenarioRun"/> under an Azure Storage container.