3 implementations of IEvaluator
Microsoft.Extensions.AI.Evaluation (1)
CompositeEvaluator.cs (1)
18public sealed class CompositeEvaluator : IEvaluator
Microsoft.Extensions.AI.Evaluation.Integration.Tests (1)
TestEvaluator.cs (1)
12internal class TestEvaluator : IEvaluator
Microsoft.Extensions.AI.Evaluation.Quality (1)
ChatConversationEvaluator.cs (1)
15public abstract class ChatConversationEvaluator : IEvaluator
80 references to IEvaluator
Microsoft.Extensions.AI.Evaluation (33)
CompositeEvaluator.cs (16)
15/// An <see cref="IEvaluator"/> that composes other <see cref="IEvaluator"/>s to execute multiple (concurrent) 22/// composed <see cref="IEvaluator"/>s. 26private readonly IReadOnlyList<IEvaluator> _evaluators; 30/// <see cref="IEvaluator"/>s. 32/// <param name="evaluators">An array of <see cref="IEvaluator"/>s that are to be composed.</param> 33public CompositeEvaluator(params IEvaluator[] evaluators) 34: this(evaluators as IEnumerable<IEvaluator>) 40/// <see cref="IEvaluator"/>s. 42/// <param name="evaluators">An enumeration of <see cref="IEvaluator"/>s that are to be composed.</param> 43public CompositeEvaluator(IEnumerable<IEvaluator> evaluators) 49foreach (IEvaluator evaluator in evaluators) 83/// <see cref="IEvaluator"/>s use an AI model to perform evaluation. 92/// <see cref="IEvaluationTokenCounter"/> that should be used if one or more composed <see cref="IEvaluator"/>s use 97/// <see cref="IEvaluator"/>s may need to accurately evaluate the supplied <paramref name="modelResponse"/>. 135async ValueTask<EvaluationResult> EvaluateAsync(IEvaluator e)
EvaluationContext.cs (1)
8/// history) that an <see cref="IEvaluator"/> may need to accurately evaluate a supplied response.
EvaluatorExtensions.cs (13)
12/// Extension methods for <see cref="IEvaluator"/>. 23/// <see cref="EvaluationResult"/> should match <see cref="IEvaluator.EvaluationMetricNames"/>. 30/// <param name="evaluator">The <see cref="IEvaluator"/> that should perform the evaluation.</param> 45this IEvaluator evaluator, 63/// <see cref="EvaluationResult"/> should match <see cref="IEvaluator.EvaluationMetricNames"/>. 70/// <param name="evaluator">The <see cref="IEvaluator"/> that should perform the evaluation.</param> 88this IEvaluator evaluator, 108/// <see cref="EvaluationResult"/> should match <see cref="IEvaluator.EvaluationMetricNames"/>. 115/// <param name="evaluator">The <see cref="IEvaluator"/> that should perform the evaluation.</param> 130this IEvaluator evaluator, 153/// <see cref="EvaluationResult"/> should match <see cref="IEvaluator.EvaluationMetricNames"/>. 160/// <param name="evaluator">The <see cref="IEvaluator"/> that should perform the evaluation.</param> 178this IEvaluator evaluator,
IEvaluationTokenCounter.cs (1)
11/// <see cref="IEvaluator"/>s that rely on an AI model to perform evaluations can use
IEvaluator.cs (2)
17/// <see cref="IEvaluator"/>. 45/// <see cref="IEvaluator"/> may need to accurately evaluate the supplied <paramref name="modelResponse"/>.
Microsoft.Extensions.AI.Evaluation.Integration.Tests (10)
AdditionalContextTests.cs (2)
33IEvaluator groundednessEvaluator = new GroundednessEvaluator(); 34IEvaluator equivalenceEvaluator = new EquivalenceEvaluator();
EndToEndTests.cs (3)
37IEvaluator rtcEvaluator = new RelevanceTruthAndCompletenessEvaluator(options); 38IEvaluator coherenceEvaluator = new CoherenceEvaluator(); 39IEvaluator fluencyEvaluator = new FluencyEvaluator();
RelevanceTruthAndCompletenessEvaluatorTests.cs (2)
33IEvaluator rtcEvaluatorWithoutReasoning = new RelevanceTruthAndCompletenessEvaluator(); 43IEvaluator rtcEvaluatorWithReasoning = new RelevanceTruthAndCompletenessEvaluator(options);
ResultsTests.cs (1)
22public static ReportingConfiguration CreateReportingConfiguration(IEvaluator evaluator) =>
TestEvaluator.cs (2)
18IReadOnlyCollection<string> IEvaluator.EvaluationMetricNames 26async ValueTask<EvaluationResult> IEvaluator.EvaluateAsync(
Microsoft.Extensions.AI.Evaluation.Quality (20)
ChatConversationEvaluator.cs (13)
13/// An <see langword="abstract"/> base class that can be used to implement an AI-based <see cref="IEvaluator"/>. 21/// Gets a value indicating whether this <see cref="IEvaluator"/> considers the entire conversation history (in 25/// <see langword="true"/> if this <see cref="IEvaluator"/> considers the entire conversation history as part of 31/// Gets the system prompt that this <see cref="IEvaluator"/> uses when performing evaluations. 188/// supplied <paramref name="message"/> as part of the evaluation prompt that this <see cref="IEvaluator"/> uses. 199/// <see cref="IEvaluationTokenCounter"/> that this <see cref="IEvaluator"/> uses to perform the evaluation. 254/// prompt that this <see cref="IEvaluator"/> uses. 279/// Renders the information present in the supplied parameters into a prompt that this <see cref="IEvaluator"/> 292/// <paramref name="includedHistory"/>) that this <see cref="IEvaluator"/> may need to accurately evaluate the 306/// <see cref="EvaluationMetric"/>s supported by this <see cref="IEvaluator"/>. 314/// <see cref="EvaluationMetric"/>s supported by this <see cref="IEvaluator"/>. 325/// <see cref="IEvaluationTokenCounter"/> that this <see cref="IEvaluator"/> uses to perform the evaluation. 333/// supported by this <see cref="IEvaluator"/>.
CoherenceEvaluator.cs (1)
11/// An <see cref="IEvaluator"/> that evaluates the 'Coherence' of a response produced by an AI model.
EquivalenceEvaluator.cs (1)
13/// An <see cref="IEvaluator"/> that evaluates the 'Equivalence' of a response produced by an AI model.
FluencyEvaluator.cs (1)
11/// An <see cref="IEvaluator"/> that evaluates the 'Fluency' of a response produced by an AI model.
GroundednessEvaluator.cs (1)
13/// An <see cref="IEvaluator"/> that evaluates the 'Groundedness' of a response produced by an AI model.
RelevanceTruthAndCompletenessEvaluator.cs (1)
19/// An <see cref="IEvaluator"/> that evaluates the 'Relevance', 'Truth' and 'Completeness' of a response produced by an
SingleNumericMetricEvaluator.cs (2)
12/// An <see langword="abstract"/> base class that can be used to implement an AI-based <see cref="IEvaluator"/> that 22/// <see cref="IEvaluator"/>.
Microsoft.Extensions.AI.Evaluation.Reporting (15)
ReportingConfiguration.cs (6)
15/// <see cref="IEvaluator"/>s that should be invoked, the <see cref="Evaluation.ChatConfiguration"/> that should be 16/// used by these <see cref="IEvaluator"/>s, how the resulting <see cref="ScenarioRunResult"/>s should be persisted, 22/// Gets the set of <see cref="IEvaluator"/>s that should be invoked to evaluate AI responses. 24public IReadOnlyList<IEvaluator> Evaluators { get; } 93/// The set of <see cref="IEvaluator"/>s that should be invoked to evaluate AI responses. 124IEnumerable<IEvaluator> evaluators,
ScenarioRun.cs (3)
80/// <see cref="IEvaluationTokenCounter"/> that are used by AI-based <see cref="IEvaluator"/>s that are invoked as 95IEnumerable<IEvaluator> evaluators, 120/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the
ScenarioRunExtensions.cs (4)
23/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 51/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the 77/// Additional contextual information that the <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> 110/// <see cref="IEvaluator"/>s included in this <see cref="ScenarioRun"/> may need to accurately evaluate the
Storage\DiskBasedReportingConfiguration.cs (2)
24/// The set of <see cref="IEvaluator"/>s that should be invoked to evaluate AI responses. 49IEnumerable<IEvaluator> evaluators,
Microsoft.Extensions.AI.Evaluation.Reporting.Azure (2)
Storage\AzureStorageReportingConfiguration.cs (2)
25/// The set of <see cref="IEvaluator"/>s that should be invoked to evaluate AI responses. 54IEnumerable<IEvaluator> evaluators,