Type of the AutoEvals evaluator function
A Langfuse-compatible evaluator function
import { Factuality, Levenshtein } from 'autoevals';
import { createEvaluatorFromAutoevals } from '@langfuse/client';
const factualityEvaluator = createEvaluatorFromAutoevals(Factuality);
const levenshteinEvaluator = createEvaluatorFromAutoevals(Levenshtein);
await langfuse.experiment.run({
name: "AutoEvals Integration Test",
data: myDataset,
task: myTask,
evaluators: [factualityEvaluator, levenshteinEvaluator]
});
import { Factuality } from 'autoevals';
const factualityEvaluator = createEvaluatorFromAutoevals(
Factuality,
{ model: 'gpt-4o' } // Additional params for AutoEvals
);
await langfuse.experiment.run({
name: "Factuality Test",
data: myDataset,
task: myTask,
evaluators: [factualityEvaluator]
});
Converts an AutoEvals evaluator to a Langfuse-compatible evaluator function.
This adapter function bridges the gap between AutoEvals library evaluators and Langfuse experiment evaluators, handling parameter mapping and result formatting automatically.
AutoEvals evaluators expect
input,output, andexpectedparameters, while Langfuse evaluators useinput,output, andexpectedOutput. This function handles the parameter name mapping.