EvSys

LLMJudgeVerifier

LLM judge - judge_model scores the completion against rubric.

The runner calls the judge model with the rubric + completion and parses a numeric score from the response (usually in a \boxed\{\} or \<score>...\</score> tag).

Attributes

attributejudge_modelstr
= ''
attributerubricstr
= ''
attributekindLiteral['llm_judge']
= 'llm_judge'

Functions

func__init__(self, judge_model='', rubric='', kind='llm_judge') -> None
paramself
paramjudge_modelstr
= ''
paramrubricstr
= ''
paramkindLiteral['llm_judge']
= 'llm_judge'

Returns

None

On this page