fns

In-process verifier functions - the SDK-local registry (D10).

InProcessVerifier(fn_name="tool_calls_match", expected=..., params=...) stores only a reference; the actual function lives here, in the SDK. This is the single source of truth for in-process verifier logic - the remote (Supabase) never stores verifier code, only the fn_name + per-task expected/params.

Ported from the backend api/experiments/verifiers.py _REGISTRY so there is exactly ONE registry. Resolve with get(fn_name); extend with register(fn_name, fn) or the @register_fn("name") decorator.

Reproducibility note: because a task references a verifier by name, the named function resolves against this SDK version - record the SDK version on the experiment/run (see EXPERIMENT_AGENT_DESIGN.md, D10 / Q-J).

attributeVerifierFn

= Callable[[str, Any, dict], float]

attribute__all__

= ['VerifierFn', 'tool_calls_match', 'exact_match', 'contains', 'regex_match', 'get', 'register', 'register_fn', 'list_fns']

func_normalize_json(s) -> dict | None

Best-effort: strip code fences, find first JSON object, parse.

paramsstr

Returns

dict | None

functool_calls_match(model_output, expected, params) -> float

1.0 if the model's emitted tool call matches expected; else 0.0.

Both must parse as JSON dicts; tool + action must match; optional ref/text must match if present in expected; coordinate must be within params['coordinate_tolerance'] (default 25).

parammodel_outputstr

paramexpectedAny

paramparamsdict

Returns

float

funcexact_match(model_output, expected, params) -> float

Strict text equality (after whitespace strip + optional case-fold).

parammodel_outputstr

paramexpectedAny

paramparamsdict

Returns

float

funccontains(model_output, expected, params) -> float

1.0 if expected is a substring of model_output.

parammodel_outputstr

paramexpectedAny

paramparamsdict

Returns

float

funcregex_match(model_output, expected, params) -> float

1.0 if regex expected matches model_output.

parammodel_outputstr

paramexpectedAny

paramparamsdict

Returns

float

funcget(fn_name) -> VerifierFn

paramfn_namestr

Returns

evsys_sdk.verifiers.fns.VerifierFn

funcregister(fn_name, fn) -> None

paramfn_namestr

paramfnVerifierFn

Returns

None

funcregister_fn(fn_name) -> Callable[[VerifierFn], VerifierFn]

Decorator form of register.

paramfn_namestr

Returns

typing.Callable[[evsys_sdk.verifiers.fns.VerifierFn], evsys_sdk.verifiers.fns.VerifierFn]

funclist_fns() -> list[str]

Returns

list[str]