EvSys

data_types

Harbor-compatible data shapes.

The schemas runners consume and dashboards render. Mirrors the production internal types so JSONL files round-trip between this SDK and the internal serving / dashboard stack without conversion.

Two row formats:

  • ChatMessagesRow - SFT example (messages prefix + supervised target)
  • HarborTask - RL task (instruction + verifier spec)

Verifier specs (the data the runner serializes for HarborTask):

  • InProcessVerifier - cheap Python fn lookup
  • E2BVerifier - sandboxed code execution
  • LLMJudgeVerifier - judge model + rubric
  • VerifierPayload - discriminated union of the three (use this in type hints).

These are data shapes (Pydantic / dataclasses) describing the verification plan; the runtime Verifier Protocol in protocols.py is what actually EXECUTES verification. They are deliberately separate concepts.

Multimodal content: messages can carry text + image blocks in OpenAI or Anthropic style. Use image_url_block(url) or image_base64_block(media_type, b64) for convenience.

attributeVerifierPayload
= Union[InProcessVerifier, E2BVerifier, LLMJudgeVerifier]

Discriminated by .kind ∈ {'in_process', 'e2b', 'llm_judge'}.

attribute__all__
= ['TargetFormat', 'ChatMessagesRow', 'HarborTask', 'PromptExample', 'InProcessVerifier', 'E2BVerifier', 'LLMJudgeVerifier', 'VerifierPayload', 'text_block', 'image_url_block', 'image_base64_block', 'block_to_image_src', 'has_images', 'detect_format', 'harbor_task_from_dict', 'chat_messages_row_from_dict', 'prompt_example_from_dict', 'from_dict', 'parse_rows', 'to_dict', 'iter_jsonl']
functext_block(text) -> dict

Standard text block. Works in either OpenAI or Anthropic shape.

paramtextstr

Returns

dict
funcimage_url_block(url, *, detail=None) -> dict

OpenAI-style image block. Works for any vision model that follows the OpenAI multimodal content schema (most do).

paramurlstr
paramdetailstr | None
= None

Returns

dict
funcimage_base64_block(media_type, b64_data) -> dict

Anthropic-style base64 image block.

media_type is the MIME type, e.g. "image/png".

parammedia_typestr
paramb64_datastr

Returns

dict
funcblock_to_image_src(block) -> str | None

Return a renderable image src URL/data-URL for a content block, or None.

Handles OpenAI image_url blocks AND Anthropic image blocks (both URL and base64 variants). Used by dashboard renderers.

paramblockAny

Returns

str | None
funchas_images(row) -> bool

True iff any message in the row carries at least one image block.

paramrowChatMessagesRow

Returns

bool
funcdetect_format(row) -> str

Returns 'chat_messages' | 'harbor_task' | 'prompt_dataset' | 'unknown'.

Checked most-specific first: harbor_task and prompt_dataset have distinctive key pairs; chat_messages is any row carrying messages (the conversation - supervision is decided by the algorithm, not the row).

paramrowAny

Returns

str
func_verifier_from_dict(d) -> VerifierPayload
paramddict

Returns

evsys_sdk.data_types.VerifierPayload
funcharbor_task_from_dict(d) -> HarborTask
paramddict

Returns

evsys_sdk.data_types.HarborTask
funcchat_messages_row_from_dict(d) -> ChatMessagesRow
paramddict

Returns

evsys_sdk.data_types.ChatMessagesRow
funcprompt_example_from_dict(d) -> PromptExample
paramddict

Returns

evsys_sdk.data_types.PromptExample
funcfrom_dict(row) -> Union[ChatMessagesRow, HarborTask, PromptExample]

Dispatch on row shape - round-trips the JSONL coming off a runner.

paramrowdict

Returns

typing.Union[evsys_sdk.data_types.ChatMessagesRow, evsys_sdk.data_types.HarborTask, evsys_sdk.data_types.PromptExample]
funcparse_rows(rows, fmt) -> list[Union[ChatMessagesRow, HarborTask, PromptExample]]

Strictly parse raw dicts into typed rows for the given fmt.

This is the standardized boundary between the transform stage and a StepBuilder/algorithm: raw rows -> transforms -> parse_rows(fmt) -> typed rows. fmt is a :class:TargetFormat (or its string value). Every row must match fmt per :func:detect_format; a mismatch raises ValueError naming the offending row - no silent coercion, mirroring the SDK's extra='forbid' philosophy. Tokenization/rollout stays downstream, owned by the algorithm.

paramrowsIterable[dict]
paramfmtUnion['TargetFormat', str]

Returns

list[typing.Union[evsys_sdk.data_types.ChatMessagesRow, evsys_sdk.data_types.HarborTask, evsys_sdk.data_types.PromptExample]]
functo_dict(obj) -> dict

Dataclass → plain dict (JSON-serializable).

paramobjUnion[ChatMessagesRow, HarborTask, PromptExample, VerifierPayload]

Returns

dict
funciter_jsonl(path) -> Iterable[Union[ChatMessagesRow, HarborTask, PromptExample]]

Iterate a mixed-format JSONL and yield typed rows.

parampathstr

Returns

typing.Iterable[typing.Union[evsys_sdk.data_types.ChatMessagesRow, evsys_sdk.data_types.HarborTask, evsys_sdk.data_types.PromptExample]]