SamplingClient
Inference-only client backed by a saved sampler checkpoint.
Created by :meth:Backend.snapshot_sampling_client. Both the loop's
in-loop eval slot and any rollout-based StepBuilder (SDFT, RL) consume
this surface - never the underlying tinker client directly.
Functions
funcsample_async(self, *, prompt, params, num_samples=1, include_prompt_logprobs=False, topk_prompt_logprobs=0) -> AnyGenerate tokens for prompt. Return shape is backend-defined.
TinkerBackend returns tinker's native SamplingResponse with
.sequences (one per sample) and optional .topk_prompt_logprobs.
MockBackend returns canned strings for tests.
paramselfparamprompttinker.ModelInputparamparamstinker.SamplingParamsparamnum_samplesint= 1paraminclude_prompt_logprobsbool= Falseparamtopk_prompt_logprobsint= 0Returns
typing.Anyfunccompute_logprobs_async(self, prompt) -> list[float | None]Per-position logprobs of the prompt under the current weights.
paramselfparamprompttinker.ModelInputReturns
list[float | None]