ChatTemplatedInference

Wraps an InferenceClient to apply a chat template before each generate call.

Attributes

attributenamestr

= 'chat_templated'

attributesystem_prompt

= system_prompt

attributeuser_template

= user_template

attributeenable_thinking

= enable_thinking

func__init__(self, base, *, system_prompt, user_template='{prompt}', enable_thinking=None) -> None

paramself

parambaseAny

paramsystem_promptstr

paramuser_templatestr

= '{prompt}'

paramenable_thinkingbool | None

= None

Returns

None

funcgenerate(self, *, prompt, max_tokens=256, temperature=0.0, stop=None) -> str

paramself

parampromptstr

parammax_tokensint

= 256

paramtemperaturefloat

= 0.0

paramstoplist[str] | None

= None

Returns

str