hF d Z ddlmZmZmZmZmZmZmZm Z m
Z
ddl Z ddl m
Z ddlmZmZ ddlmZ ddlmZ ddlmZ dd lmZ dd
lmZ dd l m Z G d d
e Z y)z
Transformation logic from OpenAI /v1/chat/completion format to Mistral's /chat/completion format.
Why separate file? Make it easy to see how transformation works
Docs - https://docs.mistral.ai/api/
) Any CoroutineListLiteralOptionalTupleUnioncastoverloadN)Logging)3handle_messages_with_content_list_to_str_conversion strip_none_values_from_message)OpenAIGPTConfig)get_secret_str)MistralToolCallMessage)AllMessageValues)
ModelResponse) convert_to_model_response_objectc e Zd ZU dZdZee ed< dZee ed< dZ ee ed< dZ
ee ed< dZ ee
d ed< dZee ed < dZee ed
< dZee ed < dZeeee f ed < d2dee dee dee dee dee
d d ee d
ee d ee d eeee f d
dfdZe fd Zded
ee fdZded
efdZe d
efd Z dedededed
ef
dZ dee dee d
e eee f fdZ e!dee" ded e
d d
e#e$e$ee" f fd Z%e! d3dee" ded e
d d
ee" fd Z% d3dee" ded ed
eee" e#e$e$ee" f f f fd!
Z%dee" ded
ee" fd"Z&ed#e"d
e"fd$ Z'ed#e"d
e"fd% Z(e d&ed
efd' Z)dedee" ded(ed)ed
ef fd* Z* d4ded+e+jX d,e-d-e.d.edee" ded(ed/e$dee d0ee d
e-fd1Z/ xZ0S )5
MistralConfiga
Reference: https://docs.mistral.ai/api/
The class `MistralConfig` provides configuration for the Mistral's Chat API interface. Below are the parameters:
- `temperature` (number or null): Defines the sampling temperature to use, varying between 0 and 2. API Default - 0.7.
- `top_p` (number or null): An alternative to sampling with temperature, used for nucleus sampling. API Default - 1.
- `max_tokens` (integer or null): This optional parameter helps to set the maximum number of tokens to generate in the chat completion. API Default - null.
- `tools` (list or null): A list of available tools for the model. Use this to specify functions for which the model can generate JSON inputs.
- `tool_choice` (string - 'auto'/'any'/'none' or null): Specifies if/how functions are called. If set to none the model won't call a function and will generate a message instead. If set to auto the model can choose to either generate a message or call a function. If set to any the model is forced to call a function. Default - 'auto'.
- `stop` (string or array of strings): Stop generation if this token is detected. Or if one of these tokens is detected when providing an array
- `random_seed` (integer or null): The seed to use for random sampling. If set, different calls will generate deterministic results.
- `safe_prompt` (boolean): Whether to inject a safety prompt before all conversations. API Default - 'false'.
- `response_format` (object or null): An object specifying the format that the model must output. Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is in JSON. When using JSON mode you MUST also instruct the model to produce JSON yourself with a system or a user message.
N temperaturetop_p
max_tokenstools)autoanynone tool_choice random_seed safe_promptresponse_formatstopreturnc
t j }
|
j D ]% \ } } | dk7 s | t | j | | ' y )Nself)localscopyitemssetattr __class__)
r% r r r r r r r r! r" locals_keyvalues
d/var/www/Befach/backend/env/lib/python3.12/site-packages/litellm/llms/mistral/chat/transformation.py__init__zMistralConfig.__init__< sA (--/!--/JCf}!2U3 * c t | S N)super
get_config)clsr* s r. r4 zMistralConfig.get_configM s w!##r0 modelc X g d}d|j v r|j ddg |S )N) streamr r r max_completion_tokensr r seedr" r! parallel_tool_calls magistralthinkingreasoning_effort)lowerextend)r% r6 supported_paramss r. get_supported_openai_paramsz)MistralConfig.get_supported_openai_paramsQ s4
%++- ' # #Z1C$D E r0 c ( |dk( s|dk( r|S |dk( ryy)Nr r requiredr )r% r s r. _map_tool_choicez MistralConfig._map_tool_choicef s% & K6$9
J
&r0 c y)z
Returns the system prompt for Mistral reasoning models.
Based on Mistral's documentation: https://huggingface.co/mistralai/Magistral-Small-2506
Mistral recommends the following system prompt for reasoning:
ay
[SYSTEM_PROMPT]system_prompt
A user will ask you to solve a task. You should first draft your thinking process (inner monologue) until you have derived the final answer. Afterwards, write a self-contained summary of your thoughts (i.e. your summary should be succinct but contain all the critical steps you needed to reach the conclusion). You should use Markdown to format your response. Write both your thoughts and summary in the same language as the task posed by the user. NEVER use oxed{} in your response.
Your thinking process must follow the template below:
Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate a correct answer.
Here, provide a concise summary that reflects your reasoning and presents a clear final answer to the user. Don't mention that this is a summary.
Problem:
[/SYSTEM_PROMPT][INST]user_message[/INST]
reasoning_traces
assistant_response[INST]user_message[/INST]
rE rE r0 r. $_get_mistral_reasoning_system_promptz2MistralConfig._get_mistral_reasoning_system_promptn s r0 non_default_paramsoptional_params drop_paramsc |j D ] \ }}|dk( r||d< |dk( r||d< |dk( r||d< |dk( r |du r||d< |dk( r||d< |dk( r||d< |dk( r||d< |d k( r%t |t r| j |
|d < |d k( rd |i|d
< |dk( r||d< |dk( rd|j v rd|d< |dk( rd|j v rd|d< |dk( s||d< |S )Nr r9 r r8 Tr r r" r )r r: r
extra_bodyr! r> r< _add_reasoning_promptr= r; )r(
isinstancestrrF r? )r% rI rJ r6 rK paramr- s r. map_openai_paramsz MistralConfig.map_openai_params sP / 4 4 6LE5 $05 -//05 - +0( Ud],1)
%16
. +0(*/'
%*UC*@151F1FSX1F1Y
.1>0F -))5: 12**{ekkm/K;? 78
"{ekkm'C;? 78--9> 569 7: r0 api_baseapi_keyc |xs t d xs d}||j d s|dz }|xs t d xs t d }||fS )NMISTRAL_AZURE_API_BASEzhttps://api.mistral.ai/v1z/v1MISTRAL_AZURE_API_KEYMISTRAL_API_KEY)r endswith)r% rS rT dynamic_api_keys r. $_get_openai_compatible_provider_infoz2MistralConfig._get_openai_compatible_provider_info sr
+ 67
+* (9(9%(@ %'H
1 56
1 /0
((r0 messagesis_asyncTc y r2 rE r% r\ r6 r] s r. _transform_messagesz!MistralConfig._transform_messages s 7:r0 Fc y r2 rE r_ s r. r` z!MistralConfig._transform_messages s "%r0 c |D ]n }|j d }|st |t s(|D ]B }|j d dk( s|rt | ||d c c S t | ||d c c S p t | }g }|D ]H }t j | }t j | }t | }|j | J |rt | ||d S t | ||d S )ap
- handles scenario where content is list and not string
- content list is just text, and no images
- if image passed in, then just return as is (user-intended)
- if `name` is passed, then drop it for mistral API: https://github.com/BerriAI/litellm/issues/6696
Motivation: mistral api doesn't support content as a list
contenttype image_urlTF) getrO listr3 r` r
r _handle_name_in_message_handle_tool_call_messager append) r% r\ r6 r] m_content_blockc new_messagesr* s r. r` z!MistralConfig._transform_messages s A UU9 -N *^T"B'AuuV} 3##(7#>xPT#U U#(7#>xPU#V V ( GxP 02 A 55a8A 77:A.q1A " 7.|UDI I7.|UEJ Jr0 c |j dd s|S t d |D }|rt | D ] \ }}|j d dk( s|j dd }| j }t |t
r| d| }n-t |t r
d |dz d
g|z }n| dt | }t t i |d|i ||< n) n't t d| j d } | g|z }|j dd |S )
zn
Add reasoning system prompt for Mistral magistral models when reasoning_effort is specified.
rN Fc 3 D K | ] }|j d dk( yw)rolesystemN)rf ).0msgs r. zGMistralConfig._add_reasoning_system_prompt_if_needed.. s Q H!< Qs rq rr rc z
text)rd rw )rq rc N)
rf r enumeraterH rO rP rg r
r pop)
r% r\ rJ has_system_messageirt existing_contentreasoning_prompt new_contentreasoning_messages
r. &_add_reasoning_system_prompt_if_neededz4MistralConfig._add_reasoning_system_prompt_if_needed s: ""#:EBO ! Q Q Q #H-3776?h.'*wwy"'=$'+'P'P'R$ ""2C8;K:LDQaPb8c #$4d;06@PSY@Y'Z&[^n&n *:(:$sCS?T>U&V "&'79XC9XK9X"YHQK# .( 37 8@i@i@k"l3 **X5H 3T:r0 messagec |j d }|Z|d dk7 r|j dd |S t |t r.t |j dk( r|j dd |S )z
Mistral API only supports `name` in tool messages
If role == tool, then we keep `name` if it's not an empty string
Otherwise, we drop `name`
nameNrq toolr )rf ry rO rP lenstrip)r5 r _names r. rh z%MistralConfig._handle_name_in_message! sj F# v&( FD)
E3'C
,>!,C FD)r0 c |j d }g }|Yt |t rI|D ]? }t |j d d|j d }|j | A ||d< |S )zc
Mistral API only supports tool_calls in Messages in `MistralToolCallMessage` spec
tool_callsidfunction)r rd r )rf rO rg r rj )r5 r _tool_callsmistral_tool_calls_tool_tool_call_messages r. ri z'MistralConfig._handle_tool_call_message5 sx
kk,/ ;= "z+t'D$%; yy# "YYz 2&"
#))*<=
% %7GL !r0
response_datac | j d rMt | d dkD r<| d D ]4 }|j d s|d j d dk( s-d|d d< 6 | S )a
Handle Mistral-specific behavior where empty string content should be converted to None.
Mistral API sometimes returns empty string content ('') instead of null,
which can cause issues with downstream processing.
Args:
response_data: The raw response data from Mistral API
Returns:
dict: The response data with empty string content converted to None
choicesr r rc rv N)rf r )r choices r. _handle_empty_content_responsez,MistralConfig._handle_empty_content_responseG sm Y 'C
i0H,IA,M' 2::i(VI->-B-B9-MQS-S37F9%i0 3 r0 litellm_paramsheadersc d|j v r$|j dd r| j || }t | ||||| S )a
Transform the overall request to be sent to the API.
For magistral models, adds reasoning system prompt when reasoning_effort is specified.
Returns:
dict: The transformed request. Sent as the body of the API call.
r< rN F)r6 r\ rJ r r )r? rf r r3 transform_request)r% r6 r\ rJ r r r* s r. r z MistralConfig.transform_request[ s^ %++- 'O,?,?@WY^,_BB8_]H w( +) )
r0 raw_responsemodel_response logging_obj request_dataencoding json_modec * |j |j |j |j d< |j } | j | } t
t t | |d|j it |j }
|
S )z
Transform the raw response from Mistral API.
Handles Mistral-specific behavior like converting empty string content to None.
)original_responseresponse_headersr )response_objectmodel_response_object
hidden_params_response_headers)
post_callrw r model_call_detailsjsonr r
r r dict)r% r6 r r r r r\ rJ r r rT r r final_response_objs r. transform_responsez MistralConfig.transform_responsew s $ 0A0A B=I=Q=Q &&'9: %))+
;;MJ
! , -&4 (,*>*> ?"&|';';"<
"!r0 ) NNNNNNNNN)F)NN)1__name__
__module__ __qualname____doc__r r int__annotations__r r r rg r r r r boolr! r r" r rP r/ classmethodr4 r rB rF staticmethodrH rR r r[ r r r r r` r rh ri r r httpxResponser LiteLLMLoggingObjr
__classcell__)r* s @r. r r s| 0 "&K# % E8C= $J
$ E8D> <@K'"789@!%K# %"&K$ &&*OXd^*'+D(5d#
$+ &* #$( $@D%)&**.+/4 c]4 }4 SM 4
~ 4 g&; <=
4 c]4 d^4 "$ 4 uS$Y'(4
4" $ $ c *C C # 4$ $ $ $
$
$ L)
)08
) sHSM! ")( : -.:7::FMdm: 3T"2 33 4: :
$) %'(% % %. %
% % NS$K -.$K7:$KFJ$K t$%yc4@P;Q1Q'RR S$KL) -.)AE)
)V .> CS & 0@ EU " d t &
'(
N "&$(#"#" nn#" & #"
' #"
#" '(#" #" #" #" # #" D>#"
#"r0 r ) r typingr r r r r r r r
r r *litellm.litellm_core_utils.litellm_loggingr r 8litellm.litellm_core_utils.prompt_templates.common_utilsr
r +litellm.llms.openai.chat.gpt_transformationr litellm.secret_managers.mainr litellm.types.llms.mistralr litellm.types.llms.openair litellm.types.utilsr
litellm.utilsr r rE r0 r. r sD Y X X S H 7 = 6 - :A"O A"r0