h d dl Z d dlmZ d dlmZmZmZ d dlZd dlm Z dZ
G d de Z ed Z g dZ
d efd
Z ed d
edefd Zy) N) lru_cache)LiteralOptionalUnion)BaseLLMExceptionzhttps://huggingface.coc
e Zd Z ddeej
deej deeej e f f fd
Z
xZ S )HuggingFaceErrorrequestresponseheadersc . t | ||||| y )N) status_codemessager
r r )super__init__)selfr r r
r r __class__s a/var/www/Befach/backend/env/lib/python3.12/site-packages/litellm/llms/huggingface/common_utils.pyr zHuggingFaceError.__init__
s& #
)NNN) __name__
__module__ __qualname__r httpxRequestResponser Headersdictr
__classcell__)r s @r r r s[
,0-18<
%--(
5>>*
%
t 345
r r )ztext-generation-inferenceconversationalztext-classificationztext-generationgenerated_textc g d}|D ]k }| j j | r| j |dd } | j | sG| ddd j |ddd dd ddd } m | S )z
Parse the output text to remove any special characters. In our current approach we just check for ChatML tokens.
Initial issue that prompted this - https://github.com/BerriAI/litellm/issues/763
)z
<|assistant|>z
<|system|>z<|user|>zz N)strip
startswithreplaceendswith)r chat_template_tokenstokens r
output_parserr+ - s V% ! , ,U 3 + 3 3E2q AN " "5 ) +DbD 1 9 9%"+r1 MdPRd SN &
r )maxsizemodelreturnc & ddi}t j d rdt j d |d< t d| }ddgi} t j ||| }|j |j
j d }|t d | |S # t j $ r^}t |d r/t |j d
d }t |j di }nd}i }t dt | || d
}~ww xY w)aT
Fetch provider mappings for a model from the Hugging Face Hub.
Args:
model: The model identifier (e.g., 'meta-llama/Llama-2-7b')
Returns:
dict: The inference provider mapping for the model
Raises:
ValueError: If no provider mapping is found
HuggingFaceError: If the API request fails
Acceptzapplication/jsonHUGGINGFACE_API_KEYzBearer
Authorizationz /api/models/expandinferenceProviderMapping)r paramsNz$No provider mapping found for model r r i r z"Failed to fetch provider mapping: )r r r )osgetenv
HF_HUB_URLr getraise_for_statusjson
ValueError HTTPErrorhasattrgetattrr r str)r. r pathr6 r provider_mappinger s r !_fetch_inference_provider_mappingrE < s +,G yy&'%,RYY7L-M,N#O \ eW -D34
5F
99T76B!!##==?../IJ # CE7 KL L ??
1j !!!**mSAK ajj)R8G KG 8QA#
s AB D2AD D)r7 functoolsr typingr r r r )litellm.llms.base_llm.chat.transformationr r9 r hf_tasks hf_task_listrA r+ r rE r r rL st + + F
%
'
$ # 3)
S )
T )
)
r