hzddlZddlZejjdejj dddlZddlZddlmZmZm Z m Z m Z m Z m Z ddlmZddlZddlmZddlmZddlmZmZddlmZdd lmZmZdd lmZdd lm Z dd l!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*m+Z+m,Z,m-Z-m.Z.dZ/GddeeZ0y)Nz../..)AnyAsyncGeneratorListLiteralOptionalTupleUnion) HTTPException)verbose_proxy_logger) DualCache)CustomGuardraillog_guardrail_information) BaseAWSLLM)get_async_httpx_clienthttpxSpecialProvider)UserAPIKeyAuthGuardrailEventHooks)AllMessageValues)BedrockContentItemBedrockGuardrailOutputBedrockGuardrailResponseBedrockRequestBedrockTextContent)Choices ModelResponseModelResponseStreamStreamingChoices TextChoicesbedrockceZdZ d-deedeedeeffd Zdeeede fdZ d e e e fde fd Z d.d ed deeed ee e e fde fd ZdZ d/dedededeefdZ d0d ed deeed ee e ejfdeedef dZd edefdZd edefdZedededededde eeedff dZededededfdZededefdZ deed edeefd!Z!ded e dede"e#dffd"Z$d edeefd#Z%deed$eedeefd%Z&d&ee d$eed'e'de(ee e'ffd(Z)d)edeeefd*Z*d e e e fd eddfd+Z+d ejd$eeddfd,Z,xZ-S)1BedrockGuardrailNguardrailIdentifierguardrailVersiondisable_exception_on_blockc $ttj|_||_||_||_|xsd|_ t|$di|tj|tjd|j|j y)N) llm_providerFzPBedrock Guardrail initialized with guardrailIdentifier: %s, guardrailVersion: %s) rrGuardrailCallback async_handlerr#r$optional_paramsr%super__init__rr debug)selfr#r$r%kwargs __class__s w/var/www/Befach/backend/env/lib/python3.12/site-packages/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.pyr-zBedrockGuardrail.__init__6s4-?? $7 0 &0J0Se' "6"D!"" ^  $ $  ! ! messagesreturnctd}g}||S|D]E}|j|}||D])}tt|}|j |+G||d<|S)zS Create a bedrock request for the input content - the LLM request. INPUTsource)messagetextcontent)rget_content_for_messagerrappend)r/r4bedrock_requestbedrock_request_contentr:message_text_content text_contentbedrock_content_items r2%_create_bedrock_input_content_requestz6BedrockGuardrail._create_bedrock_input_content_requestVs+9*H<>  " "G,,W,= !$+ 4 '9+>($(../CD !5  &= "r3responsectd}g}t|tjr|jD]}t|tj s|j js5t|j jtsZtt|j j}|j|||d<|S)zU Create a bedrock request for the output content - the LLM response. OUTPUTr8r;r=) r isinstancelitellmrchoicesrr:r=strrrr?)r/rFr@rAchoicerDs r2&_create_bedrock_output_content_requestz7BedrockGuardrail._create_bedrock_output_content_requestms+9*I<> h 5 5 6"**fgoo6~~--*..30B!39O9O!P0,0667KL+*AOI &r3r9)r7rHc~t|}|dk(r|j|}|S|dk(r|j|}|S)a Convert the litellm messages/response to the bedrock request format. If source is "INPUT", then messages is required. If source is "OUTPUT", then response is required. Returns: BedrockRequest: The bedrock request object. r8r7)r4rH)rF)rrErN)r/r9r4rFr@s r2convert_to_bedrock_formatz*BedrockGuardrail.convert_to_bedrock_formatsV+9*G W "HHRZH[Ox "IIS[I\Or3c  ddlm}|jj dd}|jj dd}|jj dd}|jj dd}|jj dd}|jj d d}|jj d d}|jj d d} |jj d d} |j | }|j |||||||| |  } | |fS#t$r tdwxYw)Nr) Credentials7Missing boto3 to call bedrock. Run 'pip install boto3'.aws_secret_access_keyaws_access_key_idaws_session_tokenaws_region_name aws_role_nameaws_session_nameaws_profile_nameaws_web_identity_tokenaws_sts_endpoint)rW) rUrTrVrWrYrZrXr[r\)botocore.credentialsrR ImportErrorr+get)get_aws_region_name_for_non_llm_api_callsget_credentials) r/rRrTrUrVrWrXrYrZr[r\ credentialss r2_load_credentialsz"BedrockGuardrail._load_credentialssr Y 8!% 4 4 8 89PRV W 00445H$O 00445H$O..223DdK,,00$G //334FM//334FM!%!5!5!9!9 $d"  //334FMHH+I $(#7#7/"7/+--'#9-$8 $ O++? YWX X Ys D44E datar+rW extra_headersc ddlm}ddlm}||d|}d|d|j d|j d } tj|jd } d d i} |d d i|} |d | | | } |j| |d|vr|d| jd<| j} | S#t$r t dwxYw)Nr) SigV4Auth) AWSRequestrSr zhttps://bedrock-runtime.z.amazonaws.com/guardrail/z /version/z/applyzutf-8z Content-Typezapplication/jsonPOST)methodurlrdheaders Authorization) botocore.authrgbotocore.awsrequestrhr^r#r$jsondumpsencodeadd_authrlprepare)r/rbrdr+rWrergrhsigv4api_base encoded_datarlrequestprepped_requests r2_prepare_requestz!BedrockGuardrail._prepare_requests  Y / 6+y/B-o->>WX\XpXpWqqz{|Q|Q{RRXYzz$'..w7 !#56  $%'9K]KGxlG  w  %/]*J/<_/MGOOO ,!//+- YWX X Ys B//C request_datacK|j\}}t|j|||}t}|r!|j |j ||j |||j|} tjd|| j| j|jj| j| j| jd{}tjd|j|j dk(r?|j#} td i| }|j%|r|j'||Stj(d|j |j|S7w) N)r9r4rF)r{)rbrdr+rWz0Bedrock AI request body: %s, url %s, headers: %s)rkrdrlzBedrock AI response: %sz>L>Y  00#% 00+ 1  "" >   $ $  ++00 $$!&&$,,1   ""##%2 %/NN3O$P !$488K &2 %>>(3y@#''3033IrB!( 99X.);#'") +5..9T*U '*$?$C$CIr$R!(9$'++H5B#):_&jr3user_api_key_dictcache call_type) completiontext_completion embeddingsimage_generation moderationaudio_transcriptionpass_through_endpointrerankcvKtjdddlm}tj }|j ||dur|S|jd}|tjd|S|jd|| d{}|j|| |d<|||j |S70w) NzInside AIM Pre-Call Hookr*add_guardrail_to_applied_guardrails_headerrd event_typeTr46Bedrock AI: not running guardrail. No messages in datar7r9r4r{r4rr{guardrail_name) r r.)litellm.proxy.common_utils.callback_utilsrrpre_callshould_run_guardrailr_warningr8_update_messages_with_updated_bedrock_guardrail_responser) r/rrrdrrr new_messagesrs r2async_pre_call_hookz$BedrockGuardrail.async_pre_call_hooks" ""#=> +>*F*F  $ $$: $ Fd RK9=*9M   ( (H K ,0+H+H\,I, & "  I I%+E J  Z 3d.A.A  -& sBB9B71B9)rrrrr responsescHKddlm}tj}|j ||dury|j d}|t jdy|jd||d{}|j|| |d<|||j |S70w) NrrrTr4rr7rrr) rrr during_callrr_r rrrr)r/rdrrrrrrs r2async_moderation_hookz&BedrockGuardrail.async_moderation_hooks +>*I*I  $ $$: $ Fd R 9=*9M   ( (H   ,0+H+H\,I, & "  I I%+E J  Z 3d.A.A  -& sA-B"/B 01B"c@Kddlm}ddlm}|j ||j dury|j d}|tjdy|jd|| d{}|j|| |||j y7,w) NrrrrTr4rrH)r9rFr{rFrr) rrlitellm.types.guardrailsrr post_callr_r rr_apply_masking_to_responser)r/rdrrFrrroutput_content_bedrocks r2async_post_call_success_hookz-BedrockGuardrail.async_post_call_success_hooks  A  % %&9&C&C &   9=*9M   ( (H   (,'D'D(E( "  '''= (  3d.A.A #" sA-B/B0-Brc|j|}|r(tjd|j||S|js |j rtjd|S)af Use the output from the bedrock guardrail to mask sensitive content in messages. Args: messages: Original list of messages bedrock_guardrail_response: Response from Bedrock guardrail containing masked content Returns: List of messages with content masked according to guardrail response z>Bedrock guardrail provided masked output, applying to messages)r4 masked_textszPMasking enabled but no masked output from guardrail, returning original messages)#_extract_masked_texts_from_responser r._apply_masking_to_messagesrr)r/r4rrs r2rzIBedrockGuardrail._update_messages_with_updated_bedrock_guardrail_response/sw ?? &  & &P 22! 3   $ $(B(B & &b r3cKddl}ddlm}ddlm}ddlm}g}|23d{} |j| 76||} t| tr|jd|jd| } |jd | } |j| | d{7\} }|j| | || }|23d{7} | 6y|D]} |  yw)z Process streaming response chunks. Collect content from the stream and make parallel bedrock api requests to get the guardrail responses. rN)MockResponseIterator)stream_chunk_builder)TextCompletionResponse)chunksr7r4rrH)r9rFr)model_response)asyncio)litellm.llms.base_llm.base_model_iteratorr litellm.mainrlitellm.types.utilsrr?rIrrr_gatherr)r/rrFr{rrrr all_chunkschunkassembled_model_response input_task output_task_output_guardrail_response mock_responses r2'async_post_call_streaming_iterator_hookz8BedrockGuardrail.async_post_call_streaming_iterator_hookVs! R5>13 # % %%   e $ %8 !  ! . >66)9)9*)ET`7J77*B8K 29K2,, (A(  + +1+D , 17M -  e  -$ $sDC%:8:C%:A%C%B" &C%C C  CC%cg}|jdgxsg}|stjdgS|D]'}|jd}||j|)|S)z Extract all masked text outputs from the guardrail response. Args: bedrock_guardrail_response: Response from Bedrock guardrail Returns: List of masked text strings rz-No masked outputs found in guardrail responser<)r_r r.r?)r/rmasked_output_textmasked_outputsrrCs r2rz4BedrockGuardrail._extract_masked_texts_from_responsesq)+ & * *9b 9 ?R  & &'V WI$F*0**V*z(BedrockGuardrail.get_content_for_messages "++i( ? gs # ' ' 0$#  &dD)fn(//V =c*(//5  $#r3c,|j|}|stjdytjdt|t |t j r|j||ytjdt|y)a Apply masked content from bedrock guardrail to the response object. Args: response: The response object to modify bedrock_guardrail_response: Response from Bedrock guardrail containing masked content z2No masked outputs found, skipping response maskingNz1Applying masking to response with %d masked textsz)Unsupported response type for masking: %s) rr r.rrIrJr _apply_masking_to_model_responsertype)r/rFrrs r2rz+BedrockGuardrail._apply_masking_to_responses?? &  & &'[ \ "" ?\AR h 5 5 6  1 1(L I ( (;T(^ r3c8d}|jD]}t|trz|jjs+t|jjt sP|t |ks_|||j_|dz }tjdt|tr{|jjst|jjt s|t |ks|||j_|dz }tjdt|ts+|js9t|jt sU|t |kse|||_ |dz }tjdy)z Apply masked texts to a ModelResponse object. Args: response: The ModelResponse object to modify in-place masked_texts: List of masked text strings from guardrail rrz)Applied masking to choice message contentz'Applied masking to choice delta contentz&Applied masking to choice text contentN) rKrIrr:r=rLrr r.rdeltarr<)r/rFrrrMs r2rz1BedrockGuardrail._apply_masking_to_model_response6s7 &&F&'*>>))j9O9OQT.U$s<'881=m1L.%* ,22GF$45<<''Jv||7K7KS,Q$s<'88/;M/J ,%* ,22EFK0;;:fkk3#?$s<'88&2=&A %* ,22D3'r3)NNF)NN)N)NNN).__name__ __module__ __qualname__rrLboolr-rrrrEr rrrNrrPrcr~rzrJrrr rrrrr Exceptionrrrrrrrrrintrrr>rr __classcell__)r1s@r2r"r"5s.2*.5:  %c] #3- %-TN  @htL\G]>^cq.uS-EW?X]k,6:8< )*4 0125m!345   .$,X)- !! !  !  ~ !L6:@D'+ :*)*:*4 012:*5g&;&;!;<= :* tn :* " :*z BZ _l .N0N N`:):: :    : y#tT) *::x55*5  55n0 0 *0 0 j%'(%%=%   %NC)CC C +T1 2 CJ"*B" c"6( -.( >B3i(  ( T* I*59#Y*OR* tCy#~ *>$/?$HTRUYDW$, s*+%=  B'--'=A#Y' 'r3r")1ossyspathinsertabspathrptypingrrrrrrr fastapir rJlitellm._loggingr litellm.cachingr %litellm.integrations.custom_guardrailr r!litellm.llms.bedrock.base_aws_llmr&litellm.llms.custom_httpx.http_handlerrrlitellm.proxy._typesrrrlitellm.types.llms.openairAlitellm.types.proxy.guardrails.guardrail_hooks.bedrock_guardrailsrrrrrrrrrrrGUARDRAIL_NAMEr"r(r3r2r"s rwww MMM!1%9086h  h r3