h*dZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z mZmZddlZddlmZmZddlmZddlmZddlmZd d lmZe r*dd lmZdd lm Z m!Z!dd l"m#Z#ddl$m%Z%e#Z&e%Z'e Z(e!Z)eee fZn e Z&e Z'e Z(e Z)e ZGddeZ*y)z Redis Cache implementation Has 4 primary methods: - set_cache - get_cache - async_set_cache - async_get_cache N) timedelta) TYPE_CHECKINGAnyListOptionalTupleUnioncast) print_verboseverbose_logger)!_get_parent_otel_span_from_kwargs)RedisPipelineIncrementOperation) ServiceTypes) BaseCache)Span)Redis RedisCluster)Pipeline)ClusterPipelinec eZdZ ddededefdZdedefdZdZde eefdeeeefd eedefdZ d=deeeefd eefdZde d ed ed eeddf dZd ed eefdZdZ d?d ed eedeedefdZdZd efd!Z d=deefd"Z!d#eedeefd$Z"d#eedeefd%Z# d=d&e eeeeefdeede$fd'Z% d=deefd(Z& d=d&e eeeeefdeede$fd)Z'de(fd*Z)de(fd+Z*d,Z+defd-Z,d.Z-d/Z.d0Z/d1Z0d efd2Z1d3Z2ded4ee3deeefd5Z4d4ee3deeefd6Z5d edeefd7Z6 d=d ed8eedeedefd9Z7ded ededee8fd:Z9 d?d edeedeede eeeffd;Z:xZ;S)@ RedisCacheNredis_flush_size namespace startup_nodessocket_timeoutc ddlm} ddlm} m} i} ||| d<||| d<||| d<||| d<||| d <|j d d&t |d | r|jd |_n | |_| j|| di| |_ d|_ | |_ | di| |_ ||_g|_|d |_n||_d |_ t%j&|js"|jj)d |_ t-j.j1|j3} t?|jdr|jj3t@jB(tD|tIt@jBytD|y#t*$rYwxYw#t*$rg}dt5|vrt7j8dn:t7j:dj=t5|dt5|iYd}~d}~wwxYw#t*$r,}t7j:ddt5|iYd}~d}~wwxYw)Nr)ServiceLogging)get_redis_clientget_redis_connection_poolhostportpasswordrrservice_logger_objdUnknown redis_versionzno running event loopz1Ignoring async redis ping. No running event loop.z+Error connecting to Async Redis client - {}error)extrapingz%Error connecting to Sync Redis client) default_ttl)%litellm._service_loggerr_redisr r!get isinstancepopr%update redis_clientredis_async_client redis_kwargsasync_redis_conn_poolrredis_batch_writing_bufferrr(inspectiscoroutinefunctioninfo Exceptionasyncioget_running_loop create_taskr+strr debugr)formathasattrlitellmdefault_redis_ttlsuper__init__int)selfr"r#r$rrrrkwargsrr r!r6_e __class__s W/var/www/Befach/backend/env/lib/python3.12/site-packages/litellm/caching/redis_cache.pyrGzRedisCache.__init__1sr ;H  #'L  #'L  '/L $  $,9L )  %-;L) * ::*D 1 =* ' (.C '-jj1E&FD #&4&6D #F#,<|<  )%>%N%N"#02'  #),D !$4D !& ..t/@/@A%)%6%6%;%;%=o%N" ((*66tyy{CA t((&1!!&&(  $ $ 0 G W-F-F)G  H G  =    &#a&0$$G$$AHHQP"CF+    7Q?P   sDAG1G 0I GG IAH>>I I;"I66I;returnc8ddlm}ddlm}m}|j d}|t tttf|}nR|d i|j|_ |d d|ji|j}|jd|j||_ |S) Nr)in_memory_llm_clients_cacher)get_redis_async_clientr!zasync-redis-clientkeyconnection_pool)rTvaluer-)rDrQr/rRr! get_cacher r async_redis_clientasync_redis_cluster_clientr6r7 set_cacher5)rIrQrRr! cached_clientr5s rNinit_async_clientzRedisCache.init_async_clients 8N3==BV=W  $!%(*DDE}"  *C)WTEVEV)WD &!7" $ : :">B>O>O"  ( 1 1(0G0G 2 #5!!rTcx|j-|j|js|jdz|z}|S)zD Make sure each key starts with the given namespace :)r startswithrIrTs rNcheck_and_fix_namespacez"RedisCache.check_and_fix_namespaces5 >> %cnnT^^.L..3&,C r]c |jd i|}td|d|d|d|j|j|} t j}|j j |t||t j}||z }|jjtj|d||y#t$r!}td t|Yd}~yd}~wwxYw) NzSet Redis Cache: key:  Value  ttl=, redis_version=rSnamerVexrZserviceduration call_type start_timeend_timez.script_callable?s+0=00SYUUPTUUs #z Error registering Redis script: N) r\rCrrrr@rr<r r))rIrrrLrrs @@rNasync_register_scriptz RedisCache.async_register_script)s  224M}&78$44V<< 6*66v> VS VcVsV'&7   #CCF8!L MG s,A8AA88 B)#B$$B)cKddlm}tj} |j}|j|}|j di|} |j#dd} t%d |d |d |  t'|d s t d |j)|t+j,|| | d{} t%d|d |d | tj}||z } t j |jj/tj| d||t|d|i| S#t$r}tj}||z } t j |jjtj| |||t|dtjdt|||d}~wwxYw7#t$r}tj}||z } t j |jjtj| |d||t|d|itjdt||Yd}~yd}~wwxYww)Nrrasync_set_cacherkrlr)rnroparent_otel_spanrmRLiteLLM Redis Caching: async set() - Got exception from REDIS %s, Writing value=%srSnxFSet ASYNC Redis Cache: key: rdrerrz3Redis client cannot set cache. Attribute not found.)rhrVrriz)Successfully Set ASYNC Redis Cache: key: rTrkrlrmrnrorevent_metadatarkrlr)rmrnrorrr-) redis.asynciorrqr\r<r=r?r%rrrtr r r)r@rbrpr0r rCrrjsondumpsr) rIrTrVrJrrnrrLrorvrurrs rNrzRedisCache.async_set_cacheGs'YY[  #'#9#9#;M,**s*3dll$V$ ZZe $4SE%seTU- =%0 UVV(,,jj' -F ;C5vVYUZ[ yy{H :-I   ''BB(..&/)%%Fv%N$)3<C Mg yy{H :-I   ''BB(..&)%%Fv%N/C   dA  G' :. yy{H :-I   ''BB(..&/)%%Fv%N$)3<C    dA   shI6D=A I69AG;G9G>GGr4cK|j|} |j|g|d{|'t|}|j||d{yy7.7#t$rwxYww)z@Helper function for async_set_cache_sadd. Separated for testing.rxNr)rpsaddrrr<)rIr4rTrVrurs rN_set_cache_sadd_helperz!RedisCache._set_cache_sadd_helpers~llsl# #,##C0%0 0 0,"))#s333 14   s>A-AA'AAAA-AA A**A-cFKddlm}tj} |j}|j|}t!d|d|d | |j#|||| d{t!d |d|d |tj} | |z } t j |jj%tj| d|| t| y#t$r}tj} | |z } t j |jjtj| ||| t|dtjdt|||d}~wwxYw7#t$r}tj} | |z } t j |jjtj| |d|| t| tjdt||Yd}~yd}~wwxYww)Nrrasync_set_cache_saddrrrSrrdre)r4rTrVruz.Successfully Set ASYNC Redis Cache SADD: key: rrz]LiteLLM Redis Caching: async set_cache_sadd() - Got exception from REDIS %s, Writing value=%s)rrrqr\r<r=r?r%rrrtr r r)r@rbr rr) rIrTrVrurJrrnrrLrorvs rNrzRedisCache.async_set_cache_sadds- (YY[  #'#9#9#;M.**s*34SE%seTU& --*5c.   @XeWTZ[^Z_` yy{H :-I   ''BB(..&4)%%Fv%N C C yy{H :-I   ''BB(..&)%%Fv%N4C   dA  G) 2 $ yy{H :-I   ''BB(..&4)%%Fv%NC   oA   sfH!C+&H!F-F.A>>yy{H :-I   ''BB(..&/)%%5 C M-M;?" yy{H :-I   ''BB(..&/)%%5C   pA  G) sbAFD (D)D DD "D#A D FD D D FA>FFFcKtdt|j|j|jd{g|_y7 w)Nz,flushing to redis....reached size of buffer )r rr8rrIs rNrzRedisCache.flush_cache_buffersN:3t?^?^;_:` a ++D,K,KLLL*,' MsAAA Acached_responsec||S|jd} tj|}|S#t$rt j |}Y|SwxYw)z[ Common 'get_cache_logic' across sync + async redis client implementations utf-8)decoderloadsr<ast literal_eval)rIrs rN_get_cache_logiczRedisCache._get_cache_logicsh  "" ")009 @"jjO  @!..?O @s.AAc  |j|}td|tj}|jj |}tj}||z }|j j tj|d|||td|d||j|S#t$r }tjd|Yd}~yd}~wwxYw) NrSzGet Redis Cache: key: rWrzGot Redis Cache: key: , cached_response rz;litellm.caching.caching: get() - Got exception from REDIS: ) rbr rqr4r0r%rsrrtrr<r r)) rIrTrrJrnrrorvrLs rNrWzRedisCache.get_caches ..3.7C 23%8 9J"//33C8Oyy{H :-I  # # 8 8$**"%%!!1 9  (-??PQ (((I I   Mq   sB9B<< C%C  C%rc:|jj|S) Wrapper to call `mget` on the redis client We use a wrapper so RedisCluster can override this method r)r4mget)rIrs rN_run_redis_mget_operationz$RedisCache._run_redis_mget_operations   %%4%00r]cbK|j}|j|d{S7w)rrN)r\r)rIrrXs rN_async_run_redis_mget_operationz*RedisCache._async_run_redis_mget_operations1 "335',,$,7777s &/-/key_listci}|Dcgc]}|| }} g}|D])}|j|xsd}|j|+tj}|j|} tj} | |z } |jj t j| d|| |tt|| }i} |jD]<\} }t| tr| jd} |j|}|| | <>| Scc}w#t$r-}t!j"dt%||cYd}~Sd}~wwxYw) a Use Redis for bulk read operations Args: key_list: List of keys to get from Redis parent_otel_span: Optional parent OpenTelemetry span Returns: dict: A dictionary mapping keys to their cached values NrSrbatch_get_cacherrz$Error occurred in batch get cache - )rbrrqrr%rsrrtdictzipitemsr1bytesrrr<r r)r@)rIrrkey_value_dictrT _key_list_keysrrnrrorvdecoded_resultskvrLs rNrzRedisCache.batch_get_caches_$,@SS@ @ "E&  88Y_"8M  Y''J :::FGyy{H :-I  # # 8 8$**"+%!!1 9 "#i"9:N O&,,.1a')A))!,%&" / # "?A@ "  #GAx!P Q! ! "s(DDDD!! E*"E EEcKddlm}|j}|j|}t j} t d||j |d{}t d|d||j|}t j} | |z } tj|jjtj| d|| |d |i |S7#t$r} t j} | |z } tj|jjtj| | d|| |d |i t d t!| Yd} ~ yd} ~ wwxYww) NrrrSzGet Async Redis Cache: key: zGot Async Redis Cache: key: rrasync_get_cacherTrrzAlitellm.caching.caching: async get() - Got exception from REDIS: )rrr\rbrqr r0rr=r?r%rrrtr<rr@) rIrTrrJrrrnrresponserorvrLs rNrzRedisCache.async_get_caches (#557 **s*3YY[ '  8> ?$1$5$5c$::O .se3EoEVW ,,_,MHyy{H :-I   ''BB(..&/)%%5$)3<C O';( yy{H :-I   ''BB(..&/)%%5$)3<C   STWXYTZS[\   sC=E8"C,"C*#BC,)E8*C,, E55A6E0+E80E55E8c Ki}tj}|Dcgc]}|| }} g}|D]%}|j|}|j|'|j|d{} tj} | |z } t j |j jtj| d|| |tt|| }i} |jD]<\} }t| tr| jd} |j!|}|| | <>| Scc}w7#t"$r}tj} | |z } t j |j j%tj| |d|| |t'j(dt+||cYd}~Sd}~wwxYww) a[ Use Redis for bulk read operations Args: key_list: List of keys to get from Redis parent_otel_span: Optional parent OpenTelemetry span Returns: dict: A dictionary mapping keys to their cached values `.mget` does not support None keys. This will filter out None keys. NrSrasync_batch_get_cacherrrz*Error occurred in async batch get cache - )rqrbrrr=r?r%rrrtrrrr1rrrr<rr r)r@)rIrrrrnrTrrrrrorvrrrrLs rNrz RedisCache.async_batch_get_cache4s$YY[ $,@SS@ @0 "E&  88Y8G  Y''!@@e@LLGyy{H :-I   ''BB(..&5)%%5 C "#i"9:N O&,,.1a')A))!,%&" / # "AA M6 "yy{H :-I   ''BB(..&5)%%5C   #McRSfX!V W! !! "sXGD5D5GAD<-D:.CD<4G:D<< GA>G GG GGctdtj} |jj}td|tj}||z }|jj t j|d|||S#t$rn}tj}||z }|jjt j||dtjdt||d}~wwxYw)zD Tests if the sync redis client is correctly setup. zPinging Sync Redis CachezRedis Cache PING: sync_pingrjrkrlr)rm7LiteLLM Redis Cache PING: - Got exception from REDIS : N) r rqr4r+r%rsrrtr<service_failure_hookr r)r@)rIrnrrorvrLs rNrzRedisCache.sync_ping{s 01YY[  !..335H .xj9 :yy{H :-I  # # 8 8$**"%%! 9 O yy{H :-I  # # 8 8$**"% 9   I#a&R G sA0B D A)DD c lK|j}tj}td |jd{}tj}||z }t j |j jtj|d|S7^#t$r}tj}||z }t j |j jtj||dtjdt||d}~wwxYww)NzPinging Async Redis Cache async_pingrkrlrmrr)r\rqr r+r=r?r%rrrtr<rr r)r@)rIrrnrrorvrLs rNr+zRedisCache.pings!335 YY[ 12 *//11Hyy{H :-I   ''BB(..&*C O2 yy{H :-I   ''BB(..&* C   I#a&R G! s<0D4B'B%AB'$D4%B'' D10Av>>Hyy{H :-I   ''BB(..&+C O? yy{H :-I   ''BB(..&+ C   J3q6(S G! s:%D,BBABD,B D)(Az(RedisCache.async_lpop..s2,0 4'2srz7LiteLLM Redis Cache LPOP: - Got exception from REDIS : )r\rqr r(rHsplitrrrr=r?r%rrrtr1rrr<listallrr r)r@)rIrTrrrJrrn major_versionrrrorvr%rLs rNr"zRedisCache.async_lpopsl"335 YY[ 3C5 %IJ: !"M!!Y. #D$6$6$<$"!"!M" E "!M" yy{H :-I   ''BB(..&* C   I#a&R G! s6I)AGFGF)F*F. G9F:GF3A,GF5I)"G6G :GG I)GI)GFGF0$F' %F0,G5 GGI)GGG GGI)GG I&%A"23327;77$,UO7 7r,S,,t,\CCIM9uS#X/96>uo9v(  e_   "FF%-e_FP,"+/ 99c] 9 #4. 9  9v- x~41d3i1DI18$s)8S 8,02"S 4 #6672"#4.2"  2"j7;0%-d^0j,0E"S 4 #667E"#4.E"  E"N"4"H"D"H* T%%L/C/ &<<<=< $u+  <0="#BC= $u+ =~sx}6,0 1 1S 1#4. 1 1f  #& /2  e $ $+/ D D}D#4. D sDI~  Dr]r)+__doc__rr=r9rrqdatetimertypingrrrrrr r rDlitellm._loggingr r 'litellm.litellm_core_utils.core_helpersr litellm.types.cachingrlitellm.types.servicesr base_cacheropentelemetry.tracer_Spanrrrredis.asyncio.clientrredis.asyncio.clusterrrr.rXrYrr-r]rNr=s  III:UA/!11-5H&!-  DH!$ Diir]