h(dZddlZddlmZddlmZmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZGd d eZy) zb Base class across routing strategies to abstract commmon functions like batch incrementing redis N)ABC)DictListOptionalSetTupleUnion)verbose_router_logger) DualCache)RedisPipelineIncrementOperation)DEFAULT_REDIS_SYNC_INTERVALc eZdZdededeeeeffdZ deeeeffdZ dZ de e eefded e efd Zd ed eeefdefd ZdeeeeffdZdZd efdZd eefdZd eefdZd eefdZdZdZy)BaseRoutingStrategy dual_cacheshould_batch_redis_writesdefault_sync_intervalcr||_g|_d|_|r|j|t |_yN)rredis_increment_operation_queue _sync_tasksetup_sync_tasksetin_memory_keys_to_update)selfrrrs i/var/www/Befach/backend/env/lib/python3.12/site-packages/litellm/router_strategy/base_routing_strategy.py__init__zBaseRoutingStrategy.__init__s; %VX,8< $  !6 7 E %c tj}|j |j ||_y#t$r,tj}tj|Y[wxYw)z;Setup the sync task in a way that's compatible with FastAPI)rN)asyncioget_running_loop RuntimeErrornew_event_loopset_event_loop create_task(periodic_sync_in_memory_spend_with_redisr)rrloops rrz#BaseRoutingStrategy.setup_sync_task sl )++-D **  9 9&; :    )))+D  " "4 ( )s=2A21A2cK|j0|jj |jd{yy7#tj$rYywxYww)z.Cleanup method to be called when shutting downN)rcancelrCancelledErrorrs rcleanupzBaseRoutingStrategy.cleanup.sS ?? & OO " " $ oo%% '&))  s7'AAAAAAAAAAincrement_listttlreturncKg}|D]2\}}|j|||d{}|j|4|S7w)zB Increment a list of values in the current window keyvaluer-N)"_increment_value_in_current_windowappend)rr,r-resultsr1r2results r'_increment_value_list_in_current_windowz;BaseRoutingStrategy._increment_value_list_in_current_window7sW (JCBBu#CF NN6 " )  s"A>Ar1r2cK|jjj|||d{}t|||}|jj ||j ||S7Aw)a Increment spend within existing budget window Runs once the budget start time exists in Redis Cache (on the 2nd and subsequent requests to the same provider) - Increments the spend in memory cache (so spend instantly updated in memory) - Queues the increment operation to Redis Pipeline (using batched pipeline to optimize performance. Using Redis for multi instance environment of LiteLLM) r0N)r1increment_valuer-r1)rin_memory_cacheasync_incrementr rr4add_to_in_memory_keys_to_update)rr1r2r-r6 increment_ops rr3z6BaseRoutingStrategy._increment_value_in_current_windowEs|66FFG   7! ,,33LA ,,,5  s,A2A0AA2c@K|xst} |jd{tj|d{77"7#t$rI}t j dt|tj|d{7Yd}~Rd}~wwxYww)z Handler that triggers sync_in_memory_spend_with_redis every DEFAULT_REDIS_SYNC_INTERVAL seconds Required for multi-instance environment usage of provider budgets NzError in periodic sync task: )r _sync_in_memory_spend_with_redisrsleep Exceptionr errorstr)rres rr%zt |j}|jjj|d{}t |jDcgc] \}}||vr| c}}|_|$t||Dcic] \}}|d| }}}|Si}|Sy7`cc}}wcc}}w#t$r2} tjdt| g|_Yd} ~ yd} ~ wwxYww)av How this works: - async_log_success_event collects all provider spend increments in `redis_increment_operation_queue` - This function compresses multiple increments for the same key into a single operation - Then pushes all increments to Redis in a batched pipeline to optimize performance Only runs if Redis is initialized Nrr1r9)r,*Error syncing in-memory cache with Redis: )r redis_cachelenr enumerater4listvaluesasync_increment_pipelineziprBr rCrD) rcompressed_ops ops_to_removeidxopcompressed_queueincrement_resultr1 return_resultrEs r#_push_in_memory_increments_to_redisz7BaseRoutingStrategy._push_in_memory_increments_to_redists/ 6??..477811+0a=%a(?"&!I!I!KKL#'!,"2"23":"?a@ 488a@EAF//99IIcIRRWVWI%&.Foo55EE6F%'LS  ! + +O5P   "5c?3 :=  e $S%Z07:4%-eCJ.?%@*86t/3/# -c#h-C .Brr)__doc__rabcrtypingrrrrrr litellm._loggingr litellm.caching.cachingr litellm.caching.redis_cacher litellm.constantsr rrZrrrs2::2-G9v#vr