vKg{ISSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK J r J r SSK J r J r SSKJrJrJrJrJr SSKJrJrJr \R0"\5rSrSS.S \4S jjr"S S 5r"S S5r"SS5r g)N) OrderedDict defaultdict)datetime timedelta) DefaultDictIteratorListOptionalUnion)RequestResponseWebSocketMessageF) memory_onlyrc U(a=[RS5 [URS5URS5S9$[RS5 [ URS5S9$)aCreate a new storage instance. Args: memory_only: When True, an in-memory implementation will be used which stores request data in memory only and nothing on disk. Default False. kwargs: Any arguments to initialise the storage with: - base_dir: The base directory under which requests are stored - maxsize: The maximum number of requests the storage can hold Returns: A request storage implementation, currently either RequestStorage (default) or InMemoryRequestStorage when memory_only is set to True. zUsing in-memory request storagebase_dirmaxsize)rrzUsing default request storage)r)loginfoInMemoryRequestStoragegetRequestStorage)rkwargss T/var/www/highfloat_scraper/venv/lib/python3.13/site-packages/seleniumwire/storage.pycreatersU 23%vzz*/EvzzZcOdeeHH ,- 6::j#9 ::c*\rSrSrS\S\S\4SjrSrg)_IndexedRequest*idurl has_responsec(XlX lX0lgNr r!r")selfr r!r"s r__init___IndexedRequest.__init__+s(r)r"r r!N)__name__ __module__ __qualname____firstlineno__strboolr'__static_attributes__rrrr*s)3)S))rrc\rSrSrSrS#S\\4SjjrS\SS4Sjr S \ \\ \ 4S \S \SS4S jr S \S\ SS4SjrS \S\\4SjrS \S\SS4SjrS \S\ SS4SjrS\\4SjrS \S\\4SjrSrS\\4SjrS\\ 4SjrS\\4SjrS$SjrS%S\S\S\\4SjjrS \S\4SjrS$S jr S$S!jr!S"r"g)&r1zResponsible for persistence of request and response data to disk. This implementation writes the request and response data to disk, but keeps an in-memory index for sequencing and fast retrieval. Instances are designed to be threadsafe. Nrc Uc[R"5n[RR US5Ul[RR UR SR [[R"5555Ul [R"URSS9 UR5 /Ul [[5Ul["R$"5Ulg)zInitialises a new RequestStorage using an optional base directory. Args: base_dir: The directory where request and response data is stored. If not specified, the system temp folder is used. N .seleniumwirez storage-{}T)exist_ok)tempfile gettempdirospathjoinhome_dirformatr-uuiduuid4 session_dirmakedirs_cleanup_old_dirs_indexrlist _ws_messages threadingLock_lock)r&rs rr'RequestStorage.__init__:s  **,HWW\\(OD " T]]L_get_request_dirr8mkdirr _saverGrBappendrr!)r&rI request_id request_dirs r save_requestRequestStorage.save_requestRsr & ++J7    73 ZZ KK  *++\ab cZZs *.B!! B/objdirnamefilenamec[[RRX#5S5n[R "X5 SSS5 g!,(df  g=f)Nwb)openr8r9r:pickledump)r&rUrVrWouts rrORequestStorage._savebs3 "'',,w14 8C KK !9 8 8s A  ArQresponsecURU5nUc[RSU5 gURU5nUR X$S5 SUlg)zSave a response to storage against a request with the specified id. Args: request_id: The id of the original request. response: The response to save. N6Cannot save response as request %s is no longer storedr_T)_get_indexed_requestrdebugrMrOr")r&rQr_indexed_requestrRs r save_responseRequestStorage.save_responsefsS33J?  " IINPZ [ ++J7  8*5'+$rcUR URSSnSSS5 WHnURU:XdMUs $ g!,(df  N,=fr$)rGrBr )r&rQindexrds rrb#RequestStorage._get_indexed_requestysH ZZKKNE %O!!Z/&& %Zs A AmessagecUR URURU5 SSS5 g!,(df  g=fzSave a websocket message against a request with the specified id. Args: request_id: The id of the original handshake request. message: The websocket message to save. N)rGrDrP)r&rQrjs rsave_ws_messageRequestStorage.save_ws_messages.ZZ   j ) 0 0 9ZZs 5 AentrycURU5nUc[RSU5 gURU5nUR X$S5 g)Save a HAR entry to storage against a request with the specified id. Args: request_id: The id of the original request. entry: The HAR entry to save. N7Cannot save HAR entry as request %s is no longer stored har_entry)rbrrcrMrO)r&rQrordrRs rsave_har_entryRequestStorage.save_har_entrysK33J?  " IIOQ[ \ ++J7  5{3rcUR URSSnSSS5 /nWH4nURUR5nUcM#UR U5 M6 U$!,(df  NL=f)a0Load all previously saved requests known to the storage (known to its index). The requests are returned as a list of request objects in the order in which they were saved. Each request will have any associated response and websocket messages attached if they exist. Returns: A list of request objects. N)rGrB _load_requestr rP)r&rhloadedrdrIs r load_requestsRequestStorage.load_requestsseZZKKNE$O(();); = ++// ;K&1# "'',,{J?F##~~c2H++3(#8V44+3==GL ( G>8GF&x0  3> =8sSD02D0*D8?D 7D D DD0DD-*D0,D--D00 D?c[R"U5$![a@ [R [ R 5(a[RS5 gf=f)zOUnpickle the object specified by the file f. If unpickling fails return None. zError unpickling objectN)r[load Exceptionr isEnabledForloggingDEBUG exception)r&fs rr~RequestStorage._unpicklesK ;;q> !  .. 78 sAA"!A"cUR UR(aURSnO SSS5 gSSS5 URWR5$!,(df  N)=f)oLoad the last saved request. Returns: The last saved request or None if no requests have yet been stored. NrGrBrwr )r& last_requests rload_last_request RequestStorage.load_last_requestsP ZZ{{#{{2  Z !!,//22 Zs #A A+cUR URSSnSSS5 /nWHwnURUR5n[ [ R RUS5S5nURU5nUbURU5 SSS5 My U$!,(df  N=f!,(df  M=f![a Mf=f)zMLoad all HAR entries known to this storage. Returns: A list of HAR entries. Nrsr|) rGrBrMr rZr8r9r:r~rPr)r&rhentriesrdrRrros rload_har_entriesRequestStorage.load_har_entriess ZZKKNE$O//0B0BCK "'',,{K@$G1 NN1-E(u- HG %%ZHG %  s;B&*C 2&B7C & B47 C C C  CCc## UR URSSnSSS5 WH nURUR5v M" g!,(df  N5=f7f)z_Return an iterator of requests known to the storage. Returns: An iterator of request objects. Nrr&rhrds r iter_requestsRequestStorage.iter_requests sJ ZZKKNE %O$$_%7%78 8 %Zs AA/A AAcDUR URSSnURR5 URR5 SSS5 WH1n[R "UR UR5SS9 M3 g!,(df  NF=f)z3Clear all requests currently known to this storage.NT ignore_errors)rGrBclearrDshutilrmtreerMr rs rclear_requestsRequestStorage.clear_requestssq ZZKKNE KK       # # % %O MM$//0B0BCSW X % Zs AB Bpatcheck_responsecFUR URSSnSSS5 WHfn[R"XR5(dM*U(aUR (d U(aMKUR UR5s $ g!,(df  N{=f)Find the first request that matches the specified pattern. Requests are searched in chronological order. Args: pat: A pattern that will be searched in the request URL. check_response: When a match is found, whether to check that the request has a corresponding response. Where check_response=True and no response has been received, this method will skip the request and continue searching. Returns: The first request in the storage that matches the pattern, or None if no requests match. N)rGrBresearchr!r"rwr )r&rrrhrds rfindRequestStorage.find!sqZZKKNE %Oyy1122"'C'CNN--o.@.@AA % Zs B B ct[RRURSR U55$)Nz request-{})r8r9r:r?r<r&rQs rrMRequestStorage._get_request_dir9s(ww||D,,l.A.A*.MNNrc>[RSUR5 UR5 [R "URSS9 [ R"[ RRUR55 g![a gf=f)zRemove all stored requests, the storage directory containing those requests, and if that is the only storage directory, also the top level parent directory. zCleaning up %sTrN) rrcr?rrrr8rmdirr9rVOSErrorr&s rcleanupRequestStorage.cleanup<so "D$4$45  d&&d;  HHRWW__T%5%56 7   s=B BBc[RRUR5n[R"U5Hn[RR X5n[RR U5[R"5[[S9- R5:a[R"USS9 MM g![a Mf=f)zkClean up and remove any old storage directories that were not previously cleaned up properly by cleanup(). )daysTrN)r8r9rVr?listdirr:getmtimernowrREMOVE_DATA_OLDER_THAN_DAYS timestamprrr)r& parent_dir storage_dirs rrA RequestStorage._cleanup_old_dirsKsWW__T%5%56 ::j1K'',,z?K GG$$[1||~ 7R(SS^^`aMM+TBa 2%  s#A'C CC)rBrGrDr;r?r$rJNT)#r)r*r+r,__doc__r r-r'r rSr r dictrOrerrbrrmrtr ryrwr~rrrrrr.rrMrrAr/r0rrrr1sh&#&0dGdd "w$67"#"QT"Y]",,x,D,&sx7P:#:8H:T:44T4d4"tG},0AB 38G#4 3$t*2 9x0 9YTXg=N0O3O3O rrc*\rSrSrSrSS\\S\\4SjjrS\ SS4S jr S \S \ SS4S jr S \S \ SS4SjrS \S\SS4SjrS \S\\ 4SjrS\\ 4SjrS\\ 4SjrS\\4SjrS\\ 4SjrSSjrSS\S\S\\ 4SjjrSSjrSrg)ri]zKeeps request and response data in memory only. By default there is no limit on the number of requests that will be stored. This can be adjusted with the 'maxsize' attribute when creating a new instance. Instances are designed to be threadsafe. NrrcUc[R"5n[RR US5UlUc[ ROUUl[5Ul [R"5Ul g)aInitialise a new InMemoryRequestStorage. Args: base_dir: The directory where certificate data is stored. If not specified, the system temp folder is used. maxsize: The maximum number of requests to store. Default no limit. When this attribute is set and the storage reaches the specified maximum size, old requests are discarded sequentially as new requests arrive. Nr4)r6r7r8r9r:r;sysr_maxsizer _requestsrErFrG)r&rrs rr'InMemoryRequestStorage.__init__fsV  **,HWW\\(OD '. G $^^% rrIrJc[[R"55UlUR UR S:a|[ UR5UR :a>URRSS9 [ UR5UR :aM>SU0URUR'SSS5 g!,(df  g=f)rLrF)lastrIN) r-r=r>r rGrlenrpopitem)r&rIs rrS#InMemoryRequestStorage.save_requestzs & ZZ}}q $..)T]]:NN***6$..)T]]:w.wzz* ZZsA0C"C CrQr_cURU5nUb,X#l[US5(aURUlU?gg[R SU-5 g)a Save a response to storage against a request with the specified id. Any certificate information will be attached to the original request against the request.cert attribute. Args: request_id: The id of the original request. response: The response to save. Nr}ra) _get_requestr_rr}rrc)r&rQr_rIs rre$InMemoryRequestStorage.save_responsesY##J/  ' x(('}} M) IINQ[[ \rrjcdURU5nUbURRU5 ggrl)rrrP)r&rQrjrIs rrm&InMemoryRequestStorage.save_ws_messages3##J/      & &w / rrocUR URUnX#S'SSS5 g![a [R SU5 N+f=f!,(df  g=f)rqrsrrN)rGrKeyErrorrrc)r&rQrovs rrt%InMemoryRequestStorage.save_har_entrysXZZ aNN:.!&+Z a SU_` a Zs&A+ A A AA AcUR URUSsSSS5 $![a SSS5 gf=f!,(df  g=f)z@Get a request with the specified id or None if no request found.rIN)rGrrrs rr#InMemoryRequestStorage._get_requestsI ZZ ~~j1)<Z  Z Zs!A* ?A?A AcUR URR5Vs/sHoSPM snsSSS5 $s snf!,(df  g=f)a5Load all previously saved requests. The requests are returned as a list of request objects in the order in which they were saved. Note that for efficiency request objects are not copied when returned, so any change made to a request will also affect the stored version. Returns: A list of request objects. rINrGrvaluesr&rs rry$InMemoryRequestStorage.load_requestss@ZZ*...*?*?*AB*AQiL*ABZBZsA AAA AcUR [[URR 555SsSSS5 $![ [ 4a SSS5 gf=f!,(df  g=f)rrIN)rGnextreversedrr StopIterationrrs rr(InMemoryRequestStorage.load_last_requests^ ZZ HT^^%:%:%<=>yIZ"8,  Z Zs(A%.AA"A%!A""A%% A3cUR URR5Vs/sHnSU;dM USPM snsSSS5 $s snf!,(df  g=f)zHLoad all previously saved HAR entries. Returns: A list of HAR entries. rsNrrs rr'InMemoryRequestStorage.load_har_entriessL ZZ,0NN,A,A,CX,Cq{VWGWNAkN,CXZXZs!A A  A A A Ac## UR [URR55nSSS5 WH nUSv M g!,(df  N=f7f)zVReturn an iterator over the saved requests. Returns: An iterator of request objects. NrI)rGrCrr)r&rrs rr$InMemoryRequestStorage.iter_requestssG ZZ$..//12FAI, Zs A$A A AAcUR URR5 SSS5 g!,(df  g=fz$Clear all previously saved requests.N)rGrrrs rr%InMemoryRequestStorage.clear_requestss# ZZ NN "ZZs1 ?rrcBUR URR5H[nUSn[R"XR 5(dM/U(aUR (d U(aMPUs sSSS5 $ SSS5 g!,(df  g=f)rrIN)rGrrrrr!r_)r&rrrrIs rrInMemoryRequestStorage.findsnZZ^^**,I,99S++..&7+;+;NN& Z,ZsABB8BB Bc$UR5 gr)rrs rrInMemoryRequestStorage.cleanup s r)rGrrr;)NNrr)r)r*r+r,rr r-intr'r rSr rerrmrrtrr ryrrrrrr.rrr/r0rrrr]s&#& &(G"]]x]D], 0# 08H 0T 0 a aT ad asx/@ CtG} C 8G#4 Y$t*Y x0 # TXg=N0rr)!rr8r[rrrr6rEr= collectionsrrrrtypingrrr r r seleniumwire.requestr r r getLoggerr)rrr.rrrrr0rrrs~  0(??DD ! #(;4;())iiX rrr