o n~b@sdZddlmZddlmZddlmZmZddlm Z m Z ddl m Z m Z ddlmZddlZddlZddlZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZmZddl m!Z!ddl"m#Z#ddl$m%Z%m&Z&ddl'm(Z(m)Z)ddl*m+Z+zddl,Z,Wne-yddl.m,Z,YnwddZ/d&ddZ0d'ddZ1Gddde2Z3Gd d!d!e)Z4Gd"d#d#e(Z5Gd$d%d%e2Z6dS)(a The :class:`.CDNClient` class provides a simple API for downloading Steam content from SteamPipe Initializing :class:`.CDNClient` requires a logged in :class:`.SteamClient` instance .. warning:: This module uses :mod:`requests` library, which is not gevent cooperative by default. It is high recommended that you use :meth:`steam.monkey.patch_minimal()`. See example below .. code:: python import steam.monkey steam.monkey.patch_minimal() from steam.client import SteamClient, EMsg from steam.client.cdn import CDNClient mysteam = SteamClient() mysteam.cli_login() ... mycdn = CDNClient(mysteam) Getting depot manifests for an app .. code:: python >>> mycdn.get_manifests(570) [, , , , , , , , , , , , , , , , , , , , ] >>> mycdn.get_manifests(570, filter_func=lambda depot_id, info: 'Dota 2 Content' in info['name']) [, , , , , ] Listing files .. code:: python >>> file_list = mycdn.iter_files(570) >>> list(file_list)[:10] [, , , , , , , , , Reading a file directly from SteamPipe .. code:: python >>> file_list = mycdn.iter_files(570, r'game\dota\gameinfo.gi') >>> myfile = next(file_list) >>> print(myfile.read(80).decode('utf-8')) "GameInfo" { game "Dota 2" title "Dota 2" gamelogo 1 type multiplayer_only ... )ZipFile)BytesIO) OrderedDictdeque) itervalues iteritems)crc32 unhexlify)datetimeN)Pool)LRUCache)webapi) SteamError)MsgProto)EResultEType)EMsg)make_requests_session)symmetric_decryptsymmetric_decrypt_ecb) DepotManifest DepotFile)ContentManifestPayload)lzmacCstdt||dS)zDecrypt manifest gid v2 bytes :param encrypted_gid: encrypted gid v2 bytes :type encrypted_gid: bytes :param password: encryption password :type password: byt :return: manifest gid :rtype: int zr$r%) __class____name__r$r6r7reprr)r8selfrrr__repr__s zContentServer.__repr__) rI __module__ __qualname__r$r6r-r7r)r8r/r9rMrrrrr5s r5c@seZdZddZddZeddZddZd d d Zd d Z ddZ ddZ ddZ ddZ ddZd!ddZddZddZdS)" CDNDepotFilecCsJt|ts tdt|tjstdt|||d|_d|_d|_ dS)a/File-like object proxy for content files located on SteamPipe :param manifest: parrent manifest instance :type manifest: :class:`.CDNDepotManifest` :param file_mapping: file mapping instance from manifest :type file_mapping: ContentManifestPayload.FileMapping z.Expected 'manifest' to be of type CDNDepotFilezHExpected 'file_mapping' to be of type ContentManifestPayload.FileMappingrN) isinstanceCDNDepotManifest TypeErrorrZ FileMappingr__init__offset_lc_lcbuff)rLmanifestZ file_mappingrrrrUs   zCDNDepotFile.__init__cCs:d|jj|jj|jj|jjt|j|jrdfS|j fS)Nz<%s(%s, %s, %s, %s, %s)>zis_directory=True) rHrIrYapp_iddepot_idgidrJZ filename_rawZ is_directorysizerKrrrrMszCDNDepotFile.__repr__cC|jS)z :type: bool)is_filerKrrrseekableszCDNDepotFile.seekablecCs|jstd|jS)z :type: intDThis file is not seekable, probably because its directory or symlink)r` ValueErrorrVrKrrrtell szCDNDepotFile.tellrcCsp|jstd|dkr|dkrtdn|dkr|j|}n|dkr(|j|}ntdtdt|j||_dS)zSeen file :param offset: file offset :type offset: int :param whence: offset mode, see :meth:`io.IOBase.seek` :type whence: int rarzInvalid argumentzInvalid value for whenceN)r`rbIOErrorrVr]maxmin)rLrVwhencerrrseeks  zCDNDepotFile.seekcCsB|jr |jj|jkr|jj|jj|jj|j|_||_|jSN) rWsharY cdn_client get_chunkrZr[hexrX)rLchunkrrr _get_chunk)szCDNDepotFile._get_chunkcC|SrkrrKrrr__iter__3zCDNDepotFile.__iter__cCs|Srk)nextrKrrr__next__6szCDNDepotFile.__next__cCrrrkrrKrrr __enter__9rtzCDNDepotFile.__enter__cCsdSrkr)rLr)value tracebackrrr__exit__<rtzCDNDepotFile.__exit__cCs|}|dkr t|S)NrQ)readline StopIteration)rLlinerrrru?szCDNDepotFile.nextcCs>|dkr |j|j}|dks|j|jks|jdkrdS|j|}|jrH|j|jjkrH||jj|jjkrH|j|j|jj|j|jj|}nNt}d}|jD]8}|j|krYn0|j}||j}||jkrl|ksyn||krw|krnqP|dur|j}|||qP| |j|| |}t |j||_|S)zRead bytes from the file :param length: number of bytes to read. Read the whole file if not set :type length: int :returns: file data :rtype: bytes r~rrQN) r]rVrWZ cb_originalrXrchunkswriterqrjreadrh)rLlength end_offsetdataZ start_offsetrpZ chunk_startZ chunk_endrrrrEs>   (     zCDNDepotFile.readcspd}tfdddD]*}|d}|dkr1|d7}||d|7}jt|||S||7}q |S)zTRead a single line :return: single file line :rtype: bytes rQcs dS)N)rrrKrr{ z'CDNDepotFile.readline.. r~rdN)iterfindrjrVlen)rLbufrpposrrKrr{ss  zCDNDepotFile.readlinecCsdd|DS)zzGet file contents as list of lines :return: list of lines :rtype: :class:`list` [:class:`bytes`] cSsg|]}|qSrr).0r}rrr sz*CDNDepotFile.readlines..rrKrrr readlinesszCDNDepotFile.readlinesN)r)r~)rIrNrOrUrMpropertyr`rcrjrqrsrvrwrzrurr{rrrrrrPs     . rPc@s,eZdZeZdZddZddZddZdS)rSNcCs||_||_t||dS)aHolds manifest metadata and file list. :param cdn_client: CDNClient instance :type cdn_client: :class:`.CDNClient` :param app_id: App ID :type app_id: int :param data: serialized manifest data :type data: bytes N)rmrZrrU)rLrmrZrrrrrUs zCDNDepotManifest.__init__c Csddt|jdt|jdt|jdtt|jj  ddg}|j r3t|j d|}|j r:|d7}d |jj|fS) N, zapp_id=z depot_id=zgid=zcreation_time=T z, filenames_encrypted=Truez<%s(%s)>)joinstrrZr[r\rJr utcfromtimestampmetadataZ creation_time isoformatreplacenameZfilenames_encryptedrHrI)rLrErrrrMs     zCDNDepotManifest.__repr__cCs2t|||jjD] }|jjddddq dS)NcSr^rk)rVxrrrrsz.CDNDepotManifest.deserialize..F)keyreverse)r deserializepayloadZmappingsrsort)rLrmappingrrrrs  zCDNDepotManifest.deserialize) rIrNrOrPZDepotFileClassrrUrMrrrrrrSs  rSc@seZdZeZedZeZ e dZ dZ ddZ ddZddZd#d d Zd$d d ZddZddZddZd%ddZddZddZd&ddZd'dd Zd!d"ZdS)( CDNClientr"rcCsjtd|_||_|jr|jj|_t|_i|_i|_i|_i|_ t |_ t |_ |j s/||dS)zCDNClient allows loading and reading of manifests for Steam apps are used to list and download content :param client: logged in SteamClient instance :type client: :class:`.SteamClient` N)GPoolgpoolsteamr8rweb depot_keys manifests app_depotsbeta_passwordssetlicensed_app_idslicensed_depot_idsrAfetch_content_servers load_licenses)rLclientrrrrUs   zCDNClient.__init__cCs"|j|j|jdS)zTCleared cached information. Next call on methods with caching will return fresh dataN)rclearrrrKrrr clear_caches  zCDNClient.clear_cachecCs|j|j|jjjtjkrdg}n|jjs"|j ddSt t ddt |jj}t|jj|ddD]\}}|j|d|j|d q:dS) zTRead licenses from SteamClient instance, required for determining accessible contentiEz/No steam licenses found on SteamClient instanceNcSs|j|jdS)N)Z packageid access_token) package_idr)lrrrrsz)CDNClient.load_licenses..)packagesrZappidsZdepotids)rrrrZsteam_idr)rZAnonUserZlicenses_LOGdebuglistmaprrget_product_infoupdatevalues)rLrrinforrrrs     zCDNClient.load_licensescCsL|j|jdt|j}tdd|}|j||js$tddS)zyUpdate CS server list :param num_servers: numbers of CS server to fetch :type num_servers: int z.Trying to fetch content servers from Steam APIcSs |jdkS)NZ OpenCache)r))rCrrrrrz1CDNClient.fetch_content_servers..zFailed to fetch content serversN) rArrrrGr8filterextendr)rLr;rArrrrs    zCDNClient.fetch_content_serversFcCs|r|jd|jdS)zGet a CS server for content download :param rotate: forcefully rotate server list and get a new server :type rotate: bool r~r)rArotate)rLrrrrget_content_server s  zCDNClient.get_content_servercCs`||jvr+|j||}|r|jtjkr|j|j|<ntd|dur%tjt|j|j|S)aGet depot key, which is needed to decrypt files :param app_id: app id :type app_id: int :param depot_id: depot id :type depot_id: int :return: returns decryption key :rtype: bytes :raises SteamError: error message zFailed getting depot keyN) rr get_depot_keyeresultrOKZdepot_encryption_keyrTimeout)rLrZr[msgrrrrs   zCDNClient.get_depot_keyc Cs|} d|jr dnd|j|j||f}z |jj|dd}Wnty8}z |jd|WYd}~n,d}~ww|j r>|Sd |j krId krZnn|jd |j t d |j |j d |jdd}q)a Run CDN command request :param command: command name :type command: str :param args: args :type args: str :returns: requests response :rtype: :class:`requests.Response` :raises SteamError: on error Tz%s://%s:%s/%s/%sr$r% timeoutzRequest error: %sNiiz Got HTTP %sz HTTP Error %sg?)r)rr$r6r7rr0 Exceptionrrokr1rrsleep)rLcommandargsrCr>r?exprrrcdn_cmd*s.    zCDNClient.cdn_cmdc Cs^||f|jvr|dd||f}t|j|||}|dddkr|dddkr7tdt|dd|dd d krKtd t|dd ttj |d d }tj tj |gd}t d|dd\}} ||d dd| }t||krtdntt|} | | jd}Wdn1swY||j||f<|j||fS)a=Download a single content chunk :param app_id: App ID :type app_id: int :param depot_id: Depot ID :type depot_id: int :param chunk_id: Chunk ID :type chunk_id: int :returns: chunk data :rtype: bytes :raises SteamError: error message depotz %s/chunk/%sNresVZszvzVZ: Invalid footer: %sazVZ: Invalid version: %s )filtersz.z"App beta password check failed. %r)rZsend_job_and_waitrrZClientCheckAppBetaPasswordrrrrrrrZ betapasswordsr rrrr)rLrZrr?rBrrrcheck_beta_passwords    zCDNClient.check_beta_passwordcCs4||jvr|j|gd|d|j|<|j|S)NZappsdepots)rrr)rLrZrrrget_app_depot_infos   zCDNClient.get_app_depot_infopublicNcs6|}d}||divrtdt||ft|d|dddkrVd}||fjvrV|s8td|||}|tjkrItd|||fjvrVtd |fd d } g} i} t |D]\} } | smqdt| } |ry|| | syqd|r| j vr| j vrj d t| d | | qdd| vr| t| dt| qd|r| di|id}|durtt|j||f}n| did}n | di|}|dur| j| || ||| d | qdg}| D] }||qt | D]\}}||fdd}|j||d7}q|S)aGet a list of CDNDepotManifest for app :param app_id: App ID :type app_id: int :param branch: branch name :type branch: str :param password: branch password for locked branches :type password: str :param filter_func: Function to filter depots. ``func(depot_id, depot_info)`` :returns: list of :class:`.CDNDepotManifest` :rtype: :class:`list` [:class:`.CDNDepotManifest`] :raises SteamError: error message FZbranchesz No branch named %s for app_id %sZ pwdrequiredrTzBranch %r requires a passwordz Branch password is not valid. %rz Incorrect password for branch %rcs||||}||_|Srk)rr)rZr[rrrrYrKrrasync_fetch_manifestsz5CDNClient.get_manifests..async_fetch_manifestz)No license for depot %s (%s). Skipping...rZ depotfromappZencryptedmanifestsZencrypted_gid_2NrrcSst||vo|dup|||Srk)int)r[ depot_info depot_idsZffuncrrr nested_ffuncs z-CDNClient.get_manifests..nested_ffunc) filter_func)rr0rrJrrrrrrisdigitrrrr setdefaultraddrr r:rspawn get_manifests)rLrZbranchrrrrZ is_enc_branchresultrtasksZ shared_depotsr[rZegidrrtaskrrrrKrrsx            zCDNClient.get_manifestsccs2|||||D] }||D]}|Vqq dS)a[Like :meth:`.get_manifests` but returns a iterator that goes through all the files in all the manifest. :param app_id: App ID :type app_id: int :param filename_filter: wildcard filter for file paths :type branch: str :param branch: branch name :type branch: str :param password: branch password for locked branches :type password: str :param filter_func: Function to filter depots. ``func(depot_id, depot_info)`` :returns: generator of of CDN files :rtype: [:class:`.CDNDepotFile`] N)r iter_files)rLrZZfilename_filterrrrrYfprrrr!s zCDNClient.iter_filesc Cs|jjd|gdddddddddd dd}|jjtjkr't|jjp"d|jj|d ur-d n|jj d}|d us=|j tjkrMtd |d urGtj t|j |j sVtd tj |j}}||||j }|j|_|S) aGet the manifest file for a worshop item that is hosted on SteamPipe :param item_id: Workshop ID :type item_id: int :returns: manifest instance :rtype: :class:`.CDNDepotManifest` :raises SteamError: error message zPublishedFile.GetDetails#1FTr) ZpublishedfileidsZ includetagsZincludeadditionalpreviewsZincludechildrenZ includekvtagsZ includevotesZshort_descriptionZincludeforsaledataZincludemetadatalanguagerrz No messageNz!Failed getting workshop file infoz!Workshop file is not on SteamPipe)rZsend_um_and_waitheaderrrrr error_messagebodyZpublishedfiledetailsrrZ hcontent_fileZ FileNotFoundZconsumer_appidrtitler)rLZitem_idr?ZwfrZZ ws_app_idrYrrrget_manifest_for_workshop_item6s:      z(CDNClient.get_manifest_for_workshop_itemr")F)T)rNNT)NrNN)rIrNrOrSrlogging getLoggerrrrAr rr8rUrrrrrrrnrrrrrrrrrrrs(    $ )  u r)r r!r"Nr)7__doc__zipfilerior collectionsrrsixrrbinasciirr r rrr2Z gevent.poolr rZ cachetoolsr rr Zsteam.exceptionsrZsteam.core.msgrZ steam.enumsrrZsteam.enums.emsgrZsteam.utils.webrZsteam.core.cryptorrZsteam.core.manifestrrZ$steam.protobufs.content_manifest_pb2rr ImportErrorZ backportsrrDrGobjectr5rPrSrrrrrsD _             /(0