o yK™]Fcã@sädZddlZddlZddlZddlZddlZddlZddlZddlZddl Zddl Zddl Zddl m Z m Z mZmZzddlmZWn eyOddlZYnwdZe  ddddd d d d d ¡ ZdZdZdaejdejdejdejdejdej dej!dej"dej#di Z$ejdejdejdejdejdej dej!dej"dej#d i Z%Gd!d"„d"e &d"gd#¢¡ƒZ'dBd$d%„Z(dCd&d'„Z)dDd(d)„Z*d*d+„Z+d,d-„Z,dEd.d/„Z-dEd0d1„Z.d2d3„Z/d4d5„Z0d6d7„Z1d8d9„Z2d:d;„Z3dd?„Z5d@dA„Z6e*Z7dS)Fat Connection and networking based utility functions. **Module Overview:** :: download - download from a given url get_connections - quieries the connections belonging to a given process system_resolvers - provides connection resolution methods that are likely to be available port_usage - brief description of the common usage for a port is_valid_ipv4_address - checks if a string is a valid IPv4 address is_valid_ipv6_address - checks if a string is a valid IPv6 address is_valid_port - checks if something is a valid representation for a port is_private_address - checks if an IPv4 address belongs to a private range or not address_to_int - provides an integer representation of an IP address expand_ipv6_address - provides an IPv6 address with its collapsed portions expanded get_mask_ipv4 - provides the mask representation for a given number of bits get_mask_ipv6 - provides the IPv6 mask representation for a given number of bits .. data:: Resolver (enum) Method for resolving a process' connections. .. versionadded:: 1.1.0 .. versionchanged:: 1.4.0 Added **NETSTAT_WINDOWS**. .. versionchanged:: 1.6.0 Added **BSD_FSTAT**. .. deprecated:: 1.6.0 The SOCKSTAT connection resolver is proving to be unreliable (:trac:`23057`), and will be dropped in the 2.0.0 release unless fixed. ==================== =========== Resolver Description ==================== =========== **PROC** /proc contents **NETSTAT** netstat **NETSTAT_WINDOWS** netstat command under Windows **SS** ss command **LSOF** lsof command **SOCKSTAT** sockstat command under \*nix **BSD_SOCKSTAT** sockstat command under FreeBSD **BSD_PROCSTAT** procstat command under FreeBSD **BSD_FSTAT** fstat command under OpenBSD ==================== =========== éN)ÚconfÚenumÚlogÚ str_toolsF)ÚPROCÚproc)ÚNETSTATZnetstat)ÚNETSTAT_WINDOWSznetstat (windows))ÚSSÚss)ÚLSOFZlsof)ÚSOCKSTATÚsockstat)Ú BSD_SOCKSTATzsockstat (bsd))Ú BSD_PROCSTATzprocstat (bsd))Ú BSD_FSTATz fstat (bsd)z255.255.255.255z'FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFFÚz netstat -npWz netstat -anozss -nptuz lsof -wnPirz sockstat -4czprocstat -f {pid}zfstat -p {pid}zF^{protocol}\s+.*\s+{local}\s+{remote}\s+ESTABLISHED\s+{pid}/{name}\s*$z=^\s*{protocol}\s+{local}\s+{remote}\s+ESTABLISHED\s+{pid}\s*$ze^{protocol}\s+ESTAB\s+.*\s+{local}\s+{remote}\s+users:\(\("{name}",(?:pid=)?{pid},(?:fd=)?[0-9]+\)\)$zF^{name}\s+{pid}\s+.*\s+{protocol}\s+{local}->{remote} \(ESTABLISHED\)$zG^\S+\s+{name}\s+{pid}\s+{protocol}4\s+{local}\s+{remote}\s+ESTABLISHED$z?^\S+\s+{name}\s+{pid}\s+\S+\s+{protocol}4\s+{local}\s+{remote}$z:^\s*{pid}\s+{name}\s+.*\s+{protocol}\s+{local}\s+{remote}$zO^\S+\s+{name}\s+{pid}\s+.*\s+{protocol}\s+\S+\s+{local}\s+[-<]-[->]\s+{remote}$c@seZdZdZdS)Ú ConnectionaÌ Network connection information. .. versionchanged:: 1.5.0 Added the **is_ipv6** attribute. :var str local_address: ip address the connection originates from :var int local_port: port the connection originates from :var str remote_address: destionation ip address :var int remote_port: destination port :var str protocol: protocol of the connection ('tcp', 'udp', etc) :var bool is_ipv6: addresses are ipv6 if true, and ipv4 otherwise N)Ú__name__Ú __module__Ú __qualname__Ú__doc__©rrú6/usr/lib/python3/dist-packages/stem/util/connection.pyr sr)Z local_addressÚ local_portZremote_addressÚ remote_portÚprotocolZis_ipv6c Csì|durd}t ¡}z tj||d ¡WStjy-}z t ||t  ¡d|¡‚d}~wt  ¡dd…\}}|durF|t ¡|8}|dkrf|dusR|dkrft   d|||f¡t |||dƒYSt   d||f¡t  |||¡‚) aµ Download from the given url. .. versionadded:: 1.8.0 :param str url: uncompressed url to download from :param int timeout: timeout when connection becomes idle, no timeout applied if **None** :param int retires: maximum attempts to impose :returns: **bytes** content of the given url :raises: * :class:`~stem.DownloadTimeout` if our request timed out * :class:`~stem.DownloadFailed` if our request fails Nr)Útimeoutéééz5Failed to download from %s (%i retries remaining): %szFailed to download from %s: %s)ÚtimeÚurllibZurlopenÚreadÚsocketrÚstemZDownloadTimeoutÚsysÚexc_inforÚdebugÚdownloadZDownloadFailed)ZurlrZretriesZ start_timeÚexcZ stacktracerrrr)°s"€r)c sº|stƒ}|r |d}ntdƒ‚|s|stdƒ‚dd„‰ˆdƒˆd|||fƒt|tƒrBzt|ƒ}Wn tyAtd|ƒ‚w|d ur„tjj  |d ¡}t |ƒdkrg|t j t j t jfvrftd ||fƒ‚nt |ƒd krr|d}n|t j t j t jfvr„td ||fƒ‚|t j kr‘tjjj|dSt|j|d}z tjj |¡}Wnty¶}ztd||fƒ‚d }~wwt|jddd|rÂ|nd|rÇ|ndd}ˆd|ƒˆdd |¡ƒg} t |¡} ‡fdd„} |D]`} |  | ¡} | rI|  ¡}| d|d| ƒ\}}| d|d| ƒ\}}|r|r|r|sqé|d ¡}|dkr%d}|d vr3ˆd!|| fƒqét|||||t|ƒƒ}|  |¡ˆt|ƒƒqéˆd"t | ƒƒ| s[td#|ƒ‚| S)$a& Retrieves a list of the current connections for a given process. This provides a list of :class:`~stem.util.connection.Connection`. Note that addresses may be IPv4 *or* IPv6 depending on what the platform supports. .. versionadded:: 1.1.0 .. versionchanged:: 1.5.0 Made our resolver argument optional. .. versionchanged:: 1.5.0 IPv6 support when resolving via proc, netstat, lsof, or ss. :param Resolver resolver: method of connection resolution to use, if not provided then one is picked from among those that should likely be available for the system :param int process_pid: pid of the process to retrieve :param str process_name: name of the process to retrieve :returns: **list** of :class:`~stem.util.connection.Connection` instances :raises: * **ValueError** if neither a process_pid nor process_name is provided * **IOError** if no connections are available or resolution fails (generally they're indistinguishable). The common causes are the command being unavailable or permissions. rz)Unable to determine a connection resolverzAYou must provide a pid or process name to provide connections forcSstr t |¡dSdS©N)ÚLOG_CONNECTION_RESOLUTIONrr()ÚmsgrrrÚ_logsÿzget_connections.._logzP================================================================================z8Querying connections for resolver: %s, pid: %s, name: %szProcess pid was non-numeric: %sNTzTUnable to determine the pid of '%s'. %s requires the pid to provide the connections.rz[There's multiple processes named '%s'. %s requires a single pid to provide the connections.)ÚpidzUnable to query '%s': %sz(?P\S+)z(?P[\[\]0-9a-f.:]+)z(?P[\[\]0-9a-f.:]+)z[0-9]*z\S*)rÚlocalÚremoter/ÚnamezResolver regex: %szResolver results: %sÚ cs‚| dd¡\}}t|ƒst|ddsˆd|||fƒdSt|ƒs,ˆd|||fƒdSˆd|||fƒ| d ¡ d ¡t|ƒfS) Nú:rT)Úallow_bracketszInvalid %s address (%s): %s©NNzInvalid %s port (%s): %szValid %s:%s: %sú[ú])ÚrsplitÚis_valid_ipv4_addressÚis_valid_ipv6_addressÚ is_valid_portÚlstripÚrstripÚint)Z addr_typeZaddr_strÚlineZaddrÚport©r.rrÚ_parse_address_str3sz+get_connections.._parse_address_strr0r1rZtcp6Útcp)rDZudpzUnrecognized protocol (%s): %sz%i connections foundzNo results found using: %s)Úsystem_resolversÚIOErrorÚ ValueErrorÚ isinstanceÚstrr?r%ÚutilÚsystemZ pid_by_nameÚlenÚResolverr rrrÚ connectionsÚRESOLVER_COMMANDÚformatÚcallÚOSErrorÚRESOLVER_FILTERÚjoinÚreÚcompileÚmatchÚ groupdictÚlowerrr;Úappend)ZresolverZ process_pidZ process_nameZavailable_resolversZall_pidsZresolver_commandÚresultsr*Zresolver_regex_strrNZresolver_regexrCr@rWÚattrZ local_addrrZ remote_addrrrZconnrrBrÚget_connectionsÙsŠ     ÿ ÿ   €ÿ  û         € r]cCsÌ|durtjj ¡r d}nt ¡}|dkrtjg}n)|dkr#tjg}n |dkr,tjg}n|dkr9tj tj tjg}n tj tj tjtj g}dd„|Dƒ}tjj ¡rdt d tj¡rdt d tj¡rdtjg|}|S) aû Provides the types of connection resolvers likely to be available on this platform. .. versionadded:: 1.1.0 .. versionchanged:: 1.3.0 Renamed from get_system_resolvers() to system_resolvers(). The old name still works as an alias, but will be dropped in Stem version 2.0.0. :param str system: system to get resolvers for, this is determined by platform.system() if not provided :returns: **list** of :data:`~stem.util.connection.Resolver` instances available on this platform NZGentooZWindowsZDarwinZOpenBSDZFreeBSDcSs"g|] }tjj t|¡r|‘qSr)r%rJrKÚ is_availablerO)Ú.0ÚrrrrÚ Šs"z$system_resolvers..z /proc/net/tcpz /proc/net/udp)r%rJrKZ is_gentooÚplatformrMr r rrrrr r rr^ÚosÚaccessÚR_OKr)rKZ resolversrrrrEas"    ( rEc Cstdurvt ¡}tj tj t¡d¡}zG| |¡i}|  di¡  ¡D]3\}}|  ¡r2||t |ƒ<q#d|vrQ|  dd¡\}}tt |ƒt |ƒdƒD]}|||<qIq#td|ƒ‚|aWntyu} zt d|| f¡WYd} ~ nd} ~ wwtszdSt|tƒr‡|  ¡r‡t |ƒ}t  |¡S)z÷ Provides the common use of a given port. For example, 'HTTP' for port 80 or 'SSH' for 22. .. versionadded:: 1.2.0 :param int port: port number to look up :returns: **str** with a description for the port, **None** if none is known Nz ports.cfgrAú-rz'%s' is an invalid keyzEBUG: stem failed to load its internal port descriptions from '%s': %s)Ú PORT_USESrZConfigrcÚpathrTÚdirnameÚ__file__ÚloadÚgetÚitemsÚisdigitr?ÚsplitÚrangerGÚ ExceptionrÚwarnrHrI) rAZconfigZ config_pathZ port_usesÚkeyÚvalueZmin_portZmax_portZ port_entryr*rrrÚ port_usage”s2  ÿ €ÿ rucCst|tƒr t |¡}ntj |¡sdS| d¡dkrdS| d¡D]$}|  ¡r3t |ƒdks3t |ƒdkr6dS|ddkrEt |ƒdkrEdSq!dS) z© Checks if a string is a valid IPv4 address. :param str address: string to be checked :returns: **True** if input is a valid IPv4 address, **False** otherwise FÚ.r réÿÚ0rT) rHÚbytesrÚ _to_unicoder%rJÚ_is_strÚcountrornr?rL)ÚaddressÚentryrrrr:Âs   €r:cCsPt|tƒr t |¡}ntj |¡sdS|r%| d¡r%| d¡r%|dd…}|  d¡dkrs|  dd |  d¡¡d}|  d|d¡}|dkrFd }t |||…ƒsPdS|d kr\|d |d…nd d |rh||dd …nd g}d  td |ƒ¡}|  d¡}|d kr~dS|d krˆd |vrˆdS|  d ¡dks“d|vr•dS| d¡D] }t d|¡s¥dSqšdS)zî Checks if a string is a valid IPv6 address. :param str address: string to be checked :param bool allow_brackets: ignore brackets which form '[address]' :returns: **True** if input is a valid IPv6 address, **False** otherwise Fr7r8réÿÿÿÿrvr r4rNzff:fféú::z:::z^[0-9a-fA-f]{0,4}$T)rHryrrzr%rJr{Ú startswithÚendswithr|ÚrfindÚfindr:rTÚfilterrorUrW)r}r5Ú ipv4_startÚipv4_endÚ addr_compZ colon_countr~rrrr;ßs8   6  ÿr;cCsšz!t|ƒ}t|ƒt|ƒkrWdS|r|dkrWdS|dko |dkWStyCt|ttfƒr@|D] }t||ƒs<YdSq1YdSYdStyLYdSw)a+ Checks if a string or int is a valid port number. :param list,str,int entry: string, integer or list to be checked :param bool allow_zero: accept port number of zero (reserved by definition) :returns: **True** if input is an integer and within the valid port range, **False** otherwise FrTi)r?rIÚ TypeErrorrHÚtupleÚlistr<rG)r~Z allow_zerortrArrrr<s$    ÿ ÿr<cCsjt|ƒs td|ƒ‚| d¡s| d¡s| d¡rdS| d¡r3t| d¡dƒ}|d kr3|d kr3dSd S) a“ Checks if the IPv4 address is in a range belonging to the local network or loopback. These include: * Private ranges: 10.*, 172.16.* - 172.31.*, 192.168.* * Loopback: 127.* .. versionadded:: 1.1.0 :param str address: string to be checked :returns: **True** if input is in a private range, **False** otherwise :raises: **ValueError** if the address isn't a valid IPv4 address z'%s' isn't a valid IPv4 addressz10.z192.168.z127.Tz172.rvrééF)r:rGr‚r?ro)r}Z second_octetrrrÚis_private_address6s  rcCstt|ƒdƒS)zÜ Provides an integer representation of a IPv4 or IPv6 address that can be used for sorting. .. versionadded:: 1.5.0 :param str address: IPv4 or IPv6 address :returns: **int** representation of the address r)r?Ú_address_to_binary©r}rrrÚaddress_to_intZsr’c sjt|ƒs td|ƒ‚| d¡dkrk| dd| d¡¡d}| d|d¡}|dkr+d}t|||…ƒ‰‡fd d „td ƒDƒ}d d d „|Dƒ¡}|dkrT|d|d…nd||r`||dd…ndg}d td|ƒ¡}d |vr€d| d¡}|  d d d|¡}tdƒD].}|d}|dkr”|  d|¡nt |ƒ} d| |} | dkr²|d|…d| ||d…}q„|S)a Expands abbreviated IPv6 addresses to their full colon separated hex format. For instance... :: >>> expand_ipv6_address('2001:db8::ff00:42:8329') '2001:0db8:0000:0000:0000:ff00:0042:8329' >>> expand_ipv6_address('::') '0000:0000:0000:0000:0000:0000:0000:0000' >>> expand_ipv6_address('::ffff:5.9.158.75') '0000:0000:0000:0000:0000:ffff:0509:9e4b' :param str address: IPv6 address to be expanded :raises: **ValueError** if the address can't be expanded due to being malformed z'%s' isn't a valid IPv6 addressrvr r4rrrNcó$g|]}ˆd|d|d…‘qS©rrr©r_Úi©Zipv4_binrrra‘ó$z'expand_ipv6_address..rcSóg|] }dt|dƒ‘qS©z%04xr©r?©r_Úgrouprrrra’órr€ééérx) r;rGr|r„r…rrprTr†ÚreplaceÚindexrL) r}r‡rˆÚ groupingsZ ipv6_snippetr‰Zmissing_groupsr£ÚstartÚendZ missing_zerosrr—rÚexpand_ipv6_addressls. 6   €r§csn|dks|dkrtd|ƒ‚|dkrtStd|ddƒddd…‰‡fdd „td ƒDƒ}d  d d „|Dƒ¡S) a! Provides the IPv4 mask for a given number of bits, in the dotted-quad format. :param int bits: number of bits to be converted :returns: **str** with the subnet mask representation for this many bits :raises: **ValueError** if given a number of bits outside the range of 0-32 é rz$A mask can only be 0-32 bits, got %irrNrcr“)rŸrrr•©Úmask_binrrra½r˜z!get_mask_ipv4..r¡rvcSsg|] }tt|dƒƒ‘qS)r)rIr?©r_ZoctetrrrraÀrž)rGÚFULL_IPv4_MASKÚ _get_binaryrprT)ÚbitsZoctetsrr©rÚ get_mask_ipv4©s r¯csr|dks|dkrtd|ƒ‚|dkrtStd|ddƒddd…‰‡fdd „td ƒDƒ}d  d d „|Dƒ¡ ¡S) a, Provides the IPv6 mask for a given number of bits, in the hex colon-delimited format. :param int bits: number of bits to be converted :returns: **str** with the subnet mask representation for this many bits :raises: **ValueError** if given a number of bits outside the range of 0-128 é€rz%A mask can only be 0-128 bits, got %irrNrcr“r”rr•r©rrraØr˜z!get_mask_ipv6..rŸr4cSr™ršr›rœrrrraÛrž)rGÚFULL_IPv6_MASKr­rprTÚupper)r®r¤rr©rÚ get_mask_ipv6Ãs r³cCsLt|ƒs td|ƒ‚t|ƒ}t d|¡}|r dt| ¡dƒStd|ƒ‚)a9 Provides the number of bits that an IPv4 subnet mask represents. Note that not all masks can be represented by a bit count. :param str mask: mask to be converted :returns: **int** with the number of bits represented by the mask :raises: **ValueError** if the mask is invalid or can't be converted z'%s' is an invalid subnet maskz ^(1*)(0*)$r¨rz)Unable to convert mask to a bit count: %s)r:rGrrUrWrLÚgroups)ÚmaskrªZ mask_matchrrrÚ_get_masked_bitsÞs   r¶cs$d ‡fdd„t|dddƒDƒ¡S)zº Provides the given value as a binary string, padded with zeros to the given number of bits. :param int value: value to be converted :param int bits: number of bits to pad to rcsg|] }tˆ|?d@ƒ‘qS)r)rI)r_Úy©rtrrraóz_get_binary..rr)rTrp)rtr®rr¸rr­÷s$ r­cCsXt|ƒrd dd„| d¡Dƒ¡St|ƒr&t|ƒ}d dd„| d¡Dƒ¡Std|ƒ‚)zÊ Provides the binary value for an IPv4 or IPv6 address. :returns: **str** with the binary representation of this address :raises: **ValueError** if address is neither an IPv4 nor IPv6 address rcSsg|] }tt|ƒdƒ‘qS)rŸ©r­r?r«rrrraržz&_address_to_binary..rvcSsg|] }tt|dƒdƒ‘qS)rrº)r_Úgroupingrrrrar¹r4z''%s' is neither an IPv4 or IPv6 address)r:rTror;r§rGr‘rrrrs   rr6)NNNr+)F)8rÚ collectionsrcrbrUr$r&r!r%Z stem.utilZstem.util.procZstem.util.systemrrrrZurllib.requestZrequestr"Ú ImportErrorZurllib2r,ÚEnumrMr¬r±rgrrr r r r rrrrOrSÚ namedtuplerr)r]rErur:r;r<rr’r§r¯r³r¶r­rZget_system_resolversrrrrÚsŽ6  ÿ÷ çç  ) 3.  7 $=