o ßOæ_|[ã@s4dZddlZddlZddlZddlZddlZddlZddlmZmZm Z zddl m Z Wne y;ddl m Z YnwzddlmZWn e yQddlZYnwdZdZej ej e¡d¡Ze d¡Ze d ¡Ze d ¡Ze d ¡Zd Ze d ¡Ze d¡Z e d¡Z!e d¡Z"e d¡Z#dTdd„Z$dTdd„Z%Gdd„de&ƒZ'Gdd„de'ƒZ(Gdd„de'ƒZ)dd„Z*e(ddd d!d"d#d$e(d%d&d'd(d)d*d+d,e(d-d.d'd(d/d0d$e(d1d2d'd(d3d4d5d,e(d6d7d'd(d8d9d:d;e(dd?d@d,e(dAdBd'd(dCdDd$e(dEdFd'd(dGdHd$e(dIdJd'd(dKdLdMd,e(dNdOdPdQdRdd$dSœ Z+dS)Uaç Directories that provide `relay descriptor information <../tutorials/mirror_mirror_on_the_wall.html>`_. At a very high level tor works as follows... 1. Volunteer starts a new tor relay, during which it sends a `server descriptor `_ to each of the directory authorities. 2. Each hour the directory authorities make a `vote `_ that says who they think the active relays are in the network and some attributes about them. 3. The directory authorities send each other their votes, and compile that into the `consensus `_. This document is very similar to the votes, the only difference being that the majority of the authorities agree upon and sign this document. The idividual relay entries in the vote or consensus is called `router status entries `_. 4. Tor clients (people using the service) download the consensus from an authority, fallback, or other mirror to determine who the active relays in the network are. They then use this to construct circuits and use the network. :: Directory - Relay we can retrieve descriptor information from | |- from_cache - Provides cached information bundled with Stem. | +- from_remote - Downloads the latest directory information from tor. | |- Authority - Tor directory authority +- Fallback - Mirrors that can be used instead of the authorities .. versionadded:: 1.7.0 éN)Ú connectionÚ str_toolsÚ tor_tools)Ú OrderedDictzHhttps://gitweb.torproject.org/tor.git/plain/src/app/config/auth_dirs.inczLhttps://gitweb.torproject.org/tor.git/plain/src/app/config/fallback_dirs.inczcached_fallbacks.cfgz"(\S+) orport=(\d+) .*"z"v3ident=([\dA-F]{40}) "z"ipv6=\[([\da-f:]+)\]:(\d+) "z!"([\d\.]+):(\d+) ([\dA-F ]{49})",z /* ===== */z/\*\s+(\S+)=(\S*)\s+\*/z0"([\d\.]+):(\d+) orport=(\d+) id=([\dA-F]{40}).*z/\* nickname=(\S+) \*/z/\* extrainfo=([0-1]) \*/z" ipv6=\[([\da-f:]+)\]:(\d+)"c Cs|i}|D]#}|D]}| t |¡¡}|r&| ¡}t|ƒdkr |n|d||<qq|r<|D]}||vr;tdd |¡ƒ‚q,|S)a÷ Scans the given content against a series of regex matchers, providing back a mapping of regexes to their capture groups. This maping is with the value if the regex has just a single capture group, and a tuple otherwise. :param list lines: text to parse :param list regexes: regexes to match against :param list required: matches that must be in the content :returns: **dict** mapping matchers against their capture groups :raises: **ValueError** if a required match is not present érz(Failed to parse mandatory data from: %sÚ )ÚsearchrÚ _to_unicodeÚgroupsÚlenÚ ValueErrorÚjoin) ÚlinesÚregexesÚrequiredÚmatchesÚlineZmatcherÚmZ match_groupsZrequired_matcher©rú0/usr/lib/python3/dist-packages/stem/directory.pyÚ _match_withQs€ûÿrccs0||ƒ}|rt|||ƒV||ƒ}|sdSdS©N)r)rZpop_section_funcrrZ next_sectionrrrÚ_directory_entriesrs € þrc@sJeZdZdZdd„Zedd„ƒZeddd„ƒZd d „Zd d „Z d d„Z dS)Ú Directorya Relay we can contact for descriptor information. Our :func:`~stem.directory.Directory.from_cache` and :func:`~stem.directory.Directory.from_remote` functions key off a different identifier based on our subclass... * :class:`~stem.directory.Authority` keys off the nickname. * :class:`~stem.directory.Fallback` keys off fingerprints. This is because authorities are highly static and canonically known by their names, whereas fallbacks vary more and don't necessarily have a nickname to key off of. :var str address: IPv4 address of the directory :var int or_port: port on which the relay services relay traffic :var int dir_port: port on which directory information is available :var str fingerprint: relay fingerprint :var str nickname: relay nickname :var str orport_v6: **(address, port)** tuple for the directory's IPv6 ORPort, or **None** if it doesn't have one cCsZ|rd||fn|}t |¡std||fƒ‚t |¡s$td||fƒ‚t |¡s1td||fƒ‚t |¡s>td||fƒ‚|rMt |¡sMtd||fƒ‚|r†t|tƒrZt |ƒdkrdtd|t |ƒfƒ‚t  |d ¡sutd ||d fƒ‚t |d ¡s†td ||d fƒ‚||_ t |ƒ|_t |ƒ|_||_||_|r¨|d t |d ƒf|_dSd|_dS) Nú%s (%s)z"%s has an invalid IPv4 address: %sz%s has an invalid ORPort: %sz%s has an invalid DirPort: %sz!%s has an invalid fingerprint: %sz%s has an invalid nickname: %séz,%s orport_v6 should be a two value tuple: %srz"%s has an invalid IPv6 address: %srz%s has an invalid IPv6 port: %s)rZis_valid_ipv4_addressr Z is_valid_portrÚis_valid_fingerprintZis_valid_nicknameÚ isinstanceÚtupler ÚstrZis_valid_ipv6_addressÚaddressÚintÚor_portÚdir_portÚ fingerprintÚnicknameÚ orport_v6)Úselfr r"r#r$r%r&Ú identifierrrrÚ__init__’s0      (zDirectory.__init__cCótdƒ‚)aÄ Provides cached Tor directory information. This information is hardcoded into Tor and occasionally changes, so the information provided by this method may not necessarily match the latest version of tor. .. versionadded:: 1.5.0 .. versionchanged:: 1.7.0 Support added to the :class:`~stem.directory.Authority` class. :returns: **dict** of **str** identifiers to :class:`~stem.directory.Directory` instances úKUnsupported Operation: this should be implemented by the Directory subclass©ÚNotImplementedErrorrrrrÚ from_cache¯szDirectory.from_cacheé<cCr*)aù Reads and parses tor's directory data `from gitweb.torproject.org `_. Note that while convenient, this reliance on GitWeb means you should alway call with a fallback, such as... :: try: authorities = stem.directory.Authority.from_remote() except IOError: authorities = stem.directory.Authority.from_cache() .. versionadded:: 1.5.0 .. versionchanged:: 1.7.0 Support added to the :class:`~stem.directory.Authority` class. :param int timeout: seconds to wait before timing out the request :returns: **dict** of **str** identifiers to their :class:`~stem.directory.Directory` :raises: **IOError** if unable to retrieve the fallback directories r+r,©ÚtimeoutrrrÚ from_remoteÁszDirectory.from_remotec Cstj |dddddd¡S)Nr r"r#r$r%r&)ÚstemÚutilÚ _hash_attr©r'rrrÚ__hash__ÞszDirectory.__hash__cCót|tƒr t|ƒt|ƒkSdS©NF)rrÚhash©r'ÚotherrrrÚ__eq__áózDirectory.__eq__cCó ||k Srrr;rrrÚ__ne__äó zDirectory.__ne__N©r/) Ú__name__Ú __module__Ú __qualname__Ú__doc__r)Ú staticmethodr.r2r7r=r@rrrrrzs   rcs`eZdZdZd‡fdd„ Zedd„ƒZedd d „ƒZed d „ƒZd d„Z dd„Z dd„Z ‡Z S)Ú Authoritya Tor directory authority, a special type of relay `hardcoded into tor `_ to enumerate the relays in the network. .. versionchanged:: 1.3.0 Added the is_bandwidth_authority attribute. .. versionchanged:: 1.7.0 Added the orport_v6 attribute. .. deprecated:: 1.7.0 The is_bandwidth_authority attribute is deprecated and will be removed in the future. :var str v3ident: identity key fingerprint used to sign votes and consensus NFc s\tt|ƒ ||||||¡|r&t |¡s&|rd||fn|} td| |fƒ‚||_||_dS)Nrz%s has an invalid v3ident: %s)ÚsuperrHr)rrr Úv3identÚis_bandwidth_authority) r'r r"r#r$r%r&rJrKr(©Ú __class__rrr)ûs  zAuthority.__init__cCsttƒSr)ÚdictÚDIRECTORY_AUTHORITIESrrrrr.szAuthority.from_cacher/c Cszt tjt|d ¡¡ ¡}|stdƒ‚Wnt  ¡dd…\}}dt|f}t   t|||¡‚z=i}t |t jttttfttfdD](}| t¡\}}| t¡\} } } t | || |  dd¡|| t¡| t¡d ||<qDW|Sty}ztt|ƒƒ‚d}~ww) Nr0ú no contentréz:Unable to download tor's directory authorities from %s: %s©rú Ú)r r"r#r$r%r&rJ)rr ÚurllibÚurlopenÚGITWEB_AUTHORITY_URLÚreadÚ splitlinesÚIOErrorÚsysÚexc_infor3ÚDownloadFailedrrHÚ _pop_sectionÚAUTHORITY_NAMEÚAUTHORITY_V3IDENTÚAUTHORITY_IPV6ÚAUTHORITY_ADDRÚgetÚreplacer r) r1rÚexcÚ stacktraceÚmessageÚresultsrr%r"r r#r$rrrr2 s:ÿ "  ùüý €ÿzAuthority.from_remotecCsPg}|r&| | d¡¡|r&|d d¡r&| | d¡¡|r&|d d¡s|S)z, Provides the next authority entry. rrS)ÚappendÚpopÚ startswith)rÚ section_linesrrrr^0sÿzAuthority._pop_sectioncCótjj|ddtddS)NrJrKT©ÚparentÚcache©r3r4r5rr6rrrr7@ózAuthority.__hash__cCr8r9)rrHr:r;rrrr=Cr>zAuthority.__eq__cCr?rrr;rrrr@FrAzAuthority.__ne__)NNNNNNNFrB) rCrDrErFr)rGr.r2r^r7r=r@Ú __classcell__rrrLrrHès  & rHcsreZdZdZd‡fdd„ Zeddd„ƒZedd d „ƒZed d „ƒZee fd d„ƒZ dd„Z dd„Z dd„Z ‡ZS)ÚFallbacka¦ Particularly stable relays tor can instead of authorities when bootstrapping. These relays are `hardcoded in tor `_. For example, the following checks the performance of tor's fallback directories... :: import time from stem.descriptor.remote import get_consensus from stem.directory import Fallback for fallback in Fallback.from_cache().values(): start = time.time() get_consensus(endpoints = [(fallback.address, fallback.dir_port)]).run() print('Downloading the consensus took %0.2f from %s' % (time.time() - start, fallback.fingerprint)) :: % python example.py Downloading the consensus took 5.07 from 0AD3FA884D18F89EEA2D89C019379E0E7FD94417 Downloading the consensus took 3.59 from C871C91489886D5E2E94C13EA1A5FDC4B6DC5204 Downloading the consensus took 4.16 from 74A910646BCEEFBCD2E874FC1DC997430F968145 ... .. versionadded:: 1.5.0 .. versionchanged:: 1.7.0 Added the has_extrainfo and header attributes which are part of the `second version of the fallback directories `_. :var bool has_extrainfo: **True** if the relay should be able to provide extrainfo descriptors, **False** otherwise. :var collections.OrderedDict header: metadata about the fallback directory file this originated from NFc s>tt|ƒ ||||||¡||_|rt|ƒ|_dStƒ|_dSr)rIrtr)Ú has_extrainforÚheader) r'r r"r#r$r%rur&rvrLrrr)qszFallback.__init__c s|durt}tjj ¡‰ˆ |¡t‡fdd„ˆ ¡Dƒƒ}i}tdd„ˆ ¡DƒƒD]^}|dvr2q+i}dD]}d||f}ˆ  |¡||<||sU|dvrUt d|tfƒ‚q6|d ri|d ri|d t |d ƒf}nd}t |d t |d ƒt |d ƒ||d|ddk||d||<q+|S)Ncs0g|]}| d¡r| dd¡dˆ |¡f‘qS)zheader.Ú.r)rkÚsplitrc)Ú.0Úk©ÚconfrrÚ |s0z'Fallback.from_cache..cSsg|] }| d¡d‘qS)rwr)rx)ryÚkeyrrrr}€s)Ú tor_commitÚ stem_commitrv)r r"r#r%ruÚorport6_addressÚ orport6_portz%s.%s)r%rurr‚z'%s' is missing from %srr‚r r"r#r%ruÚtrue©r r"r#r$r%rur&rv) ÚFALLBACK_CACHE_PATHr3r4r|ÚConfigÚloadrÚkeysÚsetrcrZr!rt)ÚpathÚheadersrhr$ÚattrZ attr_namer~r&rr{rr.vs<   €    ø zFallback.from_cacher/c Csdzt tjt|d ¡¡ ¡}|stdƒ‚Wnt  ¡dd…\}}dt|f}t   t|||¡‚|ddkr>tdtƒ‚i}t   |¡D]}t |¡}|rY| d ¡|| d¡<qEtd |ƒ‚t   |¡z;i}t|t j ttttftfd D]'} | t\} } } } t | t| ƒt| ƒ| |  t¡|  t¡d k|  t¡|d || <quW|Sty±}ztt|ƒƒ‚d}~ww)Nr0rPrrQz9Unable to download tor's fallback directories from %s: %srz/* type=fallback */zJ%s does not have a type field indicating it is fallback directory metadatarz,Malformed fallback directory header line: %srRÚ1r„)rr rUrVÚGITWEB_FALLBACK_URLrXrYrZr[r\r3r]rtr^ÚFALLBACK_MAPPINGÚmatchÚgrouprÚ FALLBACK_ADDRÚFALLBACK_NICKNAMEÚFALLBACK_EXTRAINFOÚ FALLBACK_IPV6r!rcr r)r1rrerfrgrvrÚmappingrhrr r#r"r$rrrr2ŸsLÿ          øýý €ÿzFallback.from_remotecCsNg}|r%| d¡}|r%|tkr%| ¡dkr| |¡| d¡}|r%|tks|S)z“ Provides lines up through the next divider. This excludes lines with just a comma since they're an artifact of these being C strings. rú,)rjÚ FALLBACK_DIVÚstripri)rrlrrrrr^Øs      üzFallback._pop_sectionc Cstjj ¡}| d|¡| d|¡| ¡D] \}}| d||¡qt| ¡dd„dD]W}|j} | d| |j ¡| d| t |j ƒ¡| d | t |j ƒ¡| d | |j ¡| d | |jrdd nd ¡|jr„| d| t |jdƒ¡| d| t |jdƒ¡q-| |¡dS)a° Persists fallback directories to a location in a way that can be read by from_cache(). :param dict fallbacks: mapping of fingerprints to their fallback directory :param str tor_commit: tor commit the fallbacks came from :param str stem_commit: stem commit the fallbacks came from :param dict headers: metadata about the file these came from :param str path: location fallbacks will be persisted to rr€z header.%scSs|jSr)r$)ÚxrrrÚsz!Fallback._write..)r~z %s.addressz %s.or_portz %s.dir_portz %s.nicknamez%s.has_extrainforƒZfalsez%s.orport6_addressrz%s.orport6_portrN)r3r4r|r†r‰ÚitemsÚsortedÚvaluesr$r rr"r#r%rur&Zsave) Z fallbacksrr€r‹rŠr|rzÚvÚ directoryr$rrrÚ_writeìs"  €zFallback._writecCrm)NrurvTrnrqr6rrrr7rrzFallback.__hash__cCr8r9)rrtr:r;rrrr=r>zFallback.__eq__cCr?rrr;rrrr@rAzFallback.__ne__)NNNNNFNNrrB)rCrDrErFr)rGr.r2r^r…r¡r7r=r@rsrrrLrrtJs& ( 8 !rtc Cs(g}t| ¡ƒ | ¡¡}t| ¡ƒ | ¡¡}|D]0}||}|jr(d|jnd}|d|jd|jd|jd|jd|jd|j d |d g7}q|D] }|  d |¡qM|D]5}||vsc||vrdqY||}||} || krŽd D]} t || ƒ} t | | ƒ} | | kr|  d | || | f¡qrqYd  |¡S)z@ Provides a description of how fallback directories differ. z%s:%sz[none]z'* Added %s as a new fallback directory:z address: %sz or_port: %sz dir_port: %sz nickname: %sz has_extrainfo: %sz orport_v6: %srTz$* Removed %s as a fallback directory)r r"r#r$r&z$* Changed the %s of %s from %s to %sr) r‰rˆÚ differencer&r$r r"r#r%ruriÚgetattrr ) Zprevious_directoriesZnew_directoriesrZadded_fpZ removed_fpÚfpr r&Zprevious_directoryZ new_directoryrŒZold_attrZnew_attrrrrÚ_fallback_directory_differencess>ø   € r¥Úmoria1z 128.31.0.39i#i«#Z(9695DFC35FFEB861329B9F1AB04C46397020CE31Z(D586D18309DED4CD6D57C18FDB97EFA96D330566)r%r r"r#r$rJÚtor26z 86.59.21.38é»éPZ(847B1F850344D7876491A54892F904934E4EB85D)z2001:858:2:2:aabb:0:563b:1526r¨Z(14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4)r%r r"r#r$r&rJÚdizumz 45.66.33.45Z(7EA6EAD6FD83083C538F44038BBFA077587DD755Z(E8A9C45EDE6D711294FADF8E7951F4DE6CA56B58Úgabelmooz131.188.40.189Z(F2044413DAC2E02E3D6BCF4735A19BCA1DE97281)z2001:638:a000:4140::ffff:189r¨Z(ED03BB616EB2F60BEC80151114BB25CEF515B226Ú dannenbergz193.23.244.244)z2001:678:558:1000::244r¨Z(7BE683E65D48141321C5ED92F075C55364AC7123Z(0232AF901C31A04EE9848595AF9BB7620D4C5B2E)r%r r"r#r&r$rJÚmaatuskaz 171.25.193.9Z(BD6A829255CB08E66FBE7D3748363586E46B3810)z2001:67c:289c::9r©Z(49015F787433103580E3B66A1707A00E60F2D15BÚ Faravaharz154.35.175.225Z(CF6D0AAFB385BE71B8E111FC5CFF4B47923733BCZ(EFCBE720AB3A82B99F9E953CD5BF50F7EEFC7B97Úlongclawz 199.58.81.140Z(74A910646BCEEFBCD2E874FC1DC997430F968145Z(23D15D965BC35114467363C165C4F724B64B4F66Úbastetz204.13.164.118Z(24E2F139121D4394C54B5BCC368B3B411857C413)z2620:13:4000:6000::1000:118r¨Z(27102BC123E7AF1D4741AE047E160C91ADC76B21ÚSergez 66.111.2.131i)#iF#Z(BA44A889E64B93FAA2B114E02C2A279A8555C533) r¦r§rªr«r¬r­r®r¯r°r±r),rFÚosÚrer[r3Z stem.utilZstem.util.confrrrÚ collectionsrÚ ImportErrorZstem.util.ordereddictZurllib.requestZrequestrUZurllib2rWrŽrŠr ÚdirnameÚ__file__r…Úcompiler_r`rarbr˜rr’r“r”r•rrÚobjectrrHrtr¥rOrrrrÚsø% ÿ  ÿ           !nbO/úù úù ù ù úúù ú ²