o u]- @sdZddlZddlZddlZddlZddlmZmZzddl m Z Wne y1ddl m Z YnwdZ dZGdddeZGd d d eZd d Zd dZddZddZiddefddefddefddefddefddefddefddefddefd d!efd"d#efd$d%efd&d&efd'd'efd(d(efd)d*efd+d,efd-efd.efd/efd0efd1efd2efd3efd4Zdd5iZdAd7d8Zd9d:Zd;d<Zd=d>ZGd?d@d@eZdS)Ba  Parsing for Bandwidth Authority metrics as described in Tor's `bandwidth-file-spec `_. **Module Overview:** :: BandwidthFile - Tor bandwidth authority measurements. .. versionadded:: 1.8.0 N) _mappings_for Descriptor) OrderedDicts=====s====c@eZdZdZddZdS) RecentStatsaE Statistical information collected over the last 'data_period' (by default five days). :var int consensus_count: number of consensuses published during this period :var int prioritized_relays: number of relays prioritized to be measured :var int prioritized_relay_lists: number of times a set of relays were prioritized to be measured :var int measurement_attempts: number of relay measurements we attempted :var int measurement_failures: number of measurement attempts that failed :var RelayFailures relay_failures: number of relays we failed to measure cCs*d|_d|_d|_d|_d|_t|_dSN)Zconsensus_countZprioritized_relaysZprioritized_relay_listsZmeasurement_attemptsZmeasurement_failures RelayFailuresZrelay_failuresselfr @/usr/lib/python3/dist-packages/stem/descriptor/bandwidth_file.py__init__:s  zRecentStats.__init__N__name__ __module__ __qualname____doc__r r r r r r)s rc@r)ra Summary of the number of relays we were unable to measure. :var int no_measurement: number of relays that did not have any successful measurements :var int insuffient_period: number of relays whos measurements were collected over a period that was too small (1 day by default) :var int insufficient_measurements: number of relays we did not collect enough measurements for (2 by default) :var int stale: number of relays whos latest measurement is too old (5 days by default) cCsd|_d|_d|_d|_dSr)Zno_measurementZinsuffient_periodZinsufficient_measurementsZstaler r r r r Qs zRelayFailures.__init__Nrr r r r rCs rcCs|Srr valr r r _str[srcCs|r |r t|SdSr)isdigitintrr r r _int_srcCs&ztjj|WStyYdSwr)stemutil str_toolsZ_parse_iso_timestamp ValueErrorrr r r _datecs  rcCs$|durttdd|dSdS)NcSs|Sr)strip)vr r r ksz_csv..,)listmapsplitrr r r _csvjs$r%versionZsoftwareZsoftware_versionZearliest_bandwidthZlatest_bandwidthZ created_atZ file_createdZ generated_atZgenerator_startedZconsensus_sizeZnumber_consensus_relaysZeligible_countZnumber_eligible_relaysZeligible_percentZpercent_eligible_relaysZ min_countZminimum_number_eligible_relaysZ min_percentZminimum_percent_eligible_relaysZscanner_countryZdestinations_countriesZtime_to_report_half_networkzrecent_stats.consensus_countZrecent_consensus_countz$recent_stats.prioritized_relay_listsZrecent_priority_list_countZrecent_priority_relay_countZ recent_measurement_attempt_countZ recent_measurement_failure_countZ(recent_measurements_excluded_error_countZ'recent_measurements_excluded_near_countZ&recent_measurements_excluded_few_countZ&recent_measurements_excluded_old_count)zrecent_stats.prioritized_relaysz!recent_stats.measurement_attemptsz!recent_stats.measurement_failuresz*recent_stats.relay_failures.no_measurementz-recent_stats.relay_failures.insuffient_periodz5recent_stats.relay_failures.insufficient_measurementsz!recent_stats.relay_failures.stale1.0.0Fckst||fi|VdS)a Iterates over the bandwidth authority metrics in a file. :param file descriptor_file: file with descriptor content :param bool validate: checks the validity of the descriptor's content if **True**, skips these checks otherwise :param dict kwargs: additional arguments for the descriptor constructor :returns: :class:`stem.descriptor.bandwidth_file.BandwidthFile` object :raises: * **ValueError** if the contents is malformed and validate is **True** * **IOError** if the file can't be read N) BandwidthFileread)Zdescriptor_filevalidatekwargsr r r _parse_filesr,c Cs4t}t|}|d}d} |}|sn5|ttfvr#n.|s*d|vr*n'd|vrFtj j | dd\}}|||<|dkrE|}nt d||d7}q||_t|_tD]-\} \} } |} | ddd D]} t| | } qmt| | dd | || t| q\|dur|dkrt d dSdS) NTsnode_id===r&z3Header expected to be key=value pairs, but had '%s'.z3The 'version' header must be in the second position)rioBytesIO get_bytesreadliner HEADER_DIVHEADER_DIV_ALTrrr _to_unicoder$rheaderrZ recent_stats HEADER_ATTRitemsgetattrsetattrgetHEADER_DEFAULT) descriptorentriesr9contentindexZ version_indexlinekeyvalueZ full_attrkeywordclsobjattrr r r _parse_headers@     *rKcCs@t|}|rtjt||_ dSt d|)Nz3First line should be a unix timestamp, but was '%s') r2r3r4r5rrdatetimeZutcfromtimestampr timestampr)r@rAZ first_liner r r _parse_timestamps rNcCst|}|jdkr|n|dttfvr( |dttfvsi}|D]1}t j j |}t td|}|ddd}|sQtd|||vr[td||||<q.||_dS)Nr' measurementZnode_id$z+Every meaurement must include 'node_id': %szBRelay %s is listed multiple times. It should only be present once.)r2r3r4r&r5rr6r7 readlinesrrrr8dictrr>lstripr measurements)r@rArBrUrDrJZ fingerprintr r r _parse_bodys"       rVcsheZdZdZdZdefiefiefdZe e dde De d d d Zdfd d ZZS)r(aX Tor bandwidth authority measurements. :var dict measurements: **\*** mapping of relay fingerprints to their bandwidth measurement metadata :var dict header: **\*** header metadata :var datetime timestamp: **\*** time when these metrics were published :var str version: **\*** document format version :var str software: application that generated these metrics :var str software_version: version of the application that generated these metrics :var datetime earliest_bandwidth: time of the first sampling :var datetime latest_bandwidth: time of the last sampling :var datetime created_at: time when this file was created :var datetime generated_at: time when collection of these metrics started :var int consensus_size: number of relays in the consensus :var int eligible_count: relays with enough measurements to be included :var int eligible_percent: percentage of consensus with enough measurements :var int min_count: minimum eligible relays for results to be provided :var int min_percent: minimum measured percentage of the consensus :var str scanner_country: country code where this scan took place :var list destinations_countries: all country codes that were scanned :var int time_to_report_half_network: estimated number of seconds required to measure half the network, given recent measurements :var RecentStats recent_stats: statistical information collected over the last 'data_period' (by default five days) **\*** attribute is either required when we're parsed with validation or has a default value, others are left as **None** if undefined zbandwidth-fileN)rMr9rUcCsg|]}|dtffqSr)rK).0kr r r 2szBandwidthFile.r Fc Cs |r td|j|durt|nt}|dttt}|dg}|dtd}g}d|vr?| t j j ||dkrI|rItd|dkr|d|vr`| t j j d|d|D]\} } | t j j d | | fqd| t|D] } | t j j | q~d |S) a Creates descriptor content with the given attributes. This descriptor type differs somewhat from others and treats our attr/exclude attributes as follows... * 'timestamp' is a reserved key for our mandatory header unix timestamp. * 'content' is a reserved key for our bandwidth measurement lines. * All other keys are treated as header fields. For example... :: BandwidthFile.content({ 'timestamp': '12345', 'version': '1.2.0', 'content': [], }) zSigning of %s not implementedNrMrBr&r'z2Headers require BandwidthFile version 1.1 or laterz version=%sz%s=%s )NotImplementedErrorrrpopstrrtimer>r?appendrrrZ _to_bytesrr;r6join) rHrJZexcludesignr9rMrBr&linesrXrrPr r r rB4s(    zBandwidthFile.contentcs@tt|j|| d|rt|dt|dt|ddSdS)N)Z lazy_load)superr(r rNrKrV)r Z raw_contentr* __class__r r r ks   zBandwidthFile.__init__)Nr FF)rrrrZTYPE_ANNOTATION_NAMErNrKrVZ ATTRIBUTESupdaterSr:keys classmethodrBr __classcell__r r rdr r(s% 6r(rf)rrLr2r^Zstem.util.str_toolsrZstem.descriptorrr collectionsr ImportErrorZstem.util.ordereddictr6r7objectrrrrrr%r:r?r,rKrNrVr(r r r r s      * -