o u]-b@sdZddlZddlZddlZddlZddlmZzddlm Z Wne y/ddl m Z YnwiZ Gddde ZdddZd d Zdd d ZddZdddZGddde ZdS)aJ Handlers for text configuration files. Configurations are simple string to string mappings, with the configuration files using the following rules... * the key/value is separated by a space * anything after a '#' is ignored as a comment * excess whitespace is trimmed * empty lines are ignored * multi-line values can be defined by following the key with lines starting with a '|' For instance... :: # This is my sample config user.name Galen user.password yabba1234 # here's an inline comment user.notes takes a fancy to pepperjack cheese blankEntry.example msg.greeting |Multi-line message exclaiming of the |wonder and awe that is pepperjack! ... would be loaded as... :: config = { 'user.name': 'Galen', 'user.password': 'yabba1234', 'user.notes': 'takes a fancy to pepperjack cheese', 'blankEntry.example': '', 'msg.greeting': 'Multi-line message exclaiming of the\nwonder and awe that is pepperjack!', } Configurations are managed via the :class:`~stem.util.conf.Config` class. The :class:`~stem.util.conf.Config` can be be used directly with its :func:`~stem.util.conf.Config.get` and :func:`~stem.util.conf.Config.set` methods, but usually modules will want a local dictionary with just the configurations that it cares about. To do this use the :func:`~stem.util.conf.config_dict` function. For example... :: import getpass from stem.util import conf, connection def config_validator(key, value): if key == 'timeout': # require at least a one second timeout return max(1, value) elif key == 'endpoint': if not connection.is_valid_ipv4_address(value): raise ValueError("'%s' isn't a valid IPv4 address" % value) elif key == 'port': if not connection.is_valid_port(value): raise ValueError("'%s' isn't a valid port" % value) elif key == 'retries': # negative retries really don't make sense return max(0, value) CONFIG = conf.config_dict('ssh_login', { 'username': getpass.getuser(), 'password': '', 'timeout': 10, 'endpoint': '263.12.8.0', 'port': 22, 'reconnect': False, 'retries': 3, }, config_validator) There's several things going on here so lets take it step by step... * The :func:`~stem.util.conf.config_dict` provides a dictionary that's bound to a given configuration. If the "ssh_proxy_config" configuration changes then so will the contents of CONFIG. * The dictionary we're passing to :func:`~stem.util.conf.config_dict` provides two important pieces of information: default values and their types. See the Config's :func:`~stem.util.conf.Config.get` method for how these type inferences work. * The config_validator is a hook we're adding to make sure CONFIG only gets values we think are valid. In this case it ensures that our timeout value is at least one second, and rejects endpoints or ports that are invalid. Now lets say our user has the following configuration file... :: username waddle_doo password jabberwocky timeout -15 port 9000000 retries lots reconnect true logging debug ... and we load it as follows... :: >>> from stem.util import conf >>> our_config = conf.get_config('ssh_login') >>> our_config.load('/home/atagar/user_config') >>> print CONFIG # doctest: +SKIP { "username": "waddle_doo", "password": "jabberwocky", "timeout": 1, "endpoint": "263.12.8.0", "port": 22, "reconnect": True, "retries": 3, } Here's an expanation of what happened... * the username, password, and reconnect attributes took the values in the configuration file * the 'config_validator' we added earlier allows for a minimum timeout of one and rejected the invalid port (with a log message) * we weren't able to convert the retries' "lots" value to an integer so it kept its default value and logged a warning * the user didn't supply an endpoint so that remained unchanged * our CONFIG didn't have a 'logging' attribute so it was ignored **Module Overview:** :: config_dict - provides a dictionary that's kept in sync with our config get_config - singleton for getting configurations uses_settings - provides an annotation for functions that use configurations parse_enum_csv - helper funcion for parsing confguration entries for enums Config - Custom configuration |- load - reads a configuration file |- save - writes the current configuration to a file |- clear - empties our loaded configuration contents |- add_listener - notifies the given listener when an update occurs |- clear_listeners - removes any attached listeners |- keys - provides keys in the loaded configuration |- set - sets the given key/value pair |- unused_keys - provides keys that have never been requested |- get - provides the value for a given key, with type inference +- get_value - provides the value for a given key as a string N)log) OrderedDictc@seZdZddZddZdS) _SyncListenercCs||_||_dSN) config_dict interceptor)selfrrr 0/usr/lib/python3/dist-packages/stem/util/conf.py__init__s z_SyncListener.__init__cCsZ||jvr+|||j|}||j|krdS|jr$|||}|r$|}||j|<dSdSr)rgetr)rconfigkeyZ new_valueZinterceptor_valuer r r updates  z_SyncListener.updateN)__name__ __module__ __qualname__r rr r r r rs rcCst|}|t||j|S)aW Makes a dictionary that stays synchronized with a configuration. This takes a dictionary of 'config_key => default_value' mappings and changes the values to reflect our current configuration. This will leave the previous values alone if... * we don't have a value for that config_key * we can't convert our value to be the same type as the default_value If a handler is provided then this is called just prior to assigning new values to the config_dict. The handler function is expected to accept the (key, value) for the new values and return what we should actually insert into the dictionary. If this returns None then the value is updated as normal. For more information about how we convert types see our :func:`~stem.util.conf.Config.get` method. **The dictionary you get from this is manged by the Config class and should be treated as being read-only.** :param str handle: unique identifier for a config instance :param dict conf_mappings: config key/value mappings used as our defaults :param functor handler: function referred to prior to assigning values ) get_config add_listenerrr)handleZ conf_mappingsZhandlerZselected_configr r r rsrcCs|tvr tt|<t|S)z Singleton constructor for configuration file instances. If a configuration already exists for the handle then it's returned. Otherwise a fresh instance is constructed. :param str handle: unique identifier used to access this config instance )CONFSConfig)rr r r rs rTcs6t|sjsd_fdd}|S)a Provides a function that can be used as a decorator for other functions that require settings to be loaded. Functions with this decorator will be provided with the configuration as its 'config' keyword argument. .. versionchanged:: 1.3.0 Omits the 'config' argument if the funcion we're decorating doesn't accept it. :: uses_settings = stem.util.conf.uses_settings('my_app', '/path/to/settings.cfg') @uses_settings def my_function(config): print 'hello %s!' % config.get('username', '') :param str handle: hande for the configuration :param str path: path where the configuration should be loaded from :param bool lazy_load: loads the configuration file when the decorator is used if true, otherwise it's loaded right away :returns: **function** that can be used as a decorator to provide the configuration :raises: **IOError** if we fail to read the configuration file, if **lazy_load** is true then this arises when we use the decorator Tcsfdd}|S)NcsJr js d_dtjvr|di|S|i|S)NTr )_settings_loadedloadinspectZ getargspecargs)rkwargs)r func lazy_loadpathr r wrappeds  z1uses_settings..decorator..wrappedr )rr r rr)rr decorators z uses_settings..decorator)rrr)rrrr"r r!r uses_settingss    r#cCst|||ddS)a Provides the enumeration value for a given key. This is a case insensitive lookup and raises an exception if the enum key doesn't exist. :param str key: configuration key being looked up :param str value: value to be parsed :param stem.util.enum.Enum enumeration: enumeration the values should be in :returns: enumeration value :raises: **ValueError** if the **value** isn't among the enumeration keys r)parse_enum_csv)rvalue enumerationr r r parse_enum*sr(c Cs:dd|dD}|dgkrgS|durnUt|tr+t||kr*td|||fn@t|trat|dkra|\}}|durMt||krMtd|||f|dur`t||kr`td |||fn td |t|fg}d dt|D}t|} |D]} | |vr| | | | q~td | |d |f|S)a Parses a given value as being a comma separated listing of enumeration keys, returning the corresponding enumeration values. This is intended to be a helper for config handlers. The checks this does are case insensitive. The **count** attribute can be used to make assertions based on the number of values. This can be... * None to indicate that there's no restrictions. * An int to indicate that we should have this many values. * An (int, int) tuple to indicate the range that values can be in. This range is inclusive and either can be None to indicate the lack of a lower or upper bound. :param str key: configuration key being looked up :param str value: value to be parsed :param stem.util.enum.Enum enumeration: enumeration the values should be in :param int,tuple count: validates that we have this many items :returns: list with the enumeration values :raises: **ValueError** if the count assertion fails or the **value** entries don't match the enumeration keys cSsg|]}|qSr )upperstrip).0valr r r Usz"parse_enum_csv..,NzGConfig entry '%s' is expected to be %i comma separated values, got '%s'zHConfig entry '%s' must have at least %i comma separated values, got '%s'zFConfig entry '%s' can have at most %i comma separated values, got '%s'zBThe count must be None, an int, or two value tuple. Got '%s' (%s)'cSsg|]}|qSr )r))r+kr r r r-kszKThe '%s' entry of config entry '%s' wasn't in the enumeration (expected %s)z, ) split isinstanceintlen ValueErrortupletypelistkeysappendindexjoin) rr&r'countvaluesZminimumZmaximumresultZ enum_keysZ enum_valuesr,r r r r%;s2   r%c@s|eZdZdZddZdddZddd Zd d Zdd d ZddZ ddZ ddZ dddZ dddZ d ddZddZdS)!raQ Handler for easily working with custom configurations, providing persistence to and from files. All operations are thread safe. **Example usage:** User has a file at '/home/atagar/myConfig' with... :: destination.ip 1.2.3.4 destination.port blarg startup.run export PATH=$PATH:~/bin startup.run alias l=ls And they have a script with... :: from stem.util import conf # Configuration values we'll use in this file. These are mappings of # configuration keys to the default values we'll use if the user doesn't # have something different in their config file (or it doesn't match this # type). ssh_config = conf.config_dict('ssh_login', { 'login.user': 'atagar', 'login.password': 'pepperjack_is_awesome!', 'destination.ip': '127.0.0.1', 'destination.port': 22, 'startup.run': [], }) # Makes an empty config instance with the handle of 'ssh_login'. This is # a singleton so other classes can fetch this same configuration from # this handle. user_config = conf.get_config('ssh_login') # Loads the user's configuration file, warning if this fails. try: user_config.load("/home/atagar/myConfig") except IOError as exc: print "Unable to load the user's config: %s" % exc # This replace the contents of ssh_config with the values from the user's # config file if... # # * the key is present in the config file # * we're able to convert the configuration file's value to the same type # as what's in the mapping (see the Config.get() method for how these # type inferences work) # # For instance in this case... # # * the login values are left alone because they aren't in the user's # config file # # * the 'destination.port' is also left with the value of 22 because we # can't turn "blarg" into an integer # # The other values are replaced, so ssh_config now becomes... # # {'login.user': 'atagar', # 'login.password': 'pepperjack_is_awesome!', # 'destination.ip': '1.2.3.4', # 'destination.port': 22, # 'startup.run': ['export PATH=$PATH:~/bin', 'alias l=ls']} # # Information for what values fail to load and why are reported to # 'stem.util.log'. .. versionchanged:: 1.7.0 Class can now be used as a dictionary. cCs0d|_t|_g|_t|_t|_d|_ dSNF) _pathr _contents _listeners threadingRLock_contents_lockset_requested_keysrrr r r r s   zConfig.__init__NTcCs|r||_n|js tdtj|jr7t|jD]\}}}|D]}|dr)q!|tj||q!qdSt |jd }| }Wdn1sLwY|j |r| d} |rc| dnd} | dkro| d| } | } | rd| vr| dd \} } || | d nAg} |r|dd r| dd d}|d }| ||r|dd s| r|| d | d n|| d d |sWWddSWddS1swYdS)a' Reads in the contents of the given path, adding its configuration values to our current contents. If the path is a directory then this loads each of the files, recursively. .. versionchanged:: 1.3.0 Added support for directories. .. versionchanged:: 1.3.0 Added the **commenting** argument. .. versionchanged:: 1.6.0 Avoid loading vim swap files. :param str path: file or directory path to be loaded, this uses the last loaded path if not provided :param bool commenting: ignore line content after a '#' if **True**, read otherwise :raises: * **IOError** if we fail to read the file (it doesn't exist, insufficient permissions, etc) * **ValueError** if no path was provided and we've never been provided one z.Unable to load configuration: no path providedz.swpNrr# r$F| r/)rBr6osrisdirwalkendswithrr=open readlinesrGpopfindr*r2rHlstrip startswithrstripr;)rrZ commentingrootdirnames filenamesfilenameZ config_fileZ read_contentslineZ comment_startrr&Zmultiline_bufferZcontentr r r rsN      "z Config.loadc Cs|r||_n|js td|jbtjtj|js&ttj|jt|jd.}| D]!}|j |ddD]}d|vrHd| dd}| d||fq:q1Wdn1s]wYWddSWddS1suwYdS) aa Saves configuration contents to disk. If a path is provided then it replaces the configuration location that we track. :param str path: location to be saved to :raises: * **IOError** if we fail to save the file (insufficient permissions, etc) * **ValueError** if no path was provided and we've never been provided one z.Unable to save configuration: no path providedwT)multiplerPz |z%s %s N) rBr6rGrQrexistsdirnamemakedirsrUr: get_valuereplacewrite)rrZ output_file entry_keyZ entry_valuer r r save s&  "z Config.savecCs>|j|jt|_WddS1swYdS)z[ Drops the configuration contents and reverts back to a blank, unloaded state. N)rGrCclearrHrIrJr r r rk>s  "z Config.clearcCsd|j%|j||r|D]}|||qWddSWddS1s+wYdS)a8 Registers the function to be notified of configuration updates. Listeners are expected to be functors which accept (config, key). :param functor listener: function to be notified when our configuration is changed :param bool backfill: calls the function with our current values if **True** N)rGrDr;r:)rlistenerZbackfillrr r r rHs   "zConfig.add_listenercCs g|_dS)z) Removes all attached listeners. N)rDrJr r r clear_listenersXs zConfig.clear_listenerscCst|jS)z Provides all keys in the currently loaded configuration. :returns: **list** if strings for the configuration keys we've loaded )r9rCr:rJr r r r:_sz Config.keyscCst||jS)a? Provides the configuration keys that have never been provided to a caller via :func:`~stem.util.conf.config_dict` or the :func:`~stem.util.conf.Config.get` and :func:`~stem.util.conf.Config.get_value` methods. :returns: **set** of configuration keys we've loaded but have never been requested )rHr: differencerIrJr r r unused_keyshs zConfig.unused_keyscCsF|jtjr tnt}|dur|r||jvr|j|=nd nZt|t|frH|s6||jvr6|j| |n|g|j|<|j D]}|||q?nAt|t t frq|s`||jvr`|j||7<n||j|<|j D]}|||qhn t dt|WddSWddSWddSWddS1swYdS)a Appends the given key/value configuration mapping, behaving the same as if we'd loaded this from a configuration file. .. versionchanged:: 1.5.0 Allow removal of values by overwriting with a **None** value. :param str key: key for the configuration mapping :param str,list value: value we're setting the mapping to :param bool overwrite: replaces the previous value if **True**, otherwise the values are appended Nz[Config.set() only accepts str (bytes or unicode), list, or tuple. Provided value was a '%s')rGstemZprereqZ is_python_3strZunicoderCr3bytesr;rDr9r7r6r8)rrr&Z overwriteZ unicode_typerlr r r rHts:       "z Config.setc Cst|tttf}||||}||kr|St|tr=|dkr$d}|S|dkr.d}|Std|t |f|}|St|t r_zt |}W|St y^td||f|}Y|Swt|t rzt |}W|St ytd||f|}Y|Swt|trt|}|St|trt|}|St|trt }|D] }d|vr|dd \}}|||<qtd ||fq|}|S) al Fetches the given configuration, using the key and default value to determine the type it should be. Recognized inferences are: * **default is a boolean => boolean** * values are case insensitive * provides the default if the value isn't "true" or "false" * **default is an integer => int** * provides the default if the value can't be converted to an int * **default is a float => float** * provides the default if the value can't be converted to a float * **default is a list => list** * string contents for all configuration values with this key * **default is a tuple => tuple** * string contents for all configuration values with this key * **default is a dictionary => dict** * values without "=>" in them are ignored * values are split into key/value pairs on "=>" with extra whitespace stripped :param str key: config setting to be fetched :param default object: value provided if no such key exists or fails to be converted :returns: given configuration value with its type inferred with the above rules trueTZfalseFzAConfig entry '%s' is expected to be a boolean, defaulting to '%s'zBConfig entry '%s' is expected to be an integer, defaulting to '%i'z?Config entry '%s' is expected to be a float, defaulting to '%f'z=>r$zPIgnoring invalid %s config entry (expected a mapping, but "%s" was missing "=>"))r3r9r7dictrfboollowerrdebugrqr4r6floatrr2r*) rrdefaultZ is_multivaluer,Zval_mapentryriZ entry_valr r r r s^&            z Config.getFcCs|jA||jvr+|j||r|j|WdS|j|dWdSd|}t|tjd||f|WdS1sGwYdS)a This provides the current value associated with a given key. :param str key: config setting to be fetched :param object default: value provided if no such key exists :param bool multiple: provides back a list of all values if **True**, otherwise this returns the last loaded configuration value :returns: **str** or **list** of string configuration values associated with the given key, providing the default if no such key exists NrMz$stem.util.conf.missing_config_key_%sz/config entry '%s' not found, defaulting to '%s')rGrCrIaddrZlog_onceZTRACE)rrryrbZ message_idr r r rfs    $zConfig.get_valuecCs4|j |j|WdS1swYdSr)rGrC)rrr r r __getitem__s$zConfig.__getitem__)NTrTrA)rrr__doc__r rrjrkrrmr:rorHr rfr|r r r r rwsO  K  ) P rrr})r~rrQrEZ stem.prereqrpZ stem.utilr collectionsr ImportErrorZstem.util.ordereddictrobjectrrrr#r(r%rr r r r s(   ! 4 <