obUkdZddlmZddlZddlmZmZmZddlm Z ddl m Z ddl m Z mZmZej r-dd lmZmZmZmZmZmZmZmZmZmZdd lmZdd lmZeeege fZ!ejDd d Z#eddZ$ Gdde%Z&GddejNe#Z(e&Z)e)jTZ*e)jVZ,e)j:Z-e)j\Z/y)zMachinery for walking a filesystem. *Walking* a filesystem means recursively visiting a directory and any sub-directories. It is a fairly common requirement for copying, searching etc. See :ref:`walking` for details. )unicode_literalsN) defaultdictdeque namedtuple) make_repr)FSError)abspathcombinenormpath) AnyCallable CollectionIteratorListMutableMappingOptionalTextTupleType)FS)Info_Fr)boundStepzpath, dirs, filesceZdZdZ dfd ZedZedZedZedZ dZ ddZ d Z d Z d Zd Zd Z ddZ ddZddZddZ ddZ ddZ ddZxZS)Walkerz>A walker object recursively lists directories in a filesystem.c 6|dvr td||_|r |r%td|r |jn |j}t |s t d||_||_||_||_ ||_ ||_ ||_ tt|?y)a-Create a new `Walker` instance. Arguments: ignore_errors (bool): If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable, optional): If ``ignore_errors`` is `False`, then this callable will be invoked for a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str): If ``"breadth"`` then the directory will be walked *top down*. Set to ``"depth"`` to walk *bottom up*. filter (list, optional): If supplied, this parameter should be a list of filename patterns, e.g. ``["*.py"]``. Files will only be returned if the final component matches one of the patterns. exclude (list, optional): If supplied, this parameter should be a list of filename patterns, e.g. ``["~*"]``. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional): A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list, optional): A list of patterns that will be used to filter out directories from the walk. e.g. ``['*.svn', '*.git']``. max_depth (int, optional): Maximum directory depth to walk. )breadthdepthz#search must be 'breadth' or 'depth'z,on_error is invalid when ignore_errors==Truezon_error must be callableN) ValueError ignore_errors_ignore_errors _raise_errorscallable TypeErroron_errorsearchfilterexclude filter_dirs exclude_dirs max_depthsuperr__init__) selfr"r'r(r)r*r+r,r- __class__s )/usr/lib/python3/dist-packages/fs/walk.pyr/zWalker.__init__5sP - -BC C*  !OPP.;t**ASASH!78 8     &(" fd$&cy)z#Ignore dir scan errors when called.Tclspatherrors r2r#zWalker._ignore_errorsqsr3cy)z%Re-raise dir scan errors when called.Fr5r6s r2r$zWalker._raise_errorswsr3cT|jd}|r|jddzSdS)zBCalculate the 'depth' of a directory path (i.e. count components)./rr)stripcount)r7r8_paths r2_calculate_depthzWalker._calculate_depth}s, 3',u{{3!#3!3r3ct|S)aBind a `Walker` instance to a given filesystem. This *binds* in instance of the Walker to a given filesystem, so that you won't need to explicitly provide the filesystem as a parameter. Arguments: fs (FS): A filesystem object. Returns: ~fs.walk.BoundWalker: a bound walker. Examples: Use this method to explicitly bind a filesystem instance:: >>> walker = Walker.bind(my_fs) >>> for path in walker.files(filter=['*.py']): ... print(path) /foo.py /bar.py Unless you have written a customized walker class, you will be unlikely to need to call this explicitly, as filesystem objects already have a ``walk`` attribute which is a bound walker object:: >>> for path in my_fs.walk.files(filter=['*.py']): ... print(path) /foo.py /bar.py ) BoundWalker)r7fss r2bindz Walker.bindsF2r3c t|jj|jdf|jdf|j df|j df|jdf|jdf|jdf|jdf S)NFr)r"r'r(r)r*r+r,r-) rr1__name__r"r'r(r)r*r+r,r-r0s r2__repr__zWalker.__repr__s NN # #--u5mmT*KK+KK&\\4())40++T2~~t,  r3cp|jdk(r|j|||S|j|||S)zGet the walk generator.r namespaces)r( _walk_breadth _walk_depth)r0rCr8rKs r2 _iter_walkzWalker._iter_walks@ ;;) #%%b$:%F F##B#D Dr3c|j'|j|j|jry|j'|j|j|jsy|j |||S)z6Check if a directory should be considered in the walk.F)r,matchnamer+check_open_dirr0rCr8infos r2_check_open_dirzWalker._check_open_dirse    (RXXd6G6G-S    '9I9I4990U""2tT22r3cy)adCheck if a directory should be opened. Override to exclude directories from the walk. Arguments: fs (FS): A filesystem instance. path (str): Path to directory. info (Info): A resource info object for the directory. Returns: bool: `True` if the directory should be opened. Tr5rSs r2rRzWalker.check_open_dirsr3c`|j||jk\ry|j|||S)z0Check if a directory contents should be scanned.F)r-check_scan_dir)r0rCr8rTr s r2_check_scan_dirzWalker._check_scan_dirs1 >> %%4>>*A""2tT22r3cy)aCheck if a directory should be scanned. Override to omit scanning of certain directories. If a directory is omitted, it will appear in the walk but its files and sub-directories will not. Arguments: fs (FS): A filesystem instance. path (str): Path to directory. info (Info): A resource info object for the directory. Returns: bool: `True` if the directory should be scanned. Tr5rSs r2rXzWalker.check_scan_dirs"r3c|j'|j|j|jry|j|j|jS)aCheck if a filename should be included. Override to exclude files from the walk. Arguments: fs (FS): A filesystem instance. info (Info): A resource info object. Returns: bool: `True` if the file should be included. F)r*rPrQr))r0rCrTs r2 check_filezWalker.check_files? << #tyy(Ixx TYY//r3c#K |j||D]}|y#t$r}|j||sYd}~yd}~wwxYww)aGet an iterator of `Info` objects for a directory path. Arguments: fs (FS): A filesystem instance. dir_path (str): A path to a directory on the filesystem. namespaces (list): A list of additional namespaces to include in the `Info` objects. Returns: ~collections.Iterator: iterator of `Info` objects for resources within the given path. rJN)scandirr r')r0rCdir_pathrKrTr9s r2_scanz Walker._scan sQ(  8 C    ==512 s)A !A  AAA AA c#:Ktt|}tt}|j |||}|D][\}}|@g} g} ||D]!} | j r| n| j | #t|| | ||=H||j |]yw)abWalk the directory structure of a filesystem. Arguments: fs (FS): A filesystem instance. path (str): A path to a directory on the filesystem. namespaces (list, optional): A list of additional namespaces to add to the `Info` objects. Returns: collections.Iterator: an iterator of `~fs.walk.Step` instances. The return value is an iterator of ``(, , )`` named tuples, where ```` is an absolute path to a directory, and ```` and ```` are a list of `~fs.info.Info` objects for directories and files in ````. Example: >>> walker = Walker(filter=['*.py']) >>> for path, dirs, files in walker.walk(my_fs, namespaces=["details"]): ... print("[{}]".format(path)) ... print("{} directories".format(len(dirs))) ... total = sum(info.size for info in files) ... print("{} bytes".format(total)) [/] 2 directories 55 bytes ... rJN)r r rlistrNis_dirappendr) r0rCr8rKr?dir_info_walkr_rTdirsfiles_infos r2walkz Walker.walk%sH't$EjA# 0NHd|%h/DE"\\Tu<, )``. Arguments: fs (FS): A filesystem instance. path (str): A path to a directory on the filesystem. namespaces (list, optional): A list of additional namespaces to add to the `Info` objects. Yields: (str, Info): a tuple of ``(, )``. r8rKN)r rNrQ)r0rCr8rKrorfr?rTs r2rTz Walker.info{sP&*E  7KE4udii0$66 7s %AAc#Kt|g}|j}|j}t}|j}|j } |j } |j} |j} | |} |r|}||||D]c}|jrE| || z dz}| |||s(||f| ||||s:||||jT| ||s^||fe|df|ryyw)z*Walk files using a *breadth first* search.rJrN) r appendleftpopr r`r@rUrYr\rcrQ)r0rCr8rKqueuepushrvror`r@rUrY _check_filer r_rT_depths r2rLzWalker._walk_breadthstf ii 00....oo  &uHb(zB -;;-h7%?!CF&r8T:&n,*2xvF (DII!>?"2t,&n, -D. sB*C6-C6?#C6#C64C6c#Kt}|j}|j}|j}|j}|j } ||} |||||dfg} | j } | r| d\} }}t|d}|||| df| d=no|jrT|| | z dz}||| |rK||| ||r*|| |j}| |||||| |ffn| |fn| ||r| |f| ryyw)z(Walk files using a *depth first* search.rJNr) r r`r@rUrYr\rdnextrcrQ)r0rCr8rKror`r@rUrYryr stackrxr_ iter_filesparentrTrzr?s r2rMzWalker._walk_depths@ 00....oo  &5Tj94 @ ||+09 (Hj& D)D|% Ln$"I)(3e;a?"2x6&r8T6B (499 = % %b%J G!)4 0'n,r4("D.(1s DDD)FNrNNNNN)Nr<Nr<)rF __module__ __qualname____doc__r/ classmethodr#r$r@rDrHrNrUrRrYrXr\r`rjrhrgrTrLrM __classcell__)r1s@r2rr2sH:'x  44 ""H  $ E3"3&0, < 00d1$1* 7: !!N 0)r3rcReZdZdZefdZdZdZ d dZeZ d dZ d dZ d d Z y) rBaA class that binds a `Walker` instance to a `FS` instance. You will typically not need to create instances of this class explicitly. Filesystems have a `~FS.walk` property which returns a `BoundWalker` object. Example: >>> tmp_fs = fs.tempfs.TempFS() >>> tmp_fs.walk BoundWalker(TempFS()) A `BoundWalker` is callable. Calling it is an alias for the `~fs.walk.BoundWalker.walk` method. c ||_||_y)zCreate a new walker bound to the given filesystem. Arguments: fs (FS): A filesystem instance. walker_class (type): A `~fs.walk.WalkerBase` sub-class. The default uses `~fs.walk.Walker`. N)rC walker_class)r0rCrs r2r/zBoundWalker.__init__s(r3c8dj|jS)NzBoundWalker({!r}))formatrCrGs r2rHzBoundWalker.__repr__s"))$''22r3c*|j|i|}|S)zCreate a walker instance.)r)r0argskwargswalkers r2 _make_walkerzBoundWalker._make_walker s #""D3F3 r3Nc b|jdi|}|j|j||S)aY Walk the directory structure of a filesystem. Arguments: path (str): namespaces (list, optional): A list of namespaces to include in the resource information, e.g. ``['basic', 'access']`` (defaults to ``['basic']``). Keyword Arguments: ignore_errors (bool): If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable): If ``ignore_errors`` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str): If ``'breadth'`` then the directory will be walked *top down*. Set to ``'depth'`` to walk *bottom up*. filter (list): If supplied, this parameter should be a list of file name patterns, e.g. ``['*.py']``. Files will only be returned if the final component matches one of the patterns. exclude (list, optional): If supplied, this parameter should be a list of filename patterns, e.g. ``['~*', '.*']``. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional): A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list): A list of patterns that will be used to filter out directories from the walk, e.g. ``['*.svn', '*.git']``. max_depth (int, optional): Maximum directory depth to walk. Returns: ~collections.Iterator: an iterator of ``(, , )`` named tuples, where ```` is an absolute path to a directory, and ```` and ```` are a list of `~fs.info.Info` objects for directories and files in ````. Example: >>> walker = Walker(filter=['*.py']) >>> for path, dirs, files in walker.walk(my_fs, namespaces=['details']): ... print("[{}]".format(path)) ... print("{} directories".format(len(dirs))) ... total = sum(info.size for info in files) ... print("{} bytes".format(total)) [/] 2 directories 55 bytes ... This method invokes `Walker.walk` with bound `FS` object. rsr5)rrjrCr0r8rKrrs r2rjzBoundWalker.walks3z#"",V,{{477*{EEr3c `|jdi|}|j|j|S)aAWalk a filesystem, yielding absolute paths to files. Arguments: path (str): A path to a directory. Keyword Arguments: ignore_errors (bool): If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable): If ``ignore_errors`` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str): If ``'breadth'`` then the directory will be walked *top down*. Set to ``'depth'`` to walk *bottom up*. filter (list): If supplied, this parameter should be a list of file name patterns, e.g. ``['*.py']``. Files will only be returned if the final component matches one of the patterns. exclude (list, optional): If supplied, this parameter should be a list of filename patterns, e.g. ``['~*', '.*']``. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional): A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list): A list of patterns that will be used to filter out directories from the walk, e.g. ``['*.svn', '*.git']``. max_depth (int, optional): Maximum directory depth to walk. Returns: ~collections.Iterator: An iterator over file paths (absolute from the filesystem root). This method invokes `Walker.files` with the bound `FS` object. rlr5)rrhrCr0r8rrs r2rhzBoundWalker.filesTs1N#"",V,||DGG$|//r3c `|jdi|}|j|j|S)auWalk a filesystem, yielding absolute paths to directories. Arguments: path (str): A path to a directory. Keyword Arguments: ignore_errors (bool): If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable): If ``ignore_errors`` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str): If ``'breadth'`` then the directory will be walked *top down*. Set to ``'depth'`` to walk *bottom up*. filter_dirs (list, optional): A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list): A list of patterns that will be used to filter out directories from the walk, e.g. ``['*.svn', '*.git']``. max_depth (int, optional): Maximum directory depth to walk. Returns: ~collections.Iterator: an iterator over directory paths (absolute from the filesystem root). This method invokes `Walker.dirs` with the bound `FS` object. rlr5)rrgrCrs r2rgzBoundWalker.dirs~s1@#"",V,{{477{..r3c b|jdi|}|j|j||S)a Walk a filesystem, yielding path and `Info` of resources. Arguments: path (str): A path to a directory. namespaces (list, optional): A list of namespaces to include in the resource information, e.g. ``['basic', 'access']`` (defaults to ``['basic']``). Keyword Arguments: ignore_errors (bool): If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable): If ``ignore_errors`` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str): If ``'breadth'`` then the directory will be walked *top down*. Set to ``'depth'`` to walk *bottom up*. filter (list): If supplied, this parameter should be a list of file name patterns, e.g. ``['*.py']``. Files will only be returned if the final component matches one of the patterns. exclude (list, optional): If supplied, this parameter should be a list of filename patterns, e.g. ``['~*', '.*']``. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional): A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list): A list of patterns that will be used to filter out directories from the walk, e.g. ``['*.svn', '*.git']``. max_depth (int, optional): Maximum directory depth to walk. Returns: ~collections.Iterable: an iterable yielding tuples of ``(, )``. This method invokes `Walker.info` with the bound `FS` object. rsr5)rrTrCrs r2rTzBoundWalker.infos3^#"",V,{{477*{EEr3rr) rFrrrrr/rHrrj__call__rhrgrTr5r3r2rBrBsK )/ )3>F@H(0T!/J0Fr3rB)0r __future__rtyping collectionsrrr_reprrerrorsr r8r r r TYPE_CHECKINGr rrrrrrrrrbaserrTr ExceptionboolOnErrorTypeVarrrobjectrGenericrBdefault_walkerrjrh walk_files walk_inforg walk_dirsr5r3r2rs( 66,,    i($./GV^^D%&-.u)Vu)p gF&..$gF\  ! !       r3