o u]ʧ @sdZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZddlZddlZddlmZddlmZejjddddZeeeeejeed d e ee!eiZ"iZ#d gZ$d Z%d Z&dZ'dZ(dZ)dZ*dZ+dZ,dZ-dZ.dZ/dZ0dZ1dZ2dZ3dZ4dZ5dZ6dZ7dZ8e9ej:Z;zeje9ej?e9e;ge=_@WndZ=YdaAd aBd!aCe DZEGd"d#d#eFZGGd$d%d%eGZHGd&d'd'eIZJd(d)ZKd*d+ZLd,d-ZMd.d/ZNd0d1ZOdbd3d4ZPd5d6ZQdcd7d8ZRd9d:ZSdddd?ZUd@dAZVdBdCZWdDdEZXdFdGZYdHdIZZdcdJdKZ[dLdMZ\dNdOZ]dPdQZ^dcdRdSZ_dTdUZ`ed;dddfdVdWZadXdYZbdZd[Zcd\d]Zdd^d_Zed`daZfeSZgeTZheUZieVZjeXZkeYZleZZme\Zne]ZodS)eaA Helper functions for working with the underlying system. These are mostly os dependent, only working on linux, osx, and bsd. In almost all cases they're best-effort, providing **None** if the lookup fails. .. versionchanged:: 1.3.0 Dropped the get_* prefix from several function names. The old names still work, but are deprecated aliases. .. versionchanged:: 1.5.0 Added the **SYSTEM_CALL_TIME** global, which tracks total time spent making system commands. **Module Overview:** :: is_windows - checks if we're running on windows is_mac - checks if we're running on a mac is_gentoo - checks if we're running on gentoo is_slackware - checks if we're running on slackware is_bsd - checks if we're running on the bsd family of operating systems is_available - determines if a command is available on this system is_running - determines if a given process is running size_of - provides the memory usage of an object call - runs the given system command and provides back the results name_by_pid - gets the name for a process by the given pid pid_by_name - gets the pid for a process by the given name pid_by_port - gets the pid for a process listening to a given port pid_by_open_file - gets the pid for the process with an open file pids_by_user - provides processes owned by a user cwd - provides the current working directory for a given process user - provides the user a process is running under start_time - provides the unix timestamp when the process started tail - provides lines from the end of a file bsd_jail_id - provides the BSD jail id a given process is running within bsd_jail_path - provides the path of the given BSD jail is_tarfile - checks if the given path is a tarball expand_path - expands relative paths and ~ entries files_with_suffix - provides files with the given suffix get_process_name - provides our process' name set_process_name - changes our process' name .. data:: Status (enum) State of a subprocess. .. versionadded:: 1.6.0 ==================== =========== Status Description ==================== =========== PENDING not yet started RUNNING currently being performed DONE completed successfully FAILED failed with an exception ==================== =========== N) UNDEFINED)logPENDINGRUNNINGDONEFAILEDcCstj|SN) itertoolschain from_iterableitems)dr2/usr/lib/python3/dist-packages/stem/util/system.pyfsrZulimitzps -A co commandz ps -ao ucomm=zps -p %s -o commz pgrep -x %szpidof %szps -o pid -C %szps axcz lsof -tc %sznetstat -npltuzsockstat -4l -P tcp -p %szlsof -wnP -iTCP -sTCP:LISTENz lsof -tw %szps -o pid -u %szps -o pid -U %szpwdx %szlsof -a -p %s -d cwd -Fnzps -p %s -o jidz jls -j %sigc@s eZdZdZddZddZdS) CallErrora Error response when making a system call. This is an **OSError** subclass with additional information about the process. Depending on the nature of the error not all of these attributes will be available. :var str msg: exception string :var str command: command that was ran :var int exit_status: exit code of the process :var float runtime: time the command took to run :var str stdout: stdout of the process :var str stderr: stderr of the process cCs(||_||_||_||_||_||_dSr)msgcommand exit_statusruntimestdoutstderr)selfrrrrrrrrr__init__s  zCallError.__init__cCs|jSr)r)rrrr__str__szCallError.__str__N)__name__ __module__ __qualname____doc__rrrrrrrs rcs eZdZdZfddZZS)CallTimeoutErrorz Error response when making a system call that has timed out. .. versionadded:: 1.6.0 :var float timeout: time we waited cs$tt|||||||||_dSr)superr!rtimeout)rrrrrrrr# __class__rrrs zCallTimeoutError.__init__)rrrr r __classcell__rrr$rr!sr!c@s6eZdZdZd ddZddZd d Zed d ZdS) DaemonTaska Invokes the given function in a subprocess, returning the value. .. versionadded:: 1.6.0 :var function runner: function to be invoked by the subprocess :var tuple args: arguments to provide to the subprocess :var int priority: subprocess nice priority :var stem.util.system.State status: state of the subprocess :var float runtime: seconds subprocess took to complete :var object result: return value of subprocess if successful :var exception error: exception raised by subprocess if it failed NrFcCsL||_||_||_tj|_d|_d|_d|_d|_ d|_ |r$| dSdSr) runnerargspriorityStaterstatusrresulterror_process_piperun)rr(r)r*startrrrrs zDaemonTask.__init__cCsT|jtjkr(t\|_}tjtj||j |j |j fd|_ |j tj|_dSdS)z\ Invokes the task if it hasn't already been started. If it has this is a no-op. )targetr)N)r,r+rmultiprocessingZPiper0ZProcessr' _run_wrapperr*r(r)r/r2r)rZ child_piperrrr1s    zDaemonTask.runcCs|jtjkr ||jtjkr;|j|j}|d|_|d|_ |jtj kr0|d|_ n |jtj kr;|d|_ |jtj krD|j S|jtj krM|j td|j)z Provides the result of the daemon task. If still running this blocks until the task is completed. :returns: response of the function we ran :raises: exception raised by the function if it failed with one rz+BUG: unexpected status from daemon task, %s)r,r+rr1rr/joinr0Zrecvrrr-rr. RuntimeError)rZresponserrrr8 s            zDaemonTask.joinc Cst}t|zFz|r||n|}|tjt||fWntyA}z|tjt||fWYd}~n d}~wwW|dSW|dS|wr) timeosnicesendr+r Exceptionrclose)Zconnr*r(r) start_timer-excrrrr5+s & zDaemonTask._run_wrapper)NrF) rrrr rr1r8 staticmethodr5rrrrr's   r'cC tdkS)z` Checks if we are running on Windows. :returns: **bool** to indicate if we're on Windows ZWindowsplatformsystemrrrr is_windows9 rGcCrC)z^ Checks if we are running on Mac OSX. :returns: **bool** to indicate if we're on a Mac DarwinrDrrrris_macCrHrJcC tjdS)z] Checks if we're running on Gentoo. :returns: **bool** to indicate if we're on Gentoo z/etc/gentoo-releaser;pathexistsrrrr is_gentooMrHrOcCrK)zv Checks if we are running on a Slackware system. :returns: **bool** to indicate if we're on a Slackware system z/etc/slackware-versionrLrrrr is_slackwareWrHrPcCs tdvS)z Checks if we are within the BSD family of operating systems. This currently recognizes Macs, FreeBSD, and OpenBSD but may be expanded later. :returns: **bool** to indicate if we're on a BSD OS )rIFreeBSDOpenBSDZNetBSDrDrrrris_bsdas rSTcCsd|vr |dd}|tvrdS|r|tvrt|Sdtjvr"dSd}tjdtjD]!}tj||}tr=|d7}tj |rNt |tj rNd}nq-|t|<|S)aS Checks the current PATH to see if a command is available or not. If more than one command is present (for instance "ls -a | grep foo") then this just checks the first. Note that shell (like cd and ulimit) aren't in the PATH so this lookup will try to assume that it's available. This only happends for recognized shell commands (those in SHELL_COMMANDS). :param str command: command to search for :param bool cached: makes use of available cached results if **True** :returns: **True** if an executable we can use by that name exists in the PATH, **False** otherwise  rTPATHF.exe) splitSHELL_COMMANDSCMD_AVAILABLE_CACHEr;environpathseprMr8rGrNaccessX_OK)rcachedZ cmd_existsrMZcmd_pathrrr is_availablels&  r_cCst|trz t|dWdStyYdSwtdrUtr%t}t}nt}t}t |d}|s5t |d}|rUdd|D}t j |rG|g}|D] }||vrRdSqIdSdS)a Checks for if a process with a given name or pid is running. .. versionchanged:: 1.6.0 Added support for list and pid arguments. :param str,list,int command: process name if a str, multiple process names if a list, or pid if an int to be checked :returns: **True** if the process is running, **False** if it's not among ps results, and **None** if ps can't be queried rTFpsNcSsg|]}|qSr)strip).0crrr szis_running..) isinstanceintr;killOSErrorr_rSIS_RUNNING_PS_BSDIS_RUNNING_PS_LINUXcallstemutilZ_is_str)rZprimary_resolverZsecondary_resolverZcommand_listingcmdrrr is_runnings4      rocCstjr td|durt}nt||vrdSzt|}Wnty.td}Ynw| t|t |t vrNt t ||D] }|t ||7}qD|S)aF Provides the `approximate memory usage of an object `_. This can recurse tuples, lists, deques, dicts, and sets. To teach this function to inspect additional object types expand SIZE_RECURSES... :: stem.util.system.SIZE_RECURSES[SomeClass] = SomeClass.get_elements .. versionadded:: 1.6.0 :param object obj: object to provide the size of :param set exclude: object ids to exclude from size estimation :returns: **int** with the size of the object in bytes :raises: **NotImplementedError** if using PyPy z'PyPy does not implement sys.getsizeof()Nr) rlZprereqZis_pypyNotImplementedErrorsetidsys getsizeof TypeErroraddtype SIZE_RECURSESsize_of)objZexcludesizeentryrrrrys    rycCsd}tjjr"ztjj|tjjjjd}Wn ty!Ynw|sLztt |}Wn t y7d}Ynw|rLt |dkrL|ddkrL|d }|S)a< Attempts to determine the name a given process is running under (not including arguments). This uses... :: 1. Information from /proc 2. ps -p -o command :param int pid: process id of the process to be queried :returns: **str** with the process name, **None** if it can't be determined Nrr7COMMANDr6) rlrmprocr_statsStatr}IOErrorrkGET_NAME_BY_PID_PSrhlenra)pid process_nameresultsrrr name_by_pids       rFcstdr0ttd}|r0zttt|}|r|WSt|dkr%|dWSWn ty/Ynwtdrjttd}|rjt|dkrjzttt|d }|rT|WSt|dkr_|dWSWn tyiYnwtdrt stt d}|rzttt|dd}|r|WSt|dkr|dWSWn tyYnwt rtt d}|rׇfdd|D}zttt|}|r|WSt|dkr|dWSWn tyYnwtd r tt d}|r zttt|}|r|WSt|dkr|dWSWn ty Ynwtd rYtrYd sd g}tjjd d}|rYtd }|D]}||}|rG|t|d q3|rN|St|dkrY|dStd|regSdS)a Attempts to determine the process id for a running process, using... :: 1. pgrep -x 2. pidof 3. ps -o pid -C (linux) ps axc | egrep " $" (bsd) 4. lsof -tc 5. tasklist | str .exe :param str process_name: process name for which to fetch the pid :param bool multiple: provides a list of all pids if **True**, otherwise results with multiple processes are discarded :returns: Response depends upon the 'multiple' argument as follows... * if **False** then this provides an **int** with the process id or **None** if it can't be determined * if **True** then this provides a **list** of all **int** process ids, and an empty list if it can't be determined ZpgrepNr6rpidofr`cs&g|]}|dr|dqS)z %sr)endswithrWrbrrrrrds&zpid_by_name..lsofZtasklistrVz^\s*%s\s+(?P[0-9]*)rz failed to resolve a pid for '%s')r_rkGET_PID_BY_NAME_PGREPlistmaprfr ValueErrorGET_PID_BY_NAME_PIDOFrWrSGET_PID_BY_NAME_PS_LINUXGET_PID_BY_NAME_PS_BSDGET_PID_BY_NAME_LSOFrGrrlrmrFrecompilesearchappendgrouprdebug)rZmultiplerZpidsZ process_idsZtasklist_regexlinematchrrr pid_by_name+s                    rcs*tdr=ttd}|r=fdd|D}t|dkr=t|ddkr=|dd}|d|d }|r=t|Std rittd}|rifd d|D}t|dkri|dd }|rit|Std rtt d}|rfdd|D}t|dkr|dd}|rt|SdS)a Attempts to determine the process id for a process with the given port, using... :: 1. netstat -npltu | grep 127.0.0.1: 2. sockstat -4l -P tcp -p 3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":" Most queries limit results to listening TCP connections. This function likely won't work on Mac OSX. :param int port: port where the process we're looking for is listening :returns: **int** with the process id, **None** if it can't be determined ZnetstatNcsg|] }d|vr|qS)z 127.0.0.1:%srrportrrrdszpid_by_port..r6r/Zsockstatc4g|]}t|dkrd|dvr|qS)r:%srrWrrrrrd#4r7rcr) rrrrrrrdCr) r_rkGET_PID_BY_PORT_NETSTATrrWfindisdigitrfGET_PID_BY_PORT_SOCKSTATGET_PID_BY_PORT_LSOF)rrrrrr pid_by_ports4'     rcCsBtdrtt|g}t|dkr|d}|rt|SdS)a Attempts to determine the process id for a process with the given open file, using... :: lsof -w :param str path: location of the socket file to query against :returns: **int** with the process id, **None** if it can't be determined rr6rN)r_rkGET_PID_BY_FILE_LSOFrrarrf)rMrrrrrpid_by_open_fileNs  rcCsbtdr/trtt|d}ntt|d}|r/z ttt|ddWSty.YdSwdS)z Provides processes owned by a given user. .. versionadded:: 1.5.0 :param str user: user to look up processes for :returns: **list** with the process ids, **None** if it can't be determined r`Nr6) r_rSrkGET_PIDS_BY_USER_BSDGET_PIDS_BY_USER_LINUXrrrfr)userrrrr pids_by_userts rcCsFtjjrztjj|WStyYnwd|}tdrntt|d}|s1t d|n=|d dr@t d|n.t |dksX|d d dksX|d d |sbt d ||fn |dd ddStd rtt|g}t |d kr|d dr|ddd}d |vr|SdSt d||fdS)z Provides the working directory of the given process. :param int pid: process id of the process to be queried :returns: **str** with the absolute path for the process' present working directory, **None** if it can't be determined zcwd(%s):ZpwdxNz!%s pwdx didn't return any resultsrzNo such processz'%s pwdx processes reported for this pidr6rTz%s: z)%s we got unexpected output from pwdx: %srr7rzn/z)%s we got unexpected output from lsof: %s)rlrmr~r_cwdrrk GET_CWD_PWDXrrrrcount startswithrWra GET_CWD_LSOF)rZlogging_prefixrZ lsof_resultrrrrs2  0rcCst|tr |dkr dStjjr2zddl}tjj|}|r,|r,| t|j WSWnYtdrIt d|g}t |dkrI|d SdS)z Provides the user a process is running under. :param int pid: process id of the process to be queried :returns: **str** with the username a process is running under, **None** if it can't be determined rNr`z ps -o user %sr7r6)rerfrlrmr~r_pwduidrgetpwuidZpw_namerkrra)rrrrrrrrs      rcCst|tr |dkr dStjjr,zttjj|tjjjj dWSt y+Ynwz"t d|g}t |dkrL|d }ttjj|WSWdSYdS)z Provides the unix timestamp when the given process started. :param int pid: process id of the process to be queried :returns: **float** for the unix timestamp when the process began, **None** if it can't be determined rNzps -p %s -o etimer7r6)rerfrlrmr~r_floatrrZ START_TIMErrkrrar: str_toolsZparse_short_time_label)rZ ps_resultsZetimerrrr@s$  "   r@ccsPt|tr(t|d}t||D]}|Vq WddS1s#wY|dd|}d}d}|dus>|dkr|dkr|tdkr^||td|t|dd\}}n |dd|||}t | D]}|dus{|dkr|dur|d8}t j j |Vqq|t8}|d8}|dus|dkr|dksBdSdSdSdS) a Provides lines of a file starting with the end. For instance, 'tail -n 50 /tmp/my_log' could be done with... :: reversed(list(tail('/tmp/my_log', 50))) :param str,file target: path or file object to read from :param int lines: number of lines to read :returns: **generator** that reads lines, starting with the end :raises: **IOError** if unable to read the file rbNrr7r r6)restropentailseektell BLOCK_SIZEreadrWreversed splitlinesrlrmrZ _to_unicode)r3linesZ target_filerZblock_end_byteZ block_numberZcontentZcompleted_linesrrrrs4     (rcCstt|g}t|dkr%t|ddkr%|d}|r%t|St}|dkr6t d|dSt d||fdS)a Gets the jail id for a process. These seem to only exist for FreeBSD (this style for jails does not exist on Linux, OSX, or OpenBSD). :param int pid: process id of the jail id to be queried :returns: **int** for the jail id, zero if this can't be determined r7r6rQz)Unable to get the jail id for process %s.z,bsd_jail_id(%s): jail ids do not exist on %sr) rkGET_BSD_JAIL_ID_PSrrWrarrfrErFrwarnr)r ps_outputjidZos_namerrr bsd_jail_idNs  rcCsJ|dkr#tt|g}t|dkr#t|ddkr#|ddSdS)z Provides the path of the given FreeBSD jail. :param int jid: jail id to be queried :returns: **str** of the path prefix, **None** if this can't be determined rr7r6N)rkGET_BSD_JAIL_PATHrrW)rZ jls_outputrrr bsd_jail_pathqs   rc Cs6zt|WSttfyt|ddkYSw)z Returns if the path belongs to a tarfile or not. .. versionadded:: 1.2.0 :param str path: path to be checked :returns: **True** if the path belongs to a tarball, **False** otherwise rzapplication/x-tar)tarfile is_tarfilerAttributeError mimetypesZ guess_type)rMrrrrs  rcCstr |ddd}n|d}|rtj|r |S|dr*tj|}|S|s0t}|ds:|drA|dd}n|dkrGd }|d krO|}|Stj ||}|S) a Provides an absolute path, expanding tildes with the user's home and appending a current working directory if the path was relative. :param str path: path to be expanded :param str cwd: current working directory to expand relative paths with, our process' if this is **None** :returns: **str** of the path expanded to be an absolute path, never with an ending slash r\~z./z.\r7N.) rGreplacerstripr;rMisabsr expandusergetcwdr8)rMrZ relative_pathrrr expand_paths(   rccsdtj|r||r|VdSdSt|D]\}}}|D]}||r.tj||VqqdS)a3 Iterates over files in a given directory, providing filenames with a certain suffix. .. versionadded:: 1.2.0 :param str base_path: directory to be iterated over :param str suffix: filename suffix to look for :returns: iterator that yields the absolute path for files with the given suffix N)r;rMisfilerwalkr8) base_pathsuffixroot_filesfilenamerrrfiles_with_suffixs   rcCs<t|tr |d}nttt|}d\}}} } t} zdz|dtv} tj|tj tj | ||d} |r\| dur\t| |krQt d|d |d|dd|t d| dus:| \} } | | } } t| }td ||ftrd |}| r| rt|d | | fn| rt|d | n | rt|d | | }|s|dkrtd||f| r| ddWWttt| 7aWdS1swYSgWWttt| 7aWdS1swYSt y5td||f|tkr4|YWttt| 7aWdS1s.wYSty}z@td||f|tkro|WYd}~Wttt| 7aWdS1siwYStt|d |||| | d}~wwttt| 7aWdw1swYw)a5 call(command, default = UNDEFINED, ignore_exit_status = False) Issues a command in a subprocess, blocking until completion and returning the results. This is not actually ran in a shell so pipes and other shell syntax are not permitted. .. versionchanged:: 1.5.0 Providing additional information upon failure by raising a CallError. This is a subclass of OSError, providing backward compatibility. .. versionchanged:: 1.5.0 Added env argument. .. versionchanged:: 1.6.0 Added timeout and cwd arguments. :param str,list command: command to be issued :param object default: response if the query fails :param bool ignore_exit_status: reports failure if our command's exit status was non-zero :param float timeout: maximum seconds to wait, blocks indefinitely if **None** :param dict env: environment variables :returns: **list** with the lines of output from the command :raises: * **CallError** if this fails and no default was provided * **CallTimeoutError** if the timeout is reached without a default rT)NNNNr)rrshellrenvNz)Process didn't finish after %0.1f secondsrgMbP?z System call: %s (runtime: %0.2f)zReceived from system (%s)z, stdout: %s stderr: %sz , stdout: %sz , stderr: %sz%s returned exit status %izutf-8rz(System call (timeout): %s (after %0.4fs)z$System call (failed): %s (error: %s))rerrWrrr:rX subprocessPopenPIPEZpollr!r8sleepZ communicaterarr is_tracingZtracerhdecoderSYSTEM_CALL_TIME_LOCKSYSTEM_CALL_TIMErr)rdefaultZignore_exit_statusr#rrZ command_listrrrrr@Zis_shell_commandZprocessZ trace_prefixrArrrrksv %              rkc CstdurStdtg}t|dkr|ddvr|dan1gt}}tdD]!}z ||dur4WnWn ty?Yn w|t ||q(d |atta tS) zi Provides the present name of our process. :returns: **str** with the present name of our process Nzps -p %i -o argsr7r)r}ZARGSr6drT) _PROCESS_NAMErkr;getpidrargc_trangerrrr8_MAX_NAME_LENGTH)rr)argcirrrget_process_nameJs"      rcCs<t|tdkrt|dStdvrt|dSdS)z Renames our current process from "python " to a custom name. This is best-effort, not necessarily working on all platforms. :param str process_name: new name for our process :raises: **IOError** if the process cannot be renamed ZLinux)rIrQrRN) _set_argvrErF_set_prctl_name_set_proc_titlerrrrset_process_name~s     rcCstdurdSt}tdt}}t|t|t|tkr$tdt t|t|}t |j d|d| d}t |j |t||adS)zi Overwrites our argv in a similar fashion to how it's done in C with: strcpy(argv[0], 'new_name'); NrzmCan't rename process to something longer than our initial name (this would overwrite memory used for the env)r6utf8)Py_GetArgcArgvrctypesc_intrZpointerrrrmaxZmemsetcontentsencodeZmemmover)rZ current_nameargvrZ zero_sizeZprocess_name_encodedrrrrs  rcCsPttjd}tt|d}tjj||_ | t t |ddddS)a  Sets the prctl name, which is used by top and killall. This appears to be Linux specific and has the max of 15 characters. This is from... http://stackoverflow.com/questions/564695/is-there-a-way-to-change-effective-process-name-in-python/923034#923034 rcr6rN) rCDLLrm find_librarycreate_string_bufferrrlrZ _to_bytesvalueZprctl PR_SET_NAMEbyrefrZlibcZ name_bufferrrrrs rcCsZttjd}tt|d}||_z |t |WdSt y,YdSw)z BSD specific calls (should be compataible with both FreeBSD and OpenBSD: http://fxr.watson.org/fxr/source/gen/setproctitle.c?v=FREEBSD-LIBC http://www.rootr.net/man/man/setproctitle/3 rcr6N) rrrmrrrr rZ setproctitlerrrrrrrs  r)Tr)F)pr collectionsrZ ctypes.utilr rr4r;rErrrsrZ threadingr:Z stem.prereqrlZ stem.utilZstem.util.enumZstem.util.procZstem.util.str_toolsrrrmenumZ UppercaseEnumr+tupleiterrdequedictrq frozensetrxrYrXrjrirrrrrrrrrrrrrrrrrrZPOINTERZc_char_prZ pythonapirZrestyper argtypesrrrRLockrrhrr!objectr'rGrJrOrPrSr_roryrrrrrrrr@rrrrrrrkrrrrrZget_name_by_pidZget_pid_by_nameZget_pid_by_portZget_pid_by_open_fileZget_cwdZget_userZget_start_timeZget_bsd_jail_idZget_bsd_jail_pathrrrrs?        Z    + >+ +,x& E! 5# -e4"