o _c2j@sddlZddlZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZddlZddlZddlmZddlmZdd lmZddlZdd lmZdd lmZdd lmZdd l m!Z"ddl#m$Z%ddl#m&Z&m'Z'ddl(m)Z)dZ*dZ+dZ,gdZ-eZ.de._/de._0da1eeddfZ2eee3ede3dfZ4eeddfZ5ee3e6fZ7e3Z8e3Z9e3Z:e3Z;e6Ze3Z?e'Z@ee3ee3fZAe e3ZBeZCedee3fZDee3ZEzejFjGdksJejFjHdksJWn eIyeJdwGdddeKZLGdddZMGd ddeMZNGd!ddeMZOGd"d#d#ejPZQd0d$e9fd%d&ZRd1d(d)ZSGd*d+d+ejTZUGd,d-d-eUZVGd.d/d/eUZWdS)2N)urlparse urlunparseurljoin)ThreadPoolExecutor) TimeoutError)partial)SetUnionListMutableMappingOptional)PyQuery) UserAgent)Cleaner)etree) HtmlElement)tostring) fromstring)search)findallResult)html_to_unicodezutf-8zhttps://example.org/zuMozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/603.3.8 (KHTML, like Gecko) Version/10.1.2 Safari/603.3.8)nextmoreZolderTElementrHTMLz#Requests-HTML requires Python 3.6+!c@seZdZddZdS) MaxRetriescCs ||_dSN)message)selfr r"8/usr/local/lib/python3.10/dist-packages/requests_html.py__init__As zMaxRetries.__init__N)__name__ __module__ __qualname__r$r"r"r"r#r?s rc@seZdZdZddddedededdfdd Zede fd d Z ede fd d Z e j deddfdd Z e j deddfdd Z edefddZej deddfddZedefddZedefddZedefddZedefddZd6dddddded ed!ed"ed#edef d$d%Zdddd&ded!ed"ed#edef d'd(Zd)edefd*d+Z d)ede!fd,d-Z"ede#fd.d/Z$d0d1Z%ede#fd2d3Z&edefd4d5Z'dS)7 BaseParseraJA basic HTML/Element Parser, for Humans. :param element: The element from which to base the parsing upon. :param default_encoding: Which encoding to default to. :param html: HTML from which to base the parsing upon (optional). :param url: The URL from which the HTML originated, used for ``absolute_links``. N)default_encodinghtmlr)r*urlreturncCsH||_||_d|_||_d|_t|tr|tn||_ d|_ d|_ dS)NT) elementr+ skip_anchorsr) _encoding isinstancestrencodeDEFAULT_ENCODING_html_lxml_pq)r!r-r)r*r+r"r"r#r$Os zBaseParser.__init__cCs(|jr|jStj|jdd|jS)zyBytes representation of the HTML content. (`learn more `_). unicodeencoding)r4rrr-stripr2r9r!r"r"r#raw_htmlYszBaseParser.raw_htmlcCs,|jr |jj|jddStj|jddS)zzUnicode representation of the HTML content (`learn more `_). replaceerrorsr7r8)r4r<decoder9rrr-r:r;r"r"r#r*cszBaseParser.htmlcCs||j|_dSr)r2r9r4r!r*r"r"r#r*mscC ||_dS)zProperty setter for self.html.N)r4rAr"r"r#r<q cCsj|jr|jS|jr,t|j|jd|_z |jj|jddWn ty+|j|_Ynw|jr2|jS|jS)zzThe encoding string to be used, extracted from the HTML and :class:`HTMLResponse ` headers. rr=r>)r/r4rr)r<r@r9UnicodeDecodeErrorr;r"r"r#r9vs  zBaseParser.encodingenccCrB)z"Property setter for self.encoding.N)r/)r!rEr"r"r#r9rCcCs|jdur t|j|_|jS)z`PyQuery `_ representation of the :class:`Element ` or :class:`HTML `. N)r6r lxmlr;r"r"r#pqs  z BaseParser.pqcCsP|jdur%z t|jdd|_W|jSty$tj|j|_Y|jSw|jS)zs`lxml `_ representation of the :class:`Element ` or :class:`HTML `. Nz html.parser)features)r5 soup_parser* ValueErrorrFrr<r;r"r"r#rFs  zBaseParser.lxmlcC |jS)z\The text content of the :class:`Element ` or :class:`HTML `. )rGtextr;r"r"r#rL zBaseParser.textcCrK)zsThe full text content (including links) of the :class:`Element ` or :class:`HTML `. )rFZ text_contentr;r"r"r# full_textrMzBaseParser.full_text*F) containingcleanfirstr/selectorrPrQrRr/cst|tr|g}|p jfdd|D}|r;|}g}|D]tfdd|Dr6|q$||rV|}g}|D]tt j _ |qEt ||S)aCGiven a CSS Selector, returns a list of :class:`Element ` objects or a single one. :param selector: CSS Selector to use. :param clean: Whether or not to sanitize the found HTML of ``