&thR.\dZddlmZddlZddlmZddlmZddlmZm Z ddl m Z ddl m Z dd lmZdd lmZejd ejZejd ejZejd ejZejdejejzZejdejejzZejdejZdZ d4d5dZd6d7d Zd8d9d#Zejd$ejZ d6d:d%Z! d;dd(Z# d?d@d+Z$ dAdBd,Z% dCdDd.Z& dEdFd2Z'dGd3Z(dS)Hz( Functions for dealing with markup text ) annotationsN)Iterable)name2codepoint)MatchPattern)urljoin) StrOrBytes)safe_url_string) to_unicodezI&((?P[a-z\d]+)|#(?P\d+)|#x(?P[a-f\d]+))(?P;?)z<[a-zA-Z\/!].*?>z5]*href\s*=\s*[\"\']\s*([^\"\'\s]+)\s*[\"\']z}]*http-equiv[^>]*refresh[^>]*content\s*=\s*(?P["\'])(?P(\d*\.)?\d+)\s*;\s*url=\s*(?P.*?)(?P=quote)z]*content\s*=\s*(?P["\'])(?P(\d*\.)?\d+)\s*;\s*url=\s*(?P.*?)(?P=quote)[^>]*?\shttp-equiv\s*=[^>]*refreshz<((?P.*?)(?P\]\]>))z Tutf-8textr keep Iterable[str]remove_illegalboolencodingstrreturncfdfd }t|t||S)uRemove entities from the given `text` by converting them to their corresponding unicode character. `text` can be a unicode string or a byte string encoded in the given `encoding` (which defaults to 'utf-8'). If `keep` is passed (with a list of entity names) those entities will be kept (they won't be removed). It supports both numeric entities (``&#nnnn;`` and ``&#hhhh;``) and named entities (such as `` `` or ``>``). If `remove_illegal` is ``True``, entities that can't be converted are removed. If `remove_illegal` is ``False``, entities that can't be converted are kept "as is". For more information see the tests. Always returns a unicode string (with the entities removed). >>> import w3lib.html >>> w3lib.html.replace_entities(b'Price: £100') 'Price: \xa3100' >>> print(w3lib.html.replace_entities(b'Price: £100')) Price: £100 >>> m Match[str]rrc|}d}|drt|dd}n|drt|dd}n|drm|d}|vr|dSt j|p%t j|}|Z d|cxkrdkr&nn#t |fd St|S#ttf$rYnwxYwr|d rd n|dS) Ndec hexnamedrcp1252 semicolon) groupdictgetintlowergrouprbytesdecodechr ValueError OverflowError)rgroupsnumber entity_namerrs ^/var/www/html/mycamper/aliexpress-site/backend/venv/lib/python3.11/site-packages/w3lib/html.pyconvert_entityz(replace_entities..convert_entityHs ::e   ++FF ZZ   ++FF ZZ   /K  ""d**wwqzz!#' 448J!!##99F    6))))T))))) &++228<<<6{{" .    $O ;(?(?OrrQWWQZZOs52D7(D77E  E rrrr)_ent_resubr )rrrrr2s `` r1replace_entitiesr6'sOBPPPPPPP8 ;;~z$'A'A B BB str | Noneclttt||SN)rr4searchr )rrs r1 has_entitiesr<gs& z$99:: ; ;;r7r#tokencTt|t||S)acReplace all markup tags found in the given `text` by the given token. By default `token` is an empty string so it just removes all tags. `text` can be a unicode string or a regular string encoded as `encoding` (or ``'utf-8'`` if `encoding` is not given.) Always returns a unicode string. Examples: >>> import w3lib.html >>> w3lib.html.replace_tags('This text contains some tag') 'This text contains some tag' >>> w3lib.html.replace_tags('

Je ne parle pas fran\xe7ais

', ' -- ', 'latin-1') ' -- Je ne parle pas -- fran\xe7ais -- -- ' >>> )_tag_rer5r )rr=rs r1 replace_tagsr@ks"( ;;ujx88 9 99r7z|$)cXt||}td|S)zRemove HTML Comments. >>> import w3lib.html >>> w3lib.html.remove_comments(b"test whatever") 'test whatever' >>> r#)r _REMOVECOMMENTS_REr5)rrutexts r1remove_commentsrDs) tX & &E  ! !"e , ,,r7 which_onescrrtddDdDd fd dfd }d }tj|tjtjz}||t ||S)a;Remove HTML Tags only. `which_ones` and `keep` are both tuples, there are four cases: ============== ============= ========================================== ``which_ones`` ``keep`` what it does ============== ============= ========================================== **not empty** empty remove all tags in ``which_ones`` empty **not empty** remove all tags except the ones in ``keep`` empty empty remove all tags **not empty** **not empty** not allowed ============== ============= ========================================== Remove all tags: >>> import w3lib.html >>> doc = '

This is a link: example

' >>> w3lib.html.remove_tags(doc) 'This is a link: example' >>> Keep only some tags: >>> w3lib.html.remove_tags(doc, keep=('div',)) '
This is a link: example
' >>> Remove only specific tags: >>> w3lib.html.remove_tags(doc, which_ones=('a','b')) '

This is a link: example

' >>> You can't remove some and keep some: >>> w3lib.html.remove_tags(doc, which_ones=('a',), keep=('p',)) Traceback (most recent call last): ... ValueError: Cannot use both which_ones and keep >>> z#Cannot use both which_ones and keepc6h|]}|Sr r'.0tags r1 zremove_tags..s 444##))++444r7c6h|]}|Sr rHrIs r1rLzremove_tags..s ( ( (CCIIKK ( ( (r7rKrrrc@|}r|vS|vSr:rH)rKrrEs r1 will_removez remove_tags..will_removes-iikk  %*$ $$r7rrcr|d}|rdn|dS)Nr#r)r()rrKrOs r1 remove_tagzremove_tags..remove_tags5ggajj [%%5rr1771::5r7z/]+).*?>)rKrrrr3)r,recompileDOTALL IGNORECASEr5r )rrErrrRregexretagsrOs `` @r1 remove_tagsrYsb@d@>???44444J ( (4 ( ( (D 666666 E Zry2=8 9 9F ::j*T8"<"< = ==r7ct||}|radd|D}tj|tjtjz}|d|}|S)aRemove tags and their content. `which_ones` is a tuple of which tags to remove including their content. If is empty, returns the string unmodified. >>> import w3lib.html >>> doc = '

This is a link: example

' >>> w3lib.html.remove_tags_with_content(doc, which_ones=('b',)) '' >>> |c &g|]}d|d|d|dS)|r rIs r1 z,remove_tags_with_content..s4SSS._get_fragmentss{%%c**  E$zz!}} GWfWn% % % %KKKFF&''lr7r#)rrcdata_d)rlrrmrnrro)r _cdata_re isinstancerr6r()rrrrrwrCret_textfragments r1unquote_markupr}s" tX & &EH"N5)4422 h $ $ 2 (tN HH y11 1HH Or7baseurlct||}t|x}r?tt |t |d|St |S)zReturn the base url if declared in the given HTML `text`, relative to the given base url. If no base url is found, the given `baseurl` is returned. )rrQ)rD _baseurl_rer;rr r()rr~rrCrs r1 get_base_urlr0sw!999E   u % %%q  G $ $oaggajj8&T&T&T    7 # ##r7scriptnoscript ignore_tags%tuple[None, None] | tuple[float, str]c t||}n#t$rt|wxYwt||}t t |}t |pt|x}rlt| d}t| d d|}t||}||fSdS)aXReturn the http-equiv parameter of the HTML meta element from the given HTML text and return a tuple ``(interval, url)`` where interval is an integer containing the delay in seconds (or zero if not present) and url is a string with the absolute url to redirect. If no meta redirect is found, ``(None, None)`` is returned. r&urlz "')NN)r UnicodeDecodeErrorprintrarDr6_meta_refresh_rer;_meta_refresh_re2floatr(r stripr)rr~rrrCrintervalrs r1get_meta_refreshrBs4**  d   %UK 8 8E ,U33 4 4E  # #E * * M.?.F.Fu.M.MMq((aggenn226::HEEgs##} :s.c6|tS)a Strip all leading and trailing space characters (as defined in https://www.w3.org/TR/html5/infrastructure.html#space-character). Such stripping is useful e.g. for processing HTML element attributes which contain URLs, like ``href``, ``src`` or form ``action`` - HTML5 standard defines them as "valid URL potentially surrounded by spaces" or "valid non-empty URL potentially surrounded by spaces". >>> strip_html5_whitespace(' hello\n') 'hello' )rHTML5_WHITESPACE)rs r1strip_html5_whitespacer`s ::& ' ''r7)r Tr ) rr rrrrrrrrr:)rr rr8rr)r#N)rr r=rrr8rr)rr rr8rr)r r N) rr rErrrrr8rr)r N)rr rErrr8rr)rbr#N) rr rErrfr rr8rr)r TN) rr rrrrrr8rr)r#r )rr r~r rrrr)r#r r) rr r~rrrrrrr)rrrr))__doc__ __future__rrScollections.abcr html.entitiesrrr urllib.parser w3lib._typesr w3lib.urlr w3lib.utilr rTrVr4rUr?Irrrryrr6r<r@rBrDrYrarjr}rrrr r7r1rs#""""" $$$$$$(((((( ######%%%%%%!!!!!! "*PM   "*(") 4 4bjQSUSWXX 2:EI BJLI  BJCRY   !  =C=C=C=C=C@<<<<<:::::. RZ 2BI>> - - - - - !# D>D>D>D>D>PNR4!3 0 %%%%%RAH$$$$$(!7 < ( ( ( ( ( (r7