JFIF $
d d 7
Viewing File: /usr/lib/python3.9/site-packages/pip/_vendor/html5lib/__pycache__/_tokenizer.cpython-39.pyc
a
[��f�, � @ s� d dl mZmZmZ d dlmZ d dlmZm Z d dl
m Z ddl m
Z
ddl mZ ddl mZmZ dd l mZmZmZ dd
l mZmZ dd l mZ dd lmZ dd
lmZ ee�Ze dkr�e Z ne Z G dd� de �Z dS )� )�absolute_import�division�unicode_literals)�unichr)�deque� OrderedDict)� version_info� )�spaceCharacters)�entities)� asciiLetters�asciiUpper2Lower)�digits� hexDigits�EOF)�
tokenTypes�
tagTokenTypes)�replacementCharacters)�HTMLInputStream)�Trie)� � c sd e Zd ZdZd�� fdd� Zdd� Zdd� Zd�d
d �Zd d
� Zdd� Z dd� Z
dd� Z dd� Z dd� Z
dd� Zdd� Zd d � Zd d � Zd d!� Zd"d#� Zd$d%� Zd&d'� Zd(d)� Zd*d+� Zd,d-� Zd.d/� Zd0d1� Zd2d3� Zd4d5� Z d6d7� Z d8d9� Z d:d;� Z d<d=� Z d>d?� Z!d@dA� Z"dBdC� Z#dDdE� Z$dFdG� Z%dHdI� Z&dJdK� Z'dLdM� Z(dNdO� Z)dPdQ� Z*dRdS� Z+dTdU� Z,dVdW� Z-dXdY� Z.dZd[� Z/d\d]� Z0d^d_� Z1d`da� Z2dbdc� Z3ddde� Z4dfdg� Z5dhdi� Z6djdk� Z7dldm� Z8dndo� Z9dpdq� Z:drds� Z;dtdu� Z<dvdw� Z=dxdy� Z>dzd{� Z?d|d}� Z@d~d� ZAd�d�� ZBd�d�� ZCd�d�� ZDd�d�� ZEd�d�� ZFd�d�� ZGd�d�� ZHd�d�� ZId�d�� ZJd�d�� ZKd�d�� ZL� ZMS )��
HTMLTokenizera This class takes care of tokenizing HTML.
* self.currentToken
Holds the token that is currently being processed.
* self.state
Holds a reference to the method to be invoked... XXX
* self.stream
Points to HTMLInputStream object.
Nc sJ t |fi |��| _|| _d| _g | _| j| _d| _d | _t t
| �� � d S )NF) r �stream�parserZ
escapeFlagZ
lastFourChars� dataState�state�escape� currentToken�superr �__init__)�selfr r �kwargs�� __class__� �C/usr/lib/python3.9/site-packages/pip/_vendor/html5lib/_tokenizer.pyr ( s zHTMLTokenizer.__init__c c sP t g �| _| �� rL| jjr6td | jj�d�d�V q| jr
| j�� V q6q
dS )z� This is where the magic happens.
We do our usually processing through the states and when we have a token
to return we yield the token which pauses processing until the next token
is requested.
�
ParseErrorr ��type�dataN)r �
tokenQueuer r �errorsr �pop�popleft�r! r% r% r& �__iter__7 s
zHTMLTokenizer.__iter__c C s� t }d}|rt}d}g }| j�� }||v rH|turH|�|� | j�� }q"td�|�|�}|tv r�t| }| j �t
d dd|id�� �n d| kr�d ks�n |d
kr�d }| j �t
d dd|id�� n�d | kr�d
k�s>n d| kr�dk�s>n d| k�rdk�s>n d| k�r*dk�s>n |t g d��v �rZ| j �t
d dd|id�� z t |�}W n< t
�y� |d }t d|d? B �t d|d@ B � }Y n0 |dk�r�| j �t
d dd�� | j�|� |S )z�This function returns either U+FFFD or the character based on the
decimal or hexadecimal representation. It also discards ";" if present.
If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked.
�
� � r' z$illegal-codepoint-for-numeric-entity� charAsInt�r) r* Zdatavarsi � i�� � � �r � � � � � i� i� )#� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i��
i��
i�� i�� i�� i�� i��
i��
i�� i�� i�� i�� i�� r6 i i � i� �;z numeric-entity-without-semicolonr( )r r r �charr �append�int�joinr r+ r � frozenset�chr�
ValueError�unget) r! ZisHex�allowed�radix� charStack�cr4 r? �vr% r% r& �consumeNumberEntityG sn
� � �
��
�
�
�
�
�
� &
� z!HTMLTokenizer.consumeNumberEntityFc C s� d}| j �� g}|d tv sB|d tddfv sB|d urV||d krV| j �|d � �n |d dk�r d}|�| j �� � |d dv r�d}|�| j �� � |r�|d tv s�|s�|d tv r�| j �|d � | �|�}n4| j �t
d d
d �� | j �|� � � dd � |� }�nf|d tu�rDt
�d � |���s0�qD|�| j �� � �q z$t
�d � |d d� ��}t|�}W n t�y� d }Y n0 |d u�r>|d d
k�r�| j �t
d dd �� |d d
k�r|�r|| tv �s�|| tv �s�|| dk�r| j �|� � � dd � |� }n.t| }| j �|� � � |d � ||d � �7 }n4| j �t
d dd �� | j �|� � � dd � |� }|�r�| jd d d |7 < n*|tv �r�d}nd}| j �t
| |d �� d S )N�&r �<�#F���)�x�XTr' zexpected-numeric-entityr( r3 r> z named-entity-without-semicolon�=zexpected-named-entityr* r �SpaceCharacters�
Characters)r r? r
r rF r@ r r rL r+ r r- rB � entitiesTrieZhas_keys_with_prefixZlongest_prefix�len�KeyErrorr r r ) r! � allowedChar�
fromAttribute�outputrI �hexZ
entityNameZ entityLengthZ tokenTyper% r% r& �
consumeEntity� s~ �
� �
� �
�
�
�
� �
zHTMLTokenizer.consumeEntityc C s | j |dd� dS )zIThis method replaces the need for "entityInAttributeValueState".
T)rY rZ N)r] )r! rY r% r% r& �processEntityInAttribute� s z&HTMLTokenizer.processEntityInAttributec C s� | j }|d tv r�|d �t�|d< |d td krp|d }t|�}t|�t|�krh|�|ddd� � ||d< |d td kr�|d r�| j� td d d
�� |d r�| j� td d d
�� | j� |� | j
| _ dS )
z�This method is a generic handler for emitting the tags. It also sets
the state to "data" because that's what's needed after a token has been
emitted.
r) �name�StartTagr* NrP �EndTagr' zattributes-in-end-tagr( � selfClosingz self-closing-flag-on-end-tag) r r � translater
r � attributeMaprW �updater+ r@ r r )r! �token�rawr* r% r% r& �emitCurrentToken� s( � � z HTMLTokenizer.emitCurrentTokenc C s� | j �� }|dkr | j| _n�|dkr.| j| _n�|dkrd| j�td dd�� | j�td dd�� n`|tu rpdS |t v r�| j�td || j �
t d
� d�� n&| j �
d �}| j�td || d�� d
S ) NrM rN � r' �invalid-codepointr( rU FrT T�rM rN ri ) r r? �entityDataStater � tagOpenStater+ r@ r r r
�
charsUntil�r! r* �charsr% r% r& r s.
� �
�
�zHTMLTokenizer.dataStatec C s | � � | j| _dS �NT)r] r r r/ r% r% r& rl s z HTMLTokenizer.entityDataStatec C s� | j �� }|dkr | j| _n�|dkr.| j| _n�|tkr:dS |dkrp| j�td dd�� | j�td d d�� nT|t v r�| j�td
|| j �
t d � d�� n&| j �
d �}| j�td || d�� d S )
NrM rN Fri r' rj r( rU r7 rT Trk ) r r? �characterReferenceInRcdatar �rcdataLessThanSignStater r+ r@ r r
rn ro r% r% r& � rcdataState" s.
� �
�
�zHTMLTokenizer.rcdataStatec C s | � � | j| _dS rq )r] rt r r/ r% r% r& rr ? s z(HTMLTokenizer.characterReferenceInRcdatac C s� | j �� }|dkr | j| _nh|dkrR| j�td dd�� | j�td dd�� n2|tkr^dS | j �d �}| j�td || d�� d
S � NrN ri r' rj r( rU r7 F)rN ri T) r r? �rawtextLessThanSignStater r+ r@ r r rn ro r% r% r& � rawtextStateD s"
� �
�zHTMLTokenizer.rawtextStatec C s� | j �� }|dkr | j| _nh|dkrR| j�td dd�� | j�td dd�� n2|tkr^dS | j �d �}| j�td || d�� d
S ru ) r r? �scriptDataLessThanSignStater r+ r@ r r rn ro r% r% r& �scriptDataStateV s"
� �
�z HTMLTokenizer.scriptDataStatec C sr | j �� }|tkrdS |dkrL| j�td dd�� | j�td dd�� n"| j�td || j �d� d�� dS ) NFri r' rj r( rU r7 T)r r? r r+ r@ r rn �r! r* r% r% r& �plaintextStateh s
� �
�z HTMLTokenizer.plaintextStatec C s | j �� }|dkr | j| _n�|dkr.| j| _n�|tv rVtd |g ddd�| _| j| _n�|dkr�| j �
td dd �� | j �
td
d d �� | j | _nt|d kr�| j �
td d
d �� | j � |� | j
| _n@| j �
td dd �� | j �
td
dd �� | j � |� | j | _dS )N�!�/r` F)r) r_ r* rb ZselfClosingAcknowledged�>r' z'expected-tag-name-but-got-right-bracketr( rU z<>�?z'expected-tag-name-but-got-question-markzexpected-tag-namerN T)r r? �markupDeclarationOpenStater �closeTagOpenStater r r � tagNameStater+ r@ r rF �bogusCommentStaterz r% r% r& rm w s>
�
�
�
� zHTMLTokenizer.tagOpenStatec C s� | j �� }|tv r0td |g dd�| _| j| _n�|dkrX| j�td dd�� | j | _nn|t
u r�| j�td dd�� | j�td d
d�� | j | _n0| j�td d d |id
�� | j � |� | j | _dS )Nra F�r) r_ r* rb r~ r' z*expected-closing-tag-but-got-right-bracketr( z expected-closing-tag-but-got-eofrU �</z!expected-closing-tag-but-got-charr* r5 T)
r r? r r r r� r r+ r@ r r rF r� rz r% r% r& r� � s0
�
�
�
� z HTMLTokenizer.closeTagOpenStatec C s� | j �� }|tv r | j| _n�|dkr.| �� n~|tu rV| j�t d dd�� | j
| _nV|dkrh| j | _nD|dkr�| j�t d dd�� | j d d 7 < n| j d |7 < d
S ) Nr~ r' zeof-in-tag-namer( r} ri rj r_ r7 T)
r r? r
�beforeAttributeNameStater rh r r+ r@ r r �selfClosingStartTagStater rz r% r% r& r� � s&
�
�zHTMLTokenizer.tagNameStatec C sP | j �� }|dkr"d| _| j| _n*| j�td dd�� | j �|� | j | _dS �Nr} r3 rU rN r( T)
r r? �temporaryBuffer�rcdataEndTagOpenStater r+ r@ r rF rt rz r% r% r& rs � s
z%HTMLTokenizer.rcdataLessThanSignStatec C sX | j �� }|tv r*| j|7 _| j| _n*| j�td dd�� | j � |� | j
| _dS �NrU r� r( T) r r? r r� �rcdataEndTagNameStater r+ r@ r rF rt rz r% r% r& r� � s
z#HTMLTokenizer.rcdataEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | �
� | j | _nH|t v r�| j|7 _n0| j
�td d| j d �� | j�|� | j| _d
S � Nr_ ra Fr� r} r~ rU r� r( T)r �lowerr� r r? r
r r� r r� rh r r r+ r@ rF rt �r! Z appropriater* r% r% r& r� � s:
�
�
�
� z#HTMLTokenizer.rcdataEndTagNameStatec C sP | j �� }|dkr"d| _| j| _n*| j�td dd�� | j �|� | j | _dS r� )
r r? r� �rawtextEndTagOpenStater r+ r@ r rF rw rz r% r% r& rv � s
z&HTMLTokenizer.rawtextLessThanSignStatec C sX | j �� }|tv r*| j|7 _| j| _n*| j�td dd�� | j � |� | j
| _dS r� ) r r? r r� �rawtextEndTagNameStater r+ r@ r rF rw rz r% r% r& r� s
z$HTMLTokenizer.rawtextEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | �
� | j | _nH|t v r�| j|7 _n0| j
�td d| j d �� | j�|� | j| _d
S r� )r r� r� r r? r
r r� r r� rh r r r+ r@ rF rw r� r% r% r& r� s:
�
�
�
� z$HTMLTokenizer.rawtextEndTagNameStatec C sx | j �� }|dkr"d| _| j| _nR|dkrJ| j�td dd�� | j| _n*| j�td dd�� | j � |� | j
| _dS ) Nr} r3 r| rU z<!r( rN T) r r? r� �scriptDataEndTagOpenStater r+ r@ r �scriptDataEscapeStartStaterF ry rz r% r% r& rx , s
z)HTMLTokenizer.scriptDataLessThanSignStatec C sX | j �� }|tv r*| j|7 _| j| _n*| j�td dd�� | j � |� | j
| _dS r� ) r r? r r� �scriptDataEndTagNameStater r+ r@ r rF ry rz r% r% r& r� : s
z'HTMLTokenizer.scriptDataEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | �
� | j | _nH|t v r�| j|7 _n0| j
�td d| j d �� | j�|� | j| _d
S r� )r r� r� r r? r
r r� r r� rh r r r+ r@ rF ry r� r% r% r& r� E s:
�
�
�
� z'HTMLTokenizer.scriptDataEndTagNameStatec C sJ | j �� }|dkr2| j�td dd�� | j| _n| j �|� | j| _dS �N�-rU r( T) r r? r+ r@ r � scriptDataEscapeStartDashStater rF ry rz r% r% r& r� a s
z(HTMLTokenizer.scriptDataEscapeStartStatec C sJ | j �� }|dkr2| j�td dd�� | j| _n| j �|� | j| _dS r� ) r r? r+ r@ r � scriptDataEscapedDashDashStater rF ry rz r% r% r& r� k s
z,HTMLTokenizer.scriptDataEscapeStartDashStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrD| j| _nn|dkrz| j�td dd�� | j�td dd�� n8|tkr�| j | _n&| j �
d �}| j�td || d�� d
S ) Nr� rU r( rN ri r' rj r7 )rN r� ri T) r r? r+ r@ r �scriptDataEscapedDashStater �"scriptDataEscapedLessThanSignStater r rn ro r% r% r& �scriptDataEscapedStateu s(
� �
�z$HTMLTokenizer.scriptDataEscapedStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrD| j| _nn|dkr�| j�td dd�� | j�td dd�� | j| _n0|t kr�| j
| _n | j�td |d�� | j| _d S )
Nr� rU r( rN ri r' rj r7 T) r r? r+ r@ r r� r r� r� r r rz r% r% r& r� � s&
� �
z(HTMLTokenizer.scriptDataEscapedDashStatec C s� | j �� }|dkr*| j�td dd�� n�|dkr<| j| _n�|dkrd| j�td dd�� | j| _nn|dkr�| j�td dd�� | j�td d d�� | j| _n0|t kr�| j
| _n | j�td |d�� | j| _d
S ) Nr� rU r( rN r~ ri r' rj r7 T) r r? r+ r@ r r� r ry r� r r rz r% r% r& r� � s*
� �
z,HTMLTokenizer.scriptDataEscapedDashDashStatec C s� | j �� }|dkr"d| _| j| _n\|tv rT| j�td d| d�� || _| j | _n*| j�td dd�� | j �
|� | j | _dS r� ) r r? r� � scriptDataEscapedEndTagOpenStater r r+ r@ r � scriptDataDoubleEscapeStartStaterF r� rz r% r% r& r� � s
z0HTMLTokenizer.scriptDataEscapedLessThanSignStatec C sP | j �� }|tv r"|| _| j| _n*| j�td dd�� | j � |� | j
| _dS r� ) r r? r r� � scriptDataEscapedEndTagNameStater r+ r@ r rF r� rz r% r% r& r� � s
z.HTMLTokenizer.scriptDataEscapedEndTagOpenStatec C s | j o | j d �� | j�� k}| j�� }|tv rT|rTtd | jg dd�| _ | j| _n�|dkr�|r�td | jg dd�| _ | j | _n||dkr�|r�td | jg dd�| _ | �
� | j | _nH|t v r�| j|7 _n0| j
�td d| j d �� | j�|� | j| _d
S r� )r r� r� r r? r
r r� r r� rh r r r+ r@ rF r� r� r% r% r& r� � s:
�
�
�
� z.HTMLTokenizer.scriptDataEscapedEndTagNameStatec C s� | j �� }|ttd�B v rR| j�td |d�� | j�� dkrH| j | _
q�| j | _
nB|t v r�| j�td |d�� | j|7 _n| j �
|� | j | _
dS �N)r} r~ rU r( ZscriptT)r r? r
rC r+ r@ r r� r� � scriptDataDoubleEscapedStater r� r rF rz r% r% r& r� � s
z.HTMLTokenizer.scriptDataDoubleEscapeStartStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrZ| j�td dd�� | j| _nt|dkr�| j�td dd�� | j�td dd�� n>|tkr�| j�td d d�� | j | _n| j�td |d�� d
S � Nr� rU r( rN ri r' rj r7 �eof-in-script-in-scriptT)
r r? r+ r@ r � scriptDataDoubleEscapedDashStater �(scriptDataDoubleEscapedLessThanSignStater r rz r% r% r& r� � s*
� �
�
z*HTMLTokenizer.scriptDataDoubleEscapedStatec C s� | j �� }|dkr2| j�td dd�� | j| _n�|dkrZ| j�td dd�� | j| _n�|dkr�| j�td dd�� | j�td dd�� | j| _nF|t kr�| j�td d d�� | j
| _n | j�td |d�� | j| _d
S r� ) r r? r+ r@ r �$scriptDataDoubleEscapedDashDashStater r� r� r r rz r% r% r& r� s.
� �
�
z.HTMLTokenizer.scriptDataDoubleEscapedDashStatec C s | j �� }|dkr*| j�td dd�� n�|dkrR| j�td dd�� | j| _n�|dkrz| j�td dd�� | j| _n�|dkr�| j�td dd�� | j�td d d�� | j| _nF|t kr�| j�td d
d�� | j
| _n | j�td |d�� | j| _d S ) Nr� rU r( rN r~ ri r' rj r7 r� T) r r? r+ r@ r r� r ry r� r r rz r% r% r& r� % s2
� �
�
z2HTMLTokenizer.scriptDataDoubleEscapedDashDashStatec C sP | j �� }|dkr8| j�td dd�� d| _| j| _n| j �|� | j | _dS )Nr} rU r( r3 T)
r r? r+ r@ r r� � scriptDataDoubleEscapeEndStater rF r� rz r% r% r& r� > s
z6HTMLTokenizer.scriptDataDoubleEscapedLessThanSignStatec C s� | j �� }|ttd�B v rR| j�td |d�� | j�� dkrH| j | _
q�| j | _
nB|t v r�| j�td |d�� | j|7 _n| j �
|� | j | _
dS r� )r r? r
rC r+ r@ r r� r� r� r r� r rF rz r% r% r& r� I s
z,HTMLTokenizer.scriptDataDoubleEscapeEndStatec C s0 | j �� }|tv r$| j �td� �n|tv rJ| jd �|dg� | j| _n�|dkr\| � � n�|dkrn| j
| _n�|dv r�| j �t d dd �� | jd �|dg� | j| _n�|d
kr�| j �t d d d �� | jd �d dg� | j| _nF|t
u �r| j �t d d
d �� | j| _n | jd �|dg� | j| _dS )NTr* r3 r~ r} )�'�"rS rN r' �#invalid-character-in-attribute-namer( ri rj r7 z#expected-attribute-name-but-got-eof)r r? r
rn r r r@ �attributeNameStater rh r� r+ r r r rz r% r% r& r� Y s<
�
�
�
z&HTMLTokenizer.beforeAttributeNameStatec C s� | j �� }d}d}|dkr&| j| _�n.|tv r\| jd d d || j �td� 7 < d}n�|dkrjd}n�|tv r|| j| _n�|dkr�| j | _n�|d kr�| j
� t d
d d �� | jd d d d
7 < d}n�|dv �r | j
� t d
dd �� | jd d d |7 < d}nH|t
u �r6| j
� t d
dd �� | j| _n | jd d d |7 < d}|�r�| jd d d �t�| jd d d< | jd d d� D ]>\}}| jd d d |k�r�| j
� t d
dd �� �qҐq�|�r�| �� dS )NTFrS r* rP r r~ r} ri r' rj r( r7 �r� r� rN r� zeof-in-attribute-namezduplicate-attribute)r r? �beforeAttributeValueStater r r rn r
�afterAttributeNameStater� r+ r@ r r r rc r
rh )r! r* ZleavingThisStateZ emitTokenr_ �_r% r% r& r� w s^
�
�
�
�
� �
z HTMLTokenizer.attributeNameStatec C sD | j �� }|tv r$| j �td� �n |dkr8| j| _�n|dkrJ| �� n�|tv rp| jd � |dg� | j
| _n�|dkr�| j | _n�|dkr�| j � t
d d d
�� | jd � d dg� | j
| _n�|d v r�| j � t
d d
d
�� | jd � |dg� | j
| _nF|tu �r$| j � t
d dd
�� | j| _n | jd � |dg� | j
| _dS )NTrS r~ r* r3 r} ri r' rj r( r7 r� z&invalid-character-after-attribute-namez expected-end-of-tag-but-got-eof)r r? r
rn r� r rh r r r@ r� r� r+ r r r rz r% r% r& r� � s@
�
�
�
z%HTMLTokenizer.afterAttributeNameStatec C sh | j �� }|tv r$| j �td� �n@|dkr8| j| _�n,|dkrX| j| _| j �|� �n |dkrj| j| _n�|dkr�| j �
t d dd�� | � � n�|d kr�| j �
t d d
d�� | j
d d d
d7 < | j| _n�|dv �r| j �
t d dd�� | j
d d d
|7 < | j| _nL|tu �rB| j �
t d dd�� | j| _n"| j
d d d
|7 < | j| _dS )NTr� rM r� r~ r' z.expected-attribute-value-but-got-right-bracketr( ri rj r* rP r r7 )rS rN �`z"equals-in-unquoted-attribute-valuez$expected-attribute-value-but-got-eof)r r? r
rn � attributeValueDoubleQuotedStater �attributeValueUnQuotedStaterF � attributeValueSingleQuotedStater+ r@ r rh r r r rz r% r% r& r� � sF
�
�
�
�
z'HTMLTokenizer.beforeAttributeValueStatec C s� | j �� }|dkr | j| _n�|dkr0| �d� n�|dkrj| j�td dd�� | jd d d d
7 < nN|t u r�| j�td d d�� | j
| _n&| jd d d || j � d � 7 < d
S )Nr� rM ri r' rj r( r* rP r r7 z#eof-in-attribute-value-double-quote)r� rM ri T� r r? �afterAttributeValueStater r^ r+ r@ r r r r rn rz r% r% r& r� � s&
� �
�z-HTMLTokenizer.attributeValueDoubleQuotedStatec C s� | j �� }|dkr | j| _n�|dkr0| �d� n�|dkrj| j�td dd�� | jd d d d
7 < nN|t u r�| j�td d d�� | j
| _n&| jd d d || j � d � 7 < d
S )Nr� rM ri r' rj r( r* rP r r7 z#eof-in-attribute-value-single-quote)r� rM ri Tr� rz r% r% r& r� s&
� �
�z-HTMLTokenizer.attributeValueSingleQuotedStatec C s | j �� }|tv r | j| _n�|dkr0| �d� n�|dkrB| �� n�|dv r|| j�t d dd�� | j
d d d |7 < n�|d
kr�| j�t d d d�� | j
d d d d 7 < nV|t u r�| j�t d d
d�� | j | _n.| j
d d d || j �
td�tB � 7 < dS )NrM r~ )r� r� rS rN r� r' z0unexpected-character-in-unquoted-attribute-valuer( r* rP r ri rj r7 z eof-in-attribute-value-no-quotes)rM r~ r� r� rS rN r� ri T)r r? r
r� r r^ rh r+ r@ r r r r rn rC rz r% r% r& r� s4
� � �
�
z)HTMLTokenizer.attributeValueUnQuotedStatec C s� | j �� }|tv r | j| _n�|dkr.| �� np|dkr@| j| _n^|tu rt| j� t
d dd�� | j � |� | j | _n*| j� t
d dd�� | j � |� | j| _dS )Nr~ r} r' z$unexpected-EOF-after-attribute-valuer( z*unexpected-character-after-attribute-valueT)
r r? r
r� r rh r� r r+ r@ r rF r rz r% r% r& r� . s&
�
� z&HTMLTokenizer.afterAttributeValueStatec C s� | j �� }|dkr&d| jd< | �� n^|tu rZ| j�td dd�� | j �|� | j | _
n*| j�td dd�� | j �|� | j | _
dS )Nr~ Trb r' z#unexpected-EOF-after-solidus-in-tagr( z)unexpected-character-after-solidus-in-tag) r r? r rh r r+ r@ r rF r r r� rz r% r% r& r� B s
�
� z&HTMLTokenizer.selfClosingStartTagStatec C sD | j �d�}|�dd�}| j�td |d�� | j �� | j| _dS )Nr~ ri r7 �Commentr( T) r rn �replacer+ r@ r r? r r rz r% r% r& r� T s �
z HTMLTokenizer.bogusCommentStatec C s� | j �� g}|d dkrR|�| j �� � |d dkrPtd dd�| _| j| _dS n�|d dv r�d}dD ](}|�| j �� � |d |vrfd } q�qf|r�td
dd d dd �| _| j| _dS n�|d d k�rD| jd u�rD| jj j
�rD| jj j
d j | jj j k�rDd}d
D ].}|�| j �� � |d |k�rd } �q2�q|�rD| j
| _dS | j�td dd�� |�rt| j �|�� � �qZ| j| _dS )NrP r� r� r3 r( T)�d�D))�o�O�rJ �C��t�T��y�Y��p�P��e�EFZDoctype)r) r_ �publicId�systemId�correct�[)r� r� �Ar� r� r� r' zexpected-dashes-or-doctype)r r? r@ r r �commentStartStater � doctypeStater ZtreeZ openElements� namespaceZdefaultNamespace�cdataSectionStater+ rF r- r� )r! rI �matched�expectedr% r% r& r� c sZ ����
�z(HTMLTokenizer.markupDeclarationOpenStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� | j�| j� | j| _nP|t u r�| j�td d
d�� | j�| j� | j| _n| jd |7 < | j
| _d S ) Nr� ri r' rj r( r* r7 r~ �incorrect-comment�eof-in-commentT) r r? �commentStartDashStater r+ r@ r r r r � commentStaterz r% r% r& r� � s.
� �
�
z HTMLTokenizer.commentStartStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� | j�| j� | j| _nT|t u r�| j�td d
d�� | j�| j� | j| _n | jd d| 7 < | j
| _d S ) Nr� ri r' rj r( r* � -�r~ r� r� T) r r? �commentEndStater r+ r@ r r r r r� rz r% r% r& r� � s.
� �
�
z#HTMLTokenizer.commentStartDashStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < nT|tu r�| j�td dd�� | j�| j� | j | _n | jd || j �
d � 7 < d
S ) Nr� ri r' rj r( r* r7 r� )r� ri T) r r? �commentEndDashStater r+ r@ r r r r rn rz r% r% r& r� � s$
� �
�zHTMLTokenizer.commentStatec C s� | j �� }|dkr | j| _n�|dkrV| j�td dd�� | jd d7 < | j| _nT|t u r�| j�td dd�� | j�| j� | j
| _n | jd d| 7 < | j| _d S )
Nr� ri r' rj r( r* r� zeof-in-comment-end-dashT) r r? r� r r+ r@ r r r� r r rz r% r% r& r� � s$
�
�
z!HTMLTokenizer.commentEndDashStatec C s, | j �� }|dkr*| j�| j� | j| _n�|dkrd| j�td dd�� | jd d7 < | j| _n�|dkr�| j�td d d�� | j | _n�|d
kr�| j�td d d�� | jd |7 < nj|t
u r�| j�td d d�� | j�| j� | j| _n4| j�td d
d�� | jd d| 7 < | j| _dS )Nr~ ri r' rj r( r* u --�r| z,unexpected-bang-after-double-dash-in-commentr� z,unexpected-dash-after-double-dash-in-commentzeof-in-comment-double-dashzunexpected-char-in-commentz--T) r r? r+ r@ r r r r r� �commentEndBangStater rz r% r% r& r� � s@
�
�
� �
�z HTMLTokenizer.commentEndStatec C s� | j �� }|dkr*| j�| j� | j| _n�|dkrN| jd d7 < | j| _n�|dkr�| j�td dd�� | jd d 7 < | j | _nT|t
u r�| j�td d
d�� | j�| j� | j| _n | jd d| 7 < | j | _d S ) Nr~ r� r* z--!ri r' rj r( u --!�z eof-in-comment-end-bang-stateT) r r? r+ r@ r r r r� r r� r rz r% r% r& r� s,
�
�
z!HTMLTokenizer.commentEndBangStatec C s� | j �� }|tv r | j| _nj|tu r\| j�td dd�� d| j d< | j�| j � | j
| _n*| j�td dd�� | j � |� | j| _dS )Nr' �!expected-doctype-name-but-got-eofr( Fr� zneed-space-after-doctypeT) r r? r
�beforeDoctypeNameStater r r+ r@ r r r rF rz r% r% r& r� s
�
� zHTMLTokenizer.doctypeStatec C s� | j �� }|tv rn�|dkrT| j�td dd�� d| jd< | j�| j� | j| _n�|dkr�| j�td dd�� d | jd
< | j | _nR|t
u r�| j�td d d�� d| jd< | j�| j� | j| _n|| jd
< | j | _d S )
Nr~ r' z+expected-doctype-name-but-got-right-bracketr( Fr� ri rj r7 r_ r� T) r r? r
r+ r@ r r r r �doctypeNameStater rz r% r% r& r� * s4
�
�
�
z$HTMLTokenizer.beforeDoctypeNameStatec C s | j �� }|tv r2| jd �t�| jd< | j| _n�|dkrh| jd �t�| jd< | j� | j� | j
| _n�|dkr�| j� t d dd�� | jd d7 < | j | _nh|t
u r�| j� t d dd�� d | jd
< | jd �t�| jd< | j� | j� | j
| _n| jd |7 < d S ) Nr_ r~ ri r' rj r( r7 zeof-in-doctype-nameFr� T)r r? r
r rc r
�afterDoctypeNameStater r+ r@ r r r� r rz r% r% r& r� D s0
�
�
z HTMLTokenizer.doctypeNameStatec C sH | j �� }|tv r�n.|dkr8| j�| j� | j| _�n |tu r�d| jd< | j � |� | j�t
d dd�� | j�| j� | j| _n�|dv r�d}d D ] }| j �� }||vr�d} q�q�|r�| j | _dS nD|d
v �r
d}d D ] }| j �� }||vr�d} q�q�|�r
| j | _dS | j � |� | j�t
d d d
|id�� d| jd< | j
| _dS )Nr~ Fr� r' �eof-in-doctyper( r� T))�u�U)�b�B)�l�L)�i�Ir� ��s�S)r� r� r� r� )�m�Mz*expected-space-or-right-bracket-in-doctyper* r5 )r r? r
r+ r@ r r r r rF r � afterDoctypePublicKeywordState� afterDoctypeSystemKeywordState�bogusDoctypeState)r! r* r� r� r% r% r& r� ] sT
�
�
z#HTMLTokenizer.afterDoctypeNameStatec C s� | j �� }|tv r | j| _n�|dv rP| j�td dd�� | j �|� | j| _nT|t u r�| j�td dd�� d| j
d< | j�| j
� | j | _n| j �|� | j| _dS � N)r� r� r' �unexpected-char-in-doctyper( r� Fr� T) r r? r
�"beforeDoctypePublicIdentifierStater r+ r@ r rF r r r rz r% r% r& r� � s&
�
�
z,HTMLTokenizer.afterDoctypePublicKeywordStatec C s� | j �� }|tv rn�|dkr0d| jd< | j| _n�|dkrLd| jd< | j| _n�|dkr�| j�t d dd�� d | jd
< | j�| j� | j
| _nh|t u r�| j�t d d d�� d | jd
< | j�| j� | j
| _n(| j�t d d d�� d | jd
< | j | _d
S )Nr� r3 r� r� r~ r' �unexpected-end-of-doctyper( Fr� r� r� T)
r r? r
r �(doctypePublicIdentifierDoubleQuotedStater �(doctypePublicIdentifierSingleQuotedStater+ r@ r r r r� rz r% r% r& r� � s:
�
�
�
z0HTMLTokenizer.beforeDoctypePublicIdentifierStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d
| jd < | j�| j� | j| _nR|t u r�| j�td d d�� d
| jd < | j�| j� | j| _n| jd |7 < d
S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� T�
r r? �!afterDoctypePublicIdentifierStater r+ r@ r r r r rz r% r% r& r� � s0
� �
�
z6HTMLTokenizer.doctypePublicIdentifierDoubleQuotedStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d
| jd < | j�| j� | j| _nR|t u r�| j�td d d�� d
| jd < | j�| j� | j| _n| jd |7 < d
S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� Tr� rz r% r% r& r� � s0
� �
�
z6HTMLTokenizer.doctypePublicIdentifierSingleQuotedStatec C s | j �� }|tv r | j| _n�|dkr<| j�| j� | j| _n�|dkrn| j�t d dd�� d| jd< | j
| _n�|dkr�| j�t d dd�� d| jd< | j | _nh|t u r�| j�t d d d�� d
| jd < | j�| j� | j| _n(| j�t d dd�� d
| jd < | j
| _d S )
Nr~ r� r' r� r( r3 r� r� r� Fr� T)r r? r
�-betweenDoctypePublicAndSystemIdentifiersStater r+ r@ r r r �(doctypeSystemIdentifierDoubleQuotedState�(doctypeSystemIdentifierSingleQuotedStater r� rz r% r% r& r� � s>
�
�
�
�
z/HTMLTokenizer.afterDoctypePublicIdentifierStatec C s� | j �� }|tv rn�|dkr4| j�| j� | j| _n�|dkrPd| jd< | j| _n�|dkrld| jd< | j | _nh|t
kr�| j�t d dd�� d | jd
< | j�| j� | j| _n(| j�t d d d�� d | jd
< | j | _d S )
Nr~ r� r3 r� r� r' r� r( Fr� r� T)
r r? r
r+ r@ r r r r� r� r r r� rz r% r% r& r� s2
�
�
z;HTMLTokenizer.betweenDoctypePublicAndSystemIdentifiersStatec C s� | j �� }|tv r | j| _n�|dv rP| j�td dd�� | j �|� | j| _nT|t u r�| j�td dd�� d| j
d< | j�| j
� | j | _n| j �|� | j| _dS r� ) r r? r
�"beforeDoctypeSystemIdentifierStater r+ r@ r rF r r r rz r% r% r& r� ) s&
�
�
z,HTMLTokenizer.afterDoctypeSystemKeywordStatec C s� | j �� }|tv rn�|dkr0d| jd< | j| _n�|dkrLd| jd< | j| _n�|dkr�| j�t d dd�� d | jd
< | j�| j� | j
| _nh|t u r�| j�t d d d�� d | jd
< | j�| j� | j
| _n(| j�t d dd�� d | jd
< | j | _d S )
Nr� r3 r� r� r~ r' r� r( Fr� r� T)
r r? r
r r� r r� r+ r@ r r r r� rz r% r% r& r� = s:
�
�
�
z0HTMLTokenizer.beforeDoctypeSystemIdentifierStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d
| jd < | j�| j� | j| _nR|t u r�| j�td d d�� d
| jd < | j�| j� | j| _n| jd |7 < d
S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� T�
r r? �!afterDoctypeSystemIdentifierStater r+ r@ r r r r rz r% r% r& r� Z s0
� �
�
z6HTMLTokenizer.doctypeSystemIdentifierDoubleQuotedStatec C s� | j �� }|dkr | j| _n�|dkrN| j�td dd�� | jd d7 < n�|dkr�| j�td d d�� d
| jd < | j�| j� | j| _nR|t u r�| j�td d d�� d
| jd < | j�| j� | j| _n| jd |7 < d
S )Nr� ri r' rj r( r� r7 r~ r� Fr� r� Tr� rz r% r% r& r� r s0
� �
�
z6HTMLTokenizer.doctypeSystemIdentifierSingleQuotedStatec C s� | j �� }|tv rn~|dkr4| j�| j� | j| _n^|tu rt| j�t d dd�� d| jd< | j�| j� | j| _n | j�t d dd�� | j
| _dS ) Nr~ r' r� r( Fr� r� T) r r? r
r+ r@ r r r r r r� rz r% r% r& r� � s$
�
�z/HTMLTokenizer.afterDoctypeSystemIdentifierStatec C sZ | j �� }|dkr*| j�| j� | j| _n,|tu rV| j �|� | j�| j� | j| _n dS )Nr~ T) r r? r+ r@ r r r r rF rz r% r% r& r� � s
z HTMLTokenizer.bogusDoctypeStatec C s� g }|� | j�d�� |� | j�d�� | j�� }|tkr>q�q|dksJJ �|d dd � dkrv|d d d� |d< q�q|� |� qd�|�}|�d�}|dkr�t|�D ]}| j� t d d
d �� q�|�
dd �}|r�| j� t d
|d �� | j | _ dS )N�]r~ rP ���z]]r3 ri r r' rj r( r7 rU T)
r@ r rn r? r rB �count�ranger+ r r� r r )r! r* r? Z nullCountr� r% r% r& r� � s2
�
�z HTMLTokenizer.cdataSectionState)N)NF)N�__name__�
__module__� __qualname__�__doc__r r0 rL r] r^ rh r rl rt rr rw ry r{ rm r� r� rs r� r� rv r� r� rx r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� �
__classcell__r% r% r# r& r s� H
P #
6 "- 3 r N) Z
__future__r r r Zpip._vendor.sixr rD � collectionsr r �sysr Z constantsr
r r r
r r r r r r Z _inputstreamr Z_trier rV �dictrd �objectr r% r% r% r& �<module> s Back to Directory
nL+D550H?Mx ,D"v]qv;6*Zqn)ZP 0 !1 A "#a$2Qr D8 a Ri[f\mIykIw0 cuFcRı?lO7к_f˓[C$殷WF<_W ԣs KcëIzyQy /_LKℂ;C",pFA: /]=H ~,ls/9ć:[=/#f;)x {ٛEQ )~ =𘙲r*2~ a _V=' kumF D}KYYC)({ *g&f`툪ry`=^cJ.I](*`wq 1dđ#̩͑0;H]u搂@:~וKL Nsh}OIR*8:2 !lDJVo( 3=M(zȰ+i*NA r6KnSl )!JJӁ* %݉?|D}d5:eP0R;{$X'xF@.ÊB {,WJuQɲRI;9QE琯62fT.DUJ;*cP A\ILNj!J۱+O\ ͔]ޒ SJȧc%ANolՎprULZԛerE2=XDXgVQeӓk yP7U*omQIs,K`)6\G3t?pgjrmۛجwluGtf h9uyP0D;Uڽ"OXlif$)&|ML0Zrm1[HXPlPR0'G=i2N +0e2]] 9VTPO7h(F*癈 '=QVZDF,d߬~TX
G[`le69CR(!S2!P
<0x < !1AQ "Raq02Br#SCTb
?Ζ"]mH5WR7k.ۛ!}Q~+yԏz|@T20S~Kek *zFf^2X*(@8r?CIuI|֓>^ExLgNUY+{.RѪ
τVYTD I62'8Y2 7'\TP.6d&˦@Vqi|8- OΕ]ʔ U=TL8=;6c| !qfF3aů&~$l}'NWUs$Uk^SV:U# 6w++s&r+nڐ{@29 gL
u"TÙM=6 (^"7r} =6YݾlCuhquympǦ
G jhsǜNlɻ}o7 #S6aw4!OSrD57%|?x>L
|/nD6?/8w#[)L7+6〼T ATg ! %5MmZ/c-{1_Je"|^$'O&ޱմ Trb$ w)R$&
N1EtdU3Uȉ1pM" N*(DN yd96.(jQ)X
5cQɎMyW? Q*!R>6=7)Xj5`J]e 8%t!+'!1Q 5 !1 AQaqё#2"0BRb ? Gt^## .llQT $v,,m 㵜5 ubV =sY+@d{N! dnO< .-B; _wJt6;QJd.Qc%p{ 1,sNDdFHI0ГoXшe黅XۢF:)[FGXƹ/w_cMeD,ʡcc. WDtA$j@:) -# u
c1<@ۗ9F)KJ-hpP]_x[qBlbp ʖw q"LFGd ƶ*s+ډ_Zc"? %t[IP 6J]#=ɺVvvCGsGh1 >)6|ey?Lӣm,4GWUi` ]uJVoV DG< SB 6ϏQ@ TiUly OU0kfV~~}SZ@*WUUi##; s/[ =!7}" WN]'(L! ~y5g9T̅JkbM' +s:S + B)v@Mj
e Cf jE 0Y\QnzG1д~Wo{T 9?`Rmyhsy3!HAD]mc1~2L Su7xT;j$`}4->L#vzŏ ILS ֭ T { rjGKC; bpU= -`BsK.SFw4Mq]ZdHS0)tLg