Dashboard
PHP:
8.2.30
OS:
Linux
User:
dh_nv8b2m
/
/
usr
/
lib
/
python3
/
dist-packages
/
jinja2
/
__pycache__
📤 Upload
📝 New File
📁 New Folder
Close
Editing: lexer.cpython-310.pyc
o ��g-u � @ sp U d Z ddlZddlZddlmZ zddlmZ W n ey( ddl mZ Y nw ddl m Z ddl m Z ddlmZ dd lmZ ejrNddlZdd lmZ ed �Zejejd f ed < e� d�Z e� d�Z e� dej �Z e� dej!ej"B �Z#e� dej!ej"B �Z$e d�Z%e d�Z&e d�Z'e d�Z(e d�Z)e d�Z*e d�Z+e d�Z,e d�Z-e d �Z.e d �Z/e d �Z0e d �Z1e d �Z2e d!�Z3e d"�Z4e d#�Z5e d$�Z6e d%�Z7e d&�Z8e d'�Z9e d(�Z:e d)�Z;e d*�Z<e d+�Z=e d,�Z>e d-�Z?e d.�Z@e d/�ZAe d0�ZBe d1�ZCe d2�ZDe d3�ZEe d4�ZFe d5�ZGe d6�ZHe d7�ZIe d8�ZJe d9�ZKe d:�ZLe d;�ZMe d<�ZNe d=�ZOe d>�ZPe d?�ZQe d@�ZRe dA�ZSe dB�ZTe dC�ZUi dDe%�dEe=�dFe)�dGe,�dHe5�dIe4�dJe8�dKe>�dLe0�dMe:�dNe1�dOe;�dPe/�dQe9�dRe+�dSe6�dTe-�e.e2e3e&e*e'e7e(e<dU� �ZVdVdW� eV�W� D �ZXeYeV�eYeX�k�s�J dX��e� dNdY�ZdZd[� e[eVd\d]� d^�D ��� dO��Z\e]eKeMeLe?ePeQeRg�Z^e]e?eSeMeRg�Z_d_e`d`e`fdadb�Zadcddd`e`fdedf�Zbdge`d`e`fdhdi�Zcdje`d`edfdkdl�Zedmdnd`ejfeje`e`f fdodp�ZgG dqdr� dr�ZhG dsdd� ddeji�ZjG dtdu� du�ZkG dvdw� dw�Zlddxdy�ZmG dzd{� d{en�ZoG d|d}� d}eji�ZpG d~d � d �ZqdS )�z�Implements a Jinja / Python combination lexer. The ``Lexer`` class is used to do some preprocessing. It filters out invalid operators like the bitshift operators we don't allow in templates. It separates template code and python code in expressions. � N)� literal_eval)�deque)�intern� )�pattern)�TemplateSyntaxError)�LRUCache)� Environment�2 �Lexer� _lexer_cachez\s+z (\r\n|\r|\n)z7('([^'\\]*(?:\\.[^'\\]*)*)'|"([^"\\]*(?:\\.[^"\\]*)*)")z� ( 0b(_?[0-1])+ # binary | 0o(_?[0-7])+ # octal | 0x(_?[\da-f])+ # hex | [1-9](_?\d)* # decimal | 0(_?0)* # decimal zero ) z� (?<!\.) # doesn't start with a . (\d+_)*\d+ # digits, possibly _ separated ( (\.(\d+_)*\d+)? # optional fractional part e[+\-]?(\d+_)*\d+ # exponent part | \.(\d+_)*\d+ # required fractional part ) �add�assign�colon�comma�div�dot�eq�floordiv�gt�gteq�lbrace�lbracket�lparen�lt�lteq�mod�mul�ne�pipe�pow�rbrace�rbracket�rparen� semicolon�sub�tilde� whitespace�float�integer�name�string�operator� block_begin� block_end�variable_begin� variable_end� raw_begin�raw_end� comment_begin� comment_end�comment�linestatement_begin�linestatement_end�linecomment_begin�linecomment_end� linecomment�data�initial�eof�+�-�/z//�*�%z**�~�[�]�(�)�{�}z==z!=�>) z>=�<z<=�=�.�:�|�,�;c C s i | ]\}}||�qS � rR )�.0�k�vrR rR �./usr/lib/python3/dist-packages/jinja2/lexer.py� <dictcomp>� s rW zoperators droppedrO c c s � | ]}t �|�V qd S �N)�re�escape�rS �xrR rR rV � <genexpr>� � � r] c C s t | � S rX )�len�r\ rR rR rV �<lambda>� s ra )�key� token_type�returnc C sL | t v rt | S tdtdtdtdtdtdtdtdt dt d t d t d i � | | �S ) Nzbegin of commentzend of commentr5 zbegin of statement blockzend of statement blockzbegin of print statementzend of print statementzbegin of line statementzend of line statementztemplate data / textzend of template)�reverse_operators�TOKEN_COMMENT_BEGIN�TOKEN_COMMENT_END� TOKEN_COMMENT�TOKEN_LINECOMMENT�TOKEN_BLOCK_BEGIN�TOKEN_BLOCK_END�TOKEN_VARIABLE_BEGIN�TOKEN_VARIABLE_END�TOKEN_LINESTATEMENT_BEGIN�TOKEN_LINESTATEMENT_END� TOKEN_DATA� TOKEN_EOF�get)rc rR rR rV �_describe_token_type� s" � �rs �token�Tokenc C s | j tkr| jS t| j �S )z#Returns a description of the token.)�type� TOKEN_NAME�valuers )rt rR rR rV �describe_token� s ry �exprc C s8 d| v r| � dd�\}}|tkr|S t|�S | }t|�S )z0Like `describe_token` but for token expressions.rN r )�splitrw rs )rz rv rx rR rR rV �describe_token_expr� s �r| rx c C s t t�| ��S )zsCount the number of newline characters in the string. This is useful for extensions that filter a stream. )r_ � newline_re�findall)rx rR rR rV �count_newlines� � r � environmentr c C s� t j}t| j�t|| j�ft| j�t|| j�ft| j�t|| j�fg}| j dur8|� t| j �t d|| j � f� | j durM|� t| j �t d|| j � f� dd� t|dd�D �S )zACompiles all the rules from the environment into a list of rules.Nz ^[ \t\v]*z(?:^|(?<=\S))[^\S\r\n]*c S s g | ]}|d d� �qS )r NrR r[ rR rR rV � <listcomp>� s z!compile_rules.<locals>.<listcomp>T)�reverse)rY rZ r_ �comment_start_stringrf �block_start_stringrj �variable_start_stringrl �line_statement_prefix�appendrn �line_comment_prefix�TOKEN_LINECOMMENT_BEGIN�sorted)r� �e�rulesrR rR rV � compile_rules� s: ���� �� ��r� c @ sF e Zd ZdZefdedeje ddfdd�Zde d edd fd d �Z dS ) �FailurezjClass that raises a `TemplateSyntaxError` if called. Used by the `Lexer` to specify known errors. �message�clsrd Nc C s || _ || _d S rX )r� � error_class)�selfr� r� rR rR rV �__init__ s zFailure.__init__�lineno�filenamez te.NoReturnc C s | � | j||��rX )r� r� )r� r� r� rR rR rV �__call__ s zFailure.__call__) �__name__� __module__� __qualname__�__doc__r �str�t�Typer� �intr� rR rR rR rV r� � s ��� �r� c @ sX e Zd ZU eed< eed< eed< defdd�Zdedefdd �Zd edefd d �Z d S )ru r� rv rx rd c C � t | �S rX )ry �r� rR rR rV �__str__ � z Token.__str__rz c C s2 | j |krdS d|v r|�dd�| j | jgkS dS )z�Test a token against a token expression. This can either be a token type or ``'token_type:token_value'``. This can only test against string values and types. TrN r F)rv r{ rx �r� rz rR rR rV �test s z Token.test�iterablec s t � fdd�|D ��S )z(Test against multiple token expressions.c 3 s � | ]}� � |�V qd S rX )r� )rS rz r� rR rV r] ' r^ z!Token.test_any.<locals>.<genexpr>)�any)r� r� rR r� rV �test_any% s zToken.test_anyN) r� r� r� r� �__annotations__r� r� �boolr� r� rR rR rR rV ru s c @ s2 e Zd ZdZd dd�Zd dd �Zdefd d �ZdS )�TokenStreamIteratorz`The iterator for tokenstreams. Iterate over the stream until the eof token is reached. �stream� TokenStreamrd Nc C s || _ d S rX )r� )r� r� rR rR rV r� / s z TokenStreamIterator.__init__c C s | S rX rR r� rR rR rV �__iter__2 s z TokenStreamIterator.__iter__c C s. | j j}|jtu r| j �� t�t| j � |S rX )r� �currentrv rq �close� StopIteration�next�r� rt rR rR rV �__next__5 s z TokenStreamIterator.__next__)r� r� rd N)rd r� )r� r� r� r� r� r� ru r� rR rR rR rV r� * s r� c @ s� e Zd ZdZdeje deje deje fdd�Z de fdd �Z de fd d �Z ede fd d ��Zdeddfdd�Zdefdd�Zd#deddfdd�Zdedeje fdd�Zdede fdd �Zdefd d �Zd$d d �Zdedefd!d"�ZdS )%r� z�A token stream is an iterable that yields :class:`Token`\s. The parser however does not iterate over it but calls :meth:`next` to go one token ahead. The current active token is stored as :attr:`current`. � generatorr* r� c C s> t |�| _t� | _|| _|| _d| _tdtd�| _ t | � d S )NFr � ) �iter�_iterr �_pushedr* r� �closedru � TOKEN_INITIALr� r� )r� r� r* r� rR rR rV r� F s zTokenStream.__init__rd c C r� rX )r� r� rR rR rV r� T r� zTokenStream.__iter__c C s t | j�p | jjtuS rX )r� r� r� rv rq r� rR rR rV �__bool__W s zTokenStream.__bool__c C s | S )z Are we at the end of the stream?rR r� rR rR rV �eosZ s zTokenStream.eosrt Nc C s | j �|� dS )z Push a token back to the stream.N)r� r� r� rR rR rV �push_ s zTokenStream.pushc C s" t | �}| j}| �|� || _|S )zLook at the next token.)r� r� r� )r� � old_token�resultrR rR rV �lookc s zTokenStream.lookr �nc C s t |�D ]}t| � qdS )zGot n tokens ahead.N)�ranger� )r� r� �_rR rR rV �skipk s �zTokenStream.skiprz c C s | j �|�r t| �S dS )zqPerform the token test and return the token if it matched. Otherwise the return value is `None`. N)r� r� r� r� rR rR rV �next_ifp s zTokenStream.next_ifc C s | � |�duS )z8Like :meth:`next_if` but only returns `True` or `False`.N)r� r� rR rR rV �skip_ify s zTokenStream.skip_ifc C s\ | j }| jr| j�� | _ |S | j jtur,z t| j�| _ W |S ty+ | �� Y |S w |S )z|Go one token ahead and return the old one. Use the built-in :func:`next` instead of calling this directly. ) r� r� �popleftrv rq r� r� r� r� )r� �rvrR rR rV r� } s � � �zTokenStream.__next__c C s& t | jjtd�| _td�| _d| _dS )zClose the stream.r� rR TN)ru r� r� rq r� r� r� r� rR rR rV r� � s zTokenStream.closec C sn | j �|�s3t|�}| j jtu r td|�d�| j j| j| j��td|�dt | j ���| j j| j| j��t | �S )z}Expect a given token type and return it. This accepts the same argument as :meth:`jinja2.lexer.Token.test`. z%unexpected end of template, expected rM zexpected token z, got ) r� r� r| rv rq r r� r* r� ry r� r� rR rR rV �expect� s ��zTokenStream.expect)r )rd N)r� r� r� r� r� �Iterableru �Optionalr� r� r� r� r� r� �propertyr� r� r� r� r� r� r� r� r� r� rR rR rR rV r� @ s* �� � r� c C sZ | j | j| j| j| j| j| j| j| j| j | j | j f }t � |�}|du r+t| � t |< }|S )z(Return a lexer which is probably cached.N)r� �block_end_stringr� �variable_end_stringr� �comment_end_stringr� r� � trim_blocks� lstrip_blocks�newline_sequence�keep_trailing_newliner rr r )r� rb �lexerrR rR rV � get_lexer� s" � r� c s$ e Zd ZdZdZ� fdd�Z� ZS )�OptionalLStripzWA special tuple for marking a point in the state that can have lstrip applied. rR c s t � �| |�S rX )�super�__new__)r� �members�kwargs�� __class__rR rV r� � s zOptionalLStrip.__new__)r� r� r� r� � __slots__r� � __classcell__rR rR r� rV r� � s r� c @ sN e Zd ZU eje ed< ejeejedf eje f ed< ej e ed< dS )�_Ruler .�tokens�commandN) r� r� r� r� �Patternr� r� �Union�Tupler� r� rR rR rR rV r� � s $r� c @ s� e Zd ZdZddd�Zdedefd d �Z dd ed eje d eje deje de f dd�Z ddej ej e eef d eje d eje deje fdd�Z dd ed eje d eje deje dejej e eef f dd�ZdS )r a Class that implements a lexer for a given environment. Automatically created by the environment class, usually you don't have to do that. Note that the lexer is not automatically bound to an environment. Multiple environments can share the same lexer. r� r rd Nc C s2 t j}dtdtjt fdd�}tttd �ttt d �tt t d �tt t d �tttd �tttd �g}t|�}||j�}||j�}||j�}||j�} |jrLdnd} |jrU|d�nd | _|j| _|j| _d|� d |� d |� d �} d � | gd d� |D � �} dt|d| � d��t t d�d�t|d�t d �gt t|d|� d|� d |� | � d ��t t!fd�t|d�t"d�fd �gt#t|d|� d|� d |� | � d��t$d�g| t%t|d| � d | � ��t&d�g| t't|d|� d |� d|� d |� | � d � �t t t(�d�t|d�t"d �fd �gt)t|d �t*d�g| t+t|d �t,t-fd�gi| _.d S ) Nr\ rd c S s t �| t jt jB �S rX )rY �compile�M�Sr` rR rR rV �c� s zLexer.__init__.<locals>.cz\n?r� z[^ \t]z(?P<raw_begin>z(\-|\+|)\s*raw\s*(?:\-z\s*|z))rO c S s" g | ] \}}d |� d|� d��qS )z(?P<rJ z (\-|\+|))rR )rS r� �rrR rR rV r� s " z"Lexer.__init__.<locals>.<listcomp>�rootz(.*?)(?:rG �#bygroupz.+z (.*?)((?:\+z|\-�#popz(.)zMissing end of comment tagz(?:\+z\-z (.*?)((?:z(\-|\+|))\s*endraw\s*(?:\+z Missing end of raw directivez \s*(\n|$)z(.*?)()(?=\n|$))/rY rZ r� r� r� r� � whitespace_re�TOKEN_WHITESPACE�float_re� TOKEN_FLOAT� integer_re� TOKEN_INTEGER�name_rerw � string_re� TOKEN_STRING� operator_re�TOKEN_OPERATORr� r� r� r� r� r� r� �lstrip_unless_rer� r� �joinr� rp rf rh rg r� rj rk rl rm �TOKEN_RAW_BEGIN� TOKEN_RAW_ENDrn ro r� ri �TOKEN_LINECOMMENT_ENDr� ) r� r� r� r� � tag_rules�root_tag_rules�block_start_re� block_end_re�comment_end_re�variable_end_re�block_suffix_re� root_raw_re� root_parts_rerR rR rV r� � s� � ������ ����� ����� � ��� ������ � ���� �zLexer.__init__rx c C s t �| j|�S )z`Replace all newlines with the configured sequence in strings and template data. )r} r% r� )r� rx rR rR rV �_normalize_newlinesY r� zLexer._normalize_newlines�sourcer* r� �statec C s& | � ||||�}t| �|||�||�S )z:Calls tokeniter + tokenize and wraps it in a token stream.)� tokeniterr� �wrap)r� r r* r� r r� rR rR rV �tokenize_ s zLexer.tokenizer� c c sT � |D ]�\}}}|t v r q|}|tkrt}n�|tkr t}n�|ttfv r$q|tkr.| �|�}nr|dkr5|}nk|t krG|}|� � sFt d|||��nY|t kr}z| �|dd� �� dd��d�}W nB ty| } zt|��d�d �� } t | |||�|�d }~ww |tkr�t|�d d �d �}n|tkr�t|�d d ��}n|tkr�t| }t|||�V qd S ) z�This is called with the stream as returned by `tokenize` and wraps every token in a :class:`Token` and converts the value. �keywordz Invalid character in identifierr ����ascii�backslashreplacezunicode-escaperN Nr� r� r )�ignored_tokensrn rj ro rk r� r� rp r rw � isidentifierr r� �encode�decode� Exceptionr� r{ �stripr� r� �replacer� r r� � operatorsru ) r� r� r* r� r� rt � value_strrx r� �msgrR rR rV r j sR � ������z Lexer.wrapc c s* � t �|�ddd� }| js|d dkr|d= d�|�}d}d}dg}|dur:|dkr:|d v s3J d ��|�|d � | j|d } t|�} g } | j} d} d } | D �]�\}}}|�||�}|du rbqQ| rl|t t t fv rlqQt |t ��r6|�� }t |t�r�|d }td d� |ddd� D ��}|dkr�|�� }|t|�d� �d�} |g|dd� �}n/|dkr�| dur�|�� �t�s�|�d�d }|dks�|r�| �||�s�|d|� g|dd� �}t|�D ]W\}}|jtu r�|||��|dk�r|�� �� D ]\}}|du�r|||fV ||�d�7 } nq�t |�d���q�|| } | �s#|t v�r)||| fV || �d�| 7 }d} q�ni|� � } |t k�r�| dk�rJ| �d� n@| dk�rU| �d� n5| dk�r`| �d� n*| dv �r�| �sst d| � d�|||��| �!� } | | k�r�t d| � d | � d�|||��| �s�|t v�r�||| fV || �d�7 }|� � dd� dk}|�"� } |du�r�|d k�r�|�!� n*|dk�r�|�� �� D ]\}}|du�r�|�|� n �q�t |�d ���n|�|� | j|d } n | |k�r�t |�d ���| } n|| k�rdS t d || �d!|� �|||��qO)"a This method tokenizes the text and returns the tokens in a generator. Use this method if you just want to tokenize a template. .. versionchanged:: 3.0 Only ``\n``, ``\r\n`` and ``\r`` are treated as line breaks. N� r r� � r r r� )�variable�blockz invalid state�_beginTc s s � | ] }|d ur|V qd S rX rR )rS �grR rR rV r] � s � z"Lexer.tokeniter.<locals>.<genexpr>r? r> r� z= wanted to resolve the token dynamically but no group matchedrH rI rF rG rD rE )rI rG rE z unexpected '�'z ', expected 'r� zA wanted to resolve the new state dynamically but no group matchedz* yielded empty string without stack changezunexpected char z at )#r} r{ r� r� r� r� r_ r� �matchrm rk ro � isinstance�tuple�groupsr� r� �rstrip�count� groupdictrr rl �rfind�search� enumerater� r� �items� RuntimeError�ignore_if_empty�groupr� r �pop�end) r� r r* r� r �lines�posr� �stack� statetokens� source_length�balancing_stackr� �newlines_stripped� line_starting�regexr� � new_state�mr% �text� strip_sign�stripped�l_pos�idxrt rb rx r; � expected_op�pos2rR rR rV r � s� � � � ��� � � � ��� � � ��zLexer.tokeniter)r� r rd N)NNN)NN)r� r� r� r� r� r� r r� r� r� r r� r� r� �Iteratorru r r rR rR rR rV r � sT y ����� ����� �:������)r� r rd r )rr� rY �typingr� �astr �collections.abcr � ImportError� collections�sysr � _identifierr r� � exceptionsr �utilsr � TYPE_CHECKING�typing_extensions�ter� r r �MutableMappingr� r� r� r� r} r� r� � IGNORECASE�VERBOSEr� r� � TOKEN_ADD� TOKEN_ASSIGN� TOKEN_COLON� TOKEN_COMMA� TOKEN_DIV� TOKEN_DOT�TOKEN_EQ�TOKEN_FLOORDIV�TOKEN_GT� TOKEN_GTEQ� TOKEN_LBRACE�TOKEN_LBRACKET� TOKEN_LPAREN�TOKEN_LT� TOKEN_LTEQ� TOKEN_MOD� TOKEN_MUL�TOKEN_NE� TOKEN_PIPE� TOKEN_POW� TOKEN_RBRACE�TOKEN_RBRACKET� TOKEN_RPAREN�TOKEN_SEMICOLON� TOKEN_SUB� TOKEN_TILDEr� r� r� rw r� r� rj rk rl rm r� r� rf rg rh rn ro r� r� ri rp r� rq r r, re r_ r� r� r� � frozensetr r. r� rs ry r| r� r �Listr� r� � NamedTupleru r� r� r� r$ r� r� r rR rR rR rV �<module> s6 � � � ��������� � � � � ������ &��� � ") m
Save
Cancel