Dashboard
PHP:
8.2.30
OS:
Linux
User:
dh_nv8b2m
/
/
usr
/
lib
/
python3
/
dist-packages
/
awscli
/
customizations
/
s3
/
__pycache__
📤 Upload
📝 New File
📁 New Folder
Close
Editing: s3handler.cpython-310.pyc
o .&�a[ � @ s: d dl Z d dlZd dlmZ d dlmZmZmZmZm Z m Z d dl m Z d dl mZ d dl mZ d dl mZ d dl mZ d d l mZ d d l mZ d d l mZ d d l mZ d d l mZ d dl mZ d dl mZ d dl mZ d dl mZ d dl mZ d dl m Z d dl m Z d dlm Z d dlm Z d dlm Z d dlm!Z! d dlm"Z" d dlm#Z# d dlm$Z$ d d lm%Z% d d lm&Z& d d lm'Z' d d l(m)Z) e �*e+�Z,G d d!� d!e-�Z.G d"d#� d#e-�Z/G d$d%� d%e-�Z0G d&d'� d'e0�Z1G d(d)� d)e0�Z2G d*d+� d+e0�Z3G d,d-� d-e1�Z4G d.d/� d/e2�Z5G d0d1� d1e0�Z6G d2d3� d3e0�Z7dS )4� N)�TransferManager)�human_readable_size�MAX_UPLOAD_SIZE�find_bucket_key� relative_path�create_warning�NonSeekableStream)�*create_transfer_config_from_runtime_config)�UploadResultSubscriber)�DownloadResultSubscriber)�CopyResultSubscriber)� UploadStreamResultSubscriber)� DownloadStreamResultSubscriber)�DeleteResultSubscriber)� QueuedResult)� SuccessResult)� FailureResult)� DryRunResult)�ResultRecorder)� ResultPrinter)�OnlyShowErrorsResultPrinter)�NoProgressResultPrinter)�ResultProcessor)�CommandResultRecorder)�RequestParamsMapper��StdoutBytesWriter)�ProvideSizeSubscriber)�"ProvideUploadContentTypeSubscriber)� ProvideCopyContentTypeSubscriber)�!ProvideLastModifiedTimeSubscriber)�DirectoryCreatorSubscriber)�DeleteSourceFileSubscriber)� DeleteSourceObjectSubscriber)� DeleteCopySourceObjectSubscriber)�get_binary_stdinc @ s( e Zd ZdZdd� Zdd� Zdd� ZdS ) �S3TransferHandlerFactory� c C s || _ || _dS )a Factory for S3TransferHandlers :type cli_params: dict :param cli_params: The parameters provide to the CLI command :type runtime_config: RuntimeConfig :param runtime_config: The runtime config for the CLI command being run N)� _cli_params�_runtime_config)�self� cli_params�runtime_config� r- �D/usr/lib/python3/dist-packages/awscli/customizations/s3/s3handler.py�__init__: s z!S3TransferHandlerFactory.__init__c C sr t | j�}| j|_| j|_t||�}t�d|j|j � t � }|g}| � ||� t ||�}t |||�}t|| j|�S )ag Creates a S3TransferHandler instance :type client: botocore.client.Client :param client: The client to power the S3TransferHandler :type result_queue: queue.Queue :param result_queue: The result queue to be used to process results for the S3TransferHandler :returns: A S3TransferHandler instance z7Using a multipart threshold of %s and a part size of %s)r r) �MAX_IN_MEMORY_CHUNKS�max_in_memory_upload_chunks� max_in_memory_download_chunksr �LOGGER�debug�multipart_threshold�multipart_chunksizer �_add_result_printerr r �S3TransferHandlerr( ) r* �client� result_queue�transfer_config�transfer_manager�result_recorder�result_processor_handlers�result_processor�command_result_recorderr- r- r. �__call__G s0 �� � ���z!S3TransferHandlerFactory.__call__c C sh | j �d�rd S | j �d�rt|�}n| j �d�r t|�}n| j �d�s)t|�}nt|�}|�|� d S )N�quiet�only_show_errors� is_stream�progress)r( �getr r r �append)r* r= r> �result_printerr- r- r. r7 k s z,S3TransferHandlerFactory._add_result_printerN)�__name__� __module__� __qualname__r0 r/ rA r7 r- r- r- r. r& 7 s $r& c @ s e Zd Zdd� Zdd� ZdS )r8 c C sP || _ || _| j | jj|f}t|� t|� t|� t|� t|� t|� t |� g| _ dS )a Backend for performing S3 transfers :type transfer_manager: s3transfer.manager.TransferManager :param transfer_manager: Transfer manager to use for transfers :type cli_params: dict :param cli_params: The parameters passed to the CLI command in the form of a dictionary :type result_command_recorder: ResultCommandRecorder :param result_command_recorder: The result command recorder to be used to get the final result of the transfer N) �_transfer_manager�_result_command_recorderr: � UploadStreamRequestSubmitter� DownloadStreamRequestSubmitter�UploadRequestSubmitter�DownloadRequestSubmitter�CopyRequestSubmitter�DeleteRequestSubmitter�LocalDeleteRequestSubmitter� _submitters)r* r< r+ �result_command_recorder�submitter_argsr- r- r. r/ z s � �zS3TransferHandler.__init__c C s� | j �> | j�+ d}|D ]}| jD ]}|�|�r#|�|�r!|d7 } nqq | j �|� W d � n1 s5w Y W d � n1 sDw Y | j �� S )a� Process iterable of FileInfos for transfer :type fileinfos: iterable of FileInfos param fileinfos: Set of FileInfos to submit to underlying transfer request submitters to make transfer API calls to S3 :rtype: CommandResult :returns: The result of the command that specifies the number of failures and warnings encountered. r � N)rM rL rU � can_submit�submit�notify_total_submissions�get_command_result)r* � fileinfos�total_submissions�fileinfo� submitterr- r- r. �call� s$ ��� �� � zS3TransferHandler.callN)rI rJ rK r/ ra r- r- r- r. r8 y s %r8 c @ s� e Zd ZdZdZdd� Zdd� Zdd� Zdd � Zd d � Z d d � Z dd� Z dd� Z dd� Z dd� Zdd� Zdd� Zdd� Zd d � Zd d � ZdS ) � BaseTransferRequestSubmitterNc C s || _ || _|| _dS )a\ Submits transfer requests to the TransferManager Given a FileInfo object and provided CLI parameters, it will add the necessary extra arguments and subscribers in making a call to the TransferManager. :type transfer_manager: s3transfer.manager.TransferManager :param transfer_manager: The underlying transfer manager :type result_queue: queue.Queue :param result_queue: The result queue to use :type cli_params: dict :param cli_params: The associated CLI parameters passed in to the command as a dictionary. N)rL � _result_queuer( )r* r< r: r+ r- r- r. r/ � s z%BaseTransferRequestSubmitter.__init__c C s | � |�}|s | �|�S dS )a� Submits a transfer request based on the FileInfo provided There is no guarantee that the transfer request will be made on behalf of the fileinfo as a fileinfo may be skipped based on circumstances in which the transfer is not possible. :type fileinfo: awscli.customizations.s3.fileinfo.FileInfo :param fileinfo: The FileInfo to be used to submit a transfer request to the underlying transfer manager. :rtype: s3transfer.futures.TransferFuture :returns: A TransferFuture representing the transfer if it the transfer was submitted. If it was not submitted nothing is returned. N)�_warn_and_signal_if_skip� _do_submit)r* r_ � should_skipr- r- r. rZ � s �z#BaseTransferRequestSubmitter.submitc C � t d��)a� Checks whether it can submit a particular FileInfo :type fileinfo: awscli.customizations.s3.fileinfo.FileInfo :param fileinfo: The FileInfo to check if the transfer request submitter can handle. :returns: True if it can use the provided FileInfo to make a transfer request to the underlying transfer manager. False, otherwise. z can_submit()��NotImplementedError�r* r_ r- r- r. rY � s z'BaseTransferRequestSubmitter.can_submitc C s� i }| j r | � || j� g }| �||� | jr2d| ji}| j�dd�r'd|d< |�| jdi |��� | j�d�s?| �|||�S | �|� d S )Nr: �is_moveF�move� transfer_type�dryrunr- ) �REQUEST_MAPPER_METHODr( �_add_additional_subscribers�RESULT_SUBSCRIBER_CLASSrc rF rG �_submit_transfer_request�_submit_dryrun)r* r_ � extra_args� subscribers� result_kwargsr- r- r. re � s �z'BaseTransferRequestSubmitter._do_submitc C s@ |j }| j�dd�r d}| �|�\}}| j�t|||d�� d S )Nrk Frl �rm �src�dest)�operation_namer( rF �_format_src_destrc �putr )r* r_ rm rx ry r- r- r. rs s �z+BaseTransferRequestSubmitter._submit_dryrunc C � d S �Nr- �r* ru r_ r- r- r. rp � z8BaseTransferRequestSubmitter._add_additional_subscribersc C rg )Nz_submit_transfer_request()rh )r* r_ rt ru r- r- r. rr � z5BaseTransferRequestSubmitter._submit_transfer_requestc C s | � � D ] }||�r dS qd S )NT)�_get_warning_handlers)r* r_ �warning_handlerr- r- r. rd s ��z5BaseTransferRequestSubmitter._warn_and_signal_if_skipc C s g S r~ r- �r* r- r- r. r� s z2BaseTransferRequestSubmitter._get_warning_handlersc C s | j �d�o | j �d� S )N�guess_mime_type� content_type)r( rF r� r- r- r. �_should_inject_content_type$ s �z8BaseTransferRequestSubmitter._should_inject_content_typec C sf | j �d�s1|�� s1t�d|j|jf � | j �d�s/td|j d|j|jf �}| j� |� dS dS )N�force_glacier_transferz@Encountered glacier object s3://%s. Not performing %s on object.�ignore_glacier_warnings�s3://z�Object is of storage class GLACIER. Unable to perform %s operations on GLACIER objects. You must restore the object to be able to perform the operation. See aws s3 %s help for additional parameter options to ignore or force these transfers.TF) r( rF �is_glacier_compatibler3 r4 rx rz r rc r| )r* r_ �warningr- r- r. � _warn_glacier* s" �� �� z*BaseTransferRequestSubmitter._warn_glacierc C sD dt jj }t j�|j��|�}|r t|jd�}| j�|� dS dS )Nz..z#File references a parent directory.TF) �os�path�sep�normpath� compare_key� startswithr rc r| )r* r_ � parent_prefix� escapes_cwdr� r- r- r. �_warn_parent_reference? s �� z3BaseTransferRequestSubmitter._warn_parent_referencec C rg )zAReturns formatted versions of a fileinfos source and destination.r{ rh rj r- r- r. r{ L s z-BaseTransferRequestSubmitter._format_src_destc C s t |�S r~ )r �r* r� r- r- r. �_format_local_pathP r� z/BaseTransferRequestSubmitter._format_local_pathc C s |� d�r|S d| S )Nr� )r� r� r- r- r. �_format_s3_pathS s z,BaseTransferRequestSubmitter._format_s3_path)rI rJ rK ro rq r/ rZ rY re rs rp rr rd r� r� r� r� r{ r� r� r- r- r- r. rb � s$ rb c @ sN e Zd ZejZeZdd� Zdd� Z dd� Z dd� Z d d � Z d d � Z d d� ZdS )rP c C � |j dkS )N�upload�rz rj r- r- r. rY ] � z!UploadRequestSubmitter.can_submitc C sF |� t|j�� | �� r|� t� � | j�dd�r!|� t� � d S d S �Nrk F)rG r �sizer� r r( rF r" r r- r- r. rp ` s �z2UploadRequestSubmitter._add_additional_subscribersc C �. t |j�\}}| �|�}| jj|||||d�S �N)�fileobj�bucket�keyrt ru )r ry � _get_fileinrL r� )r* r_ rt ru r� r� �fileinr- r- r. rr g � �z/UploadRequestSubmitter._submit_transfer_requestc C � |j S r~ )rx rj r- r- r. r� o � z"UploadRequestSubmitter._get_fileinc C � | j gS r~ )�_warn_if_too_larger� r- r- r. r� r r� z,UploadRequestSubmitter._get_warning_handlersc C sT t |d�r&|jtkr(t|j�}d|tt�f }t||dd�}| j�|� d S d S d S )Nr� z&File %s exceeds s3 upload limit of %s.F)� skip_file) �getattrr� r r rx r r rc r| )r* r_ � file_path�warning_messager� r- r- r. r� u s ����z)UploadRequestSubmitter._warn_if_too_largec C � | � |j�}| �|j�}||fS r~ )r� rx r� ry �r* r_ rx ry r- r- r. r{ � z'UploadRequestSubmitter._format_src_destN)rI rJ rK r �map_put_object_paramsro r rq rY rp rr r� r� r� r{ r- r- r- r. rP Y s rP c @ sF e Zd ZejZeZdd� Zdd� Z dd� Z dd� Z d d � Z d d � Z d S )rQ c C r� )N�downloadr� rj r- r- r. rY � r� z#DownloadRequestSubmitter.can_submitc C sV |� t|j�� |� t� � |� t|j| j�� | j�dd�r)|� t |j �� d S d S r� ) rG r r� r! r � last_updaterc r( rF r# � source_clientr r- r- r. rp � s � ��z4DownloadRequestSubmitter._add_additional_subscribersc C r� r� )r rx � _get_fileoutrL r� )r* r_ rt ru r� r� �fileoutr- r- r. rr � r� z1DownloadRequestSubmitter._submit_transfer_requestc C r� r~ )ry rj r- r- r. r� � r� z%DownloadRequestSubmitter._get_fileoutc C s | j | jgS r~ )r� r� r� r- r- r. r� � s z.DownloadRequestSubmitter._get_warning_handlersc C r� r~ )r� rx r� ry r� r- r- r. r{ � r� z)DownloadRequestSubmitter._format_src_destN)rI rJ rK r �map_get_object_paramsro r rq rY rp rr r� r� r{ r- r- r- r. rQ � s rQ c @ s> e Zd ZejZeZdd� Zdd� Z dd� Z dd� Z d d � Z d S ) rR c C r� )N�copyr� rj r- r- r. rY � r� z CopyRequestSubmitter.can_submitc C sJ |� t|j�� | �� r|� t� � | j�dd�r#|� t|j�� d S d S r� ) rG r r� r� r r( rF r$ r� r r- r- r. rp � s ��z0CopyRequestSubmitter._add_additional_subscribersc C s@ t |j�\}}t |j�\}}||d�}| jj||||||jd�S )N)�Bucket�Key)r� r� � copy_sourcert ru r� )r ry rx rL r� r� ) r* r_ rt ru r� r� � source_bucket� source_keyr� r- r- r. rr � s �z-CopyRequestSubmitter._submit_transfer_requestc C r� r~ )r� r� r- r- r. r� � r� z*CopyRequestSubmitter._get_warning_handlersc C s | � |j�}| � |j�}||fS r~ )r� rx ry r� r- r- r. r{ � r� z%CopyRequestSubmitter._format_src_destN) rI rJ rK r �map_copy_object_paramsro r rq rY rp rr r� r{ r- r- r- r. rR � s rR c @ �0 e Zd ZeZdd� Zdd� Zdd� Zdd� Zd S ) rN c C � |j dko | j�d�S )Nr� rD �rz r( rF rj r- r- r. rY � � �z'UploadStreamRequestSubmitter.can_submitc C s0 | j �dd �}|d ur|�tt|��� d S d S )N� expected_size)r( rF rG r �int)r* ru r_ r� r- r- r. rp � s �z8UploadStreamRequestSubmitter._add_additional_subscribersc C s t � }t|�S r~ )r% r )r* r_ � binary_stdinr- r- r. r� � s z(UploadStreamRequestSubmitter._get_fileinc C � dS �N�-r- r� r- r- r. r� � r� z/UploadStreamRequestSubmitter._format_local_pathN) rI rJ rK r rq rY rp r� r� r- r- r- r. rN � s rN c @ r� ) rO c C r� )Nr� rD r� rj r- r- r. rY � r� z)DownloadStreamRequestSubmitter.can_submitc C r} r~ r- r r- r- r. rp � r� z:DownloadStreamRequestSubmitter._add_additional_subscribersc C s t � S r~ r rj r- r- r. r� � r� z+DownloadStreamRequestSubmitter._get_fileoutc C r� r� r- r� r- r- r. r� � r� z1DownloadStreamRequestSubmitter._format_local_pathN) rI rJ rK r rq rY rp r� r� r- r- r- r. rO � s rO c @ s. e Zd ZejZeZdd� Zdd� Z dd� Z dS )rS c C � |j dko |jdkS )N�delete�s3�rz �src_typerj r- r- r. rY � � �z!DeleteRequestSubmitter.can_submitc C s" t |j�\}}| jj||||d�S )N)r� r� rt ru )r rx rL r� )r* r_ rt ru r� r� r- r- r. rr � s �z/DeleteRequestSubmitter._submit_transfer_requestc C � | � |j�d fS r~ )r� rx rj r- r- r. r{ � z'DeleteRequestSubmitter._format_src_destN) rI rJ rK r �map_delete_object_paramsro r rq rY rr r{ r- r- r- r. rS � s rS c @ s, e Zd ZdZdZdd� Zdd� Zdd� ZdS )rT Nc C r� )Nr� �localr� rj r- r- r. rY r� z&LocalDeleteRequestSubmitter.can_submitc C s� | � |�\}}d||d�}zDz"| j�tdddi|��� t�|j� | j�tdi |��� W W dS tyQ } z| j�t dd|i|��� W Y d }~W dS d }~ww Y dS )Nr� rw �total_transfer_sizer � exceptionTr- ) r{ rc r| r r� �removerx r � Exceptionr )r* r_ rt ru rx ry rv �er- r- r. rr s, � �� ���� z4LocalDeleteRequestSubmitter._submit_transfer_requestc C r� r~ )r� rx rj r- r- r. r{ - r� z,LocalDeleteRequestSubmitter._format_src_dest)rI rJ rK ro rq rY rr r{ r- r- r- r. rT s rT )8�loggingr� �s3transfer.managerr � awscli.customizations.s3.utilsr r r r r r �'awscli.customizations.s3.transferconfigr � awscli.customizations.s3.resultsr r r r r r r r r r r r r r r r r r r r r r r! r" r# r$ � awscli.compatr% � getLoggerrI r3 �objectr& r8 rb rP rQ rR rN rO rS rT r- r- r- r. �<module> sX B? ",$"
Save
Cancel