o `s @s4ddlZddlZddlZddlZddlZddlmZmZmZm Z m Z m Z m Z m Z mZmZmZmZmZmZmZmZeZzddlZWneyOddlmZYnweZ eZ eZ eZ eZ dZ dZ dZ! Gddde"Z#Gd d d e"Z$dZ%dZ&dZ'Gd d d ej(Z)d dddededddddf ddZ*dS)N)compress decompresscreate_compression_contextcompress_begincompress_chunkcompress_flushcreate_decompression_contextreset_decompression_contextdecompress_chunkget_frame_infoBLOCKSIZE_DEFAULTBLOCKSIZE_MAX64KBBLOCKSIZE_MAX256KBBLOCKSIZE_MAX1MBBLOCKSIZE_MAX4MB__doc__) _compressionc@sZeZdZdZededdddfddZddZdd Zdd d Z d dZ ddZ ddZ dS)LZ4FrameCompressora Create a LZ4 frame compressor object. This object can be used to compress data incrementally. Args: block_size (int): Specifies the maximum blocksize to use. Options: - `lz4.frame.BLOCKSIZE_DEFAULT`: the lz4 library default - `lz4.frame.BLOCKSIZE_MAX64KB`: 64 kB - `lz4.frame.BLOCKSIZE_MAX256KB`: 256 kB - `lz4.frame.BLOCKSIZE_MAX1MB`: 1 MB - `lz4.frame.BLOCKSIZE_MAX4MB`: 4 MB If unspecified, will default to `lz4.frame.BLOCKSIZE_DEFAULT` which is equal to `lz4.frame.BLOCKSIZE_MAX64KB`. block_linked (bool): Specifies whether to use block-linked compression. If ``True``, the compression ratio is improved, especially for small block sizes. If ``False`` the blocks are compressed independently. The default is ``True``. compression_level (int): Specifies the level of compression used. Values between 0-16 are valid, with 0 (default) being the lowest compression (0-2 are the same value), and 16 the highest. Values above 16 will be treated as 16. Values between 4-9 are recommended. 0 is the default. The following module constants are provided as a convenience: - `lz4.frame.COMPRESSIONLEVEL_MIN`: Minimum compression (0) - `lz4.frame.COMPRESSIONLEVEL_MINHC`: Minimum high-compression (3) - `lz4.frame.COMPRESSIONLEVEL_MAX`: Maximum compression (16) content_checksum (bool): Specifies whether to enable checksumming of the payload content. If ``True``, a checksum of the uncompressed data is stored at the end of the compressed frame which is checked during decompression. The default is ``False``. block_checksum (bool): Specifies whether to enable checksumming of the content of each block. If ``True`` a checksum of the uncompressed data in each block in the frame is stored at the end of each block. If present, these checksums will be used to validate the data during decompression. The default is ``False``, meaning block checksums are not calculated and stored. This functionality is only supported if the underlying LZ4 library has version >= 1.8.0. Attempting to set this value to ``True`` with a version of LZ4 < 1.8.0 will cause a ``RuntimeError`` to be raised. auto_flush (bool): When ``False``, the LZ4 library may buffer data until a block is full. When ``True`` no buffering occurs, and partially full blocks may be returned. The default is ``False``. return_bytearray (bool): When ``False`` a ``bytes`` object is returned from the calls to methods of this class. When ``True`` a ``bytearray`` object will be returned. The default is ``False``. TFcCsR||_||_||_||_|rtdkrtd||_||_||_ d|_ d|_ dS)Ni0*zEAttempt to set block_checksum to True with LZ4 libraryversion < 10800F) block_size block_linkedcompression_levelcontent_checksumlz4Zlibrary_version_number RuntimeErrorblock_checksum auto_flushreturn_bytearray_context_started)selfrrrrrrrr#4/usr/lib/python3/dist-packages/lz4/frame/__init__.py__init__s zLZ4FrameCompressor.__init__cC|SNr#r"r#r#r$ __enter__zLZ4FrameCompressor.__enter__cCs:d|_d|_d|_d|_d|_d|_d|_d|_d|_dS)NF) rrrrrrrr r!r"Zexception_typeZ exception tracebackr#r#r$__exit__s zLZ4FrameCompressor.__exit__rc CsN|jdur#t|_t|j|j|j|j|j|j|j |j |d }d|_|St d)aMBegin a compression frame. The returned data contains frame header information. The data returned from subsequent calls to ``compress()`` should be concatenated with this header. Keyword Args: source_size (int): Optionally specify the total size of the uncompressed data. If specified, will be stored in the compressed frame header as an 8-byte field for later use during decompression. Default is 0 (no size stored). Returns: bytes or bytearray: frame header data F)rrrrrrr source_sizeTz;LZ4FrameCompressor.begin() called after already initialized) r!rr rrrrrrrrr)r"r.resultr#r#r$begins$  zLZ4FrameCompressor.begincCs:|jdur td|jdurtdt|j||jd}|S)aCompresses data and returns it. This compresses ``data`` (a ``bytes`` object), returning a bytes or bytearray object containing compressed data the input. If ``auto_flush`` has been set to ``False``, some of ``data`` may be buffered internally, for use in later calls to `LZ4FrameCompressor.compress()` and `LZ4FrameCompressor.flush()`. The returned data should be concatenated with the output of any previous calls to `compress()` and a single call to `compress_begin()`. Args: data (str, bytes or buffer-compatible object): data to compress Returns: bytes or bytearray: compressed data Nzcompress called after flush()Fz'compress called before compress_begin())r)r rr!rr)r"datar/r#r#r$rs  zLZ4FrameCompressor.compresscCs"t|jd|jd}d|_d|_|S)aFinish the compression process. This returns a ``bytes`` or ``bytearray`` object containing any data stored in the compressor's internal buffers and a frame footer. The LZ4FrameCompressor instance may be re-used after this method has been called to create a new frame of compressed data. Returns: bytes or bytearray: compressed data and frame footer. T)Z end_framerNF)rr rr!)r"r/r#r#r$flush s zLZ4FrameCompressor.flushcCsd|_d|_dS)zReset the `LZ4FrameCompressor` instance. This allows the `LZ4FrameCompression` instance to be re-used after an error. NF)r r!r(r#r#r$resets zLZ4FrameCompressor.resetN)r) __name__ __module__ __qualname__rr COMPRESSIONLEVEL_MINr%r)r-r0rr2r3r#r#r#r$res 6  &" rc@s<eZdZdZdddZddZddZd d Zdd d ZdS)LZ4FrameDecompressoraCreate a LZ4 frame decompressor object. This can be used to decompress data incrementally. For a more convenient way of decompressing an entire compressed frame at once, see `lz4.frame.decompress()`. Args: return_bytearray (bool): When ``False`` a bytes object is returned from the calls to methods of this class. When ``True`` a bytearray object will be returned. The default is ``False``. Attributes: eof (bool): ``True`` if the end-of-stream marker has been reached. ``False`` otherwise. unused_data (bytes): Data found after the end of the compressed stream. Before the end of the frame is reached, this will be ``b''``. needs_input (bool): ``False`` if the ``decompress()`` method can provide more decompressed data before requiring new uncompressed input. ``True`` otherwise. FcCs*t|_d|_d|_d|_d|_||_dS)NFT)r r eof needs_input unused_data_unconsumed_data_return_bytearray)r"rr#r#r$r%Bs  zLZ4FrameDecompressor.__init__cCr&r'r#r(r#r#r$r)Jr*zLZ4FrameDecompressor.__enter__cCs(d|_d|_d|_d|_d|_d|_dSr')r r:r;r<r=r>r+r#r#r$r-Ns  zLZ4FrameDecompressor.__exit__cCs&t|jd|_d|_d|_d|_dS)zwReset the decompressor state. This is useful after an error occurs, allowing re-use of the instance. FTNr9)r r r:r;r<r=r(r#r#r$r3Vs  zLZ4FrameDecompressor.resetcCs||jr|j|}t|j|||jd\}}}|t|kr0|r%||d|_n||d|_d|_n d|_d|_d|_||_|S)a#Decompresses part or all of an LZ4 frame of compressed data. The returned data should be concatenated with the output of any previous calls to `decompress()`. If ``max_length`` is non-negative, returns at most ``max_length`` bytes of decompressed data. If this limit is reached and further output can be produced, the `needs_input` attribute will be set to ``False``. In this case, the next call to `decompress()` may provide data as ``b''`` to obtain more of the output. In all cases, any unconsumed data from previous calls will be prepended to the input data. If all of the input ``data`` was decompressed and returned (either because this was less than ``max_length`` bytes, or because ``max_length`` was negative), the `needs_input` attribute will be set to ``True``. If an end of frame marker is encountered in the data during decompression, decompression will stop at the end of the frame, and any data after the end of frame is available from the `unused_data` attribute. In this case, the `LZ4FrameDecompressor` instance is reset and can be used for further decompression. Args: data (str, bytes or buffer-compatible object): compressed data to decompress Keyword Args: max_length (int): If this is non-negative, this method returns at most ``max_length`` bytes of decompressed data. Returns: bytes: Uncompressed data ) max_lengthrNFr9T)r=r r r>lenr<r;r:)r"r1r@Z decompressedZ bytes_readZeoframer#r#r$rbs$%  zLZ4FrameDecompressor.decompressN)Fr?) r4r5r6rr%r)r-r3rr#r#r#r$r8*s  r8c @seZdZdZddededddddf ddZd d Zed d Z d dZ ddZ ddZ ddZ d$ddZd$ddZd$ddZd$ddZddZejfd d!Zd"d#ZdS)% LZ4FrameFileaA file object providing transparent LZ4F (de)compression. An LZ4FFile can act as a wrapper for an existing file object, or refer directly to a named file on disk. Note that LZ4FFile provides a *binary* file interface - data read is returned as bytes, and data to be written must be given as bytes. When opening a file for writing, the settings used by the compressor can be specified. The underlying compressor object is `lz4.frame.LZ4FrameCompressor`. See the docstrings for that class for details on compression options. Args: filename(str, bytes, PathLike, file object): can be either an actual file name (given as a str, bytes, or PathLike object), in which case the named file is opened, or it can be an existing file object to read from or write to. Keyword Args: mode(str): mode can be ``'r'`` for reading (default), ``'w'`` for (over)writing, ``'x'`` for creating exclusively, or ``'a'`` for appending. These can equivalently be given as ``'rb'``, ``'wb'``, ``'xb'`` and ``'ab'`` respectively. return_bytearray (bool): When ``False`` a bytes object is returned from the calls to methods of this class. When ``True`` a ``bytearray`` object will be returned. The default is ``False``. source_size (int): Optionally specify the total size of the uncompressed data. If specified, will be stored in the compressed frame header as an 8-byte field for later use during decompression. Default is ``0`` (no size stored). Only used for writing compressed files. block_size (int): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. block_linked (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. compression_level (int): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. content_checksum (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. block_checksum (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. auto_flush (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. NrTFrc  Cs0d|_d|_t|_|dvrt} n|dvr&t} t||||||| d|_d|_nt d |t j dkrszLZ4FrameFile.seekablecC||jtkS)zReturn whether the file was opened for reading. Returns: bool: ``True`` if the file was opened for reading, ``False`` otherwise. )rorSrTr(r#r#r$rqG zLZ4FrameFile.readablecCrs)zReturn whether the file was opened for writing. Returns: bool: ``True`` if the file was opened for writing, ``False`` otherwise. )rorSrUr(r#r#r$writableRrtzLZ4FrameFile.writabler?cC||j|S)zReturn buffered data without advancing the file position. Always returns at least one byte of data, unless at EOF. The exact number of bytes returned is unspecified. Returns: bytes: uncompressed data )_check_can_readrhpeekr"sizer#r#r$rx]s zLZ4FrameFile.peekcCrv)aRead up to ``size`` uncompressed bytes from the file. If ``size`` is negative or omitted, read until ``EOF`` is reached. Returns ``b''`` if the file is already at ``EOF``. Args: size(int): If non-negative, specifies the maximum number of uncompressed bytes to return. Returns: bytes: uncompressed data )rwrhrNryr#r#r$rNls zLZ4FrameFile.readcCs"||dkr tj}|j|S)aRead up to ``size`` uncompressed bytes. This method tries to avoid making multiple reads from the underlying stream. This method reads up to a buffer's worth of data if ``size`` is negative. Returns ``b''`` if the file is at EOF. Args: size(int): If non-negative, specifies the maximum number of uncompressed bytes to return. Returns: bytes: uncompressed data r)rwrfDEFAULT_BUFFER_SIZErhread1ryr#r#r$r|}s zLZ4FrameFile.read1cCrv)aRead a line of uncompressed bytes from the file. The terminating newline (if present) is retained. If size is non-negative, no more than size bytes will be read (in which case the line may be incomplete). Returns b'' if already at EOF. Args: size(int): If non-negative, specifies the maximum number of uncompressed bytes to return. Returns: bytes: uncompressed data )rwrhreadlineryr#r#r$r}s zLZ4FrameFile.readlinecCs:||j|}|j||jt|7_t|S)aWrite a bytes object to the file. Returns the number of uncompressed bytes written, which is always ``len(data)``. Note that due to buffering, the file on disk may not reflect the data written until close() is called. Args: data(bytes): uncompressed data to compress and write to the file Returns: int: the number of uncompressed bytes written to the file )_check_can_writerVrrPrOrWrA)r"r1 compressedr#r#r$rOs   zLZ4FrameFile.writecCs||j||S)aEChange the file position. The new position is specified by ``offset``, relative to the position indicated by ``whence``. Possible values for ``whence`` are: - ``io.SEEK_SET`` or 0: start of stream (default): offset must not be negative - ``io.SEEK_CUR`` or 1: current stream position - ``io.SEEK_END`` or 2: end of stream; offset must not be positive Returns the new file position. Note that seeking is emulated, so depending on the parameters, this operation may be extremely slow. Args: offset(int): new position in the file whence(int): position with which ``offset`` is measured. Allowed values are 0, 1, 2. The default is 0 (start of stream). Returns: int: new file position )_check_can_seekrhseek)r"offsetwhencer#r#r$rszLZ4FrameFile.seekcCs"||jtkr|jS|jS)z|Return the current file position. Args: None Returns: int: file position )rorSrTrhtellrWr(r#r#r$rs  zLZ4FrameFile.tellrB)r4r5r6rr r7r%rmpropertyrnrprrrqrurxrNr|r}rOrfSEEK_SETrrr#r#r#r$rCs4/ ;        rCrETFc  Csd|vrd|vrtd|fn|durtd|dur td|dur(td|dd} t|| ||||| | | d }d|vrGt||||S|S) a Open an LZ4Frame-compressed file in binary or text mode. ``filename`` can be either an actual file name (given as a str, bytes, or PathLike object), in which case the named file is opened, or it can be an existing file object to read from or write to. The ``mode`` argument can be ``'r'``, ``'rb'`` (default), ``'w'``, ``'wb'``, ``'x'``, ``'xb'``, ``'a'``, or ``'ab'`` for binary mode, or ``'rt'``, ``'wt'``, ``'xt'``, or ``'at'`` for text mode. For binary mode, this function is equivalent to the `LZ4FrameFile` constructor: `LZ4FrameFile(filename, mode, ...)`. For text mode, an `LZ4FrameFile` object is created, and wrapped in an ``io.TextIOWrapper`` instance with the specified encoding, error handling behavior, and line ending(s). Args: filename (str, bytes, os.PathLike): file name or file object to open Keyword Args: mode (str): mode for opening the file encoding (str): the name of the encoding that will be used for encoding/deconging the stream. It defaults to ``locale.getpreferredencoding(False)``. See ``io.TextIOWrapper`` for further details. errors (str): specifies how encoding and decoding errors are to be handled. See ``io.TextIOWrapper`` for further details. newline (str): controls how line endings are handled. See ``io.TextIOWrapper`` for further details. return_bytearray (bool): When ``False`` a bytes object is returned from the calls to methods of this class. When ``True`` a bytearray object will be returned. The default is ``False``. source_size (int): Optionally specify the total size of the uncompressed data. If specified, will be stored in the compressed frame header as an 8-byte field for later use during decompression. Default is 0 (no size stored). Only used for writing compressed files. block_size (int): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. block_linked (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. compression_level (int): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. content_checksum (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. block_checksum (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. auto_flush (bool): Compressor setting. See `lz4.frame.LZ4FrameCompressor`. trMzInvalid mode: %rNz0Argument 'encoding' not supported in binary modez.Argument 'errors' not supported in binary modez/Argument 'newline' not supported in binary mode)rjrrrrrrr)rXreplacerCrf TextIOWrapper)rirjencodingerrorsnewlinerrrrrrrr.rS binary_filer#r#r$rbs6@  rb)+rrfr_rarZZ_framerrrrrrr r r r r Z_BLOCKSIZE_DEFAULTrZ_BLOCKSIZE_MAX64KBrZ_BLOCKSIZE_MAX256KBrZ_BLOCKSIZE_MAX1MBrZ_BLOCKSIZE_MAX4MBrZ_docr ImportErrorrr7ZCOMPRESSIONLEVEL_MINHCZCOMPRESSIONLEVEL_MAXobjectrr8rRrTrU BaseStreamrCrbr#r#r#r$sdH   FwB