vKgSSrSSKJrJr /SQrSSKrSSKrSSKJrJ r \ "5r \ R"5r \ R"5r\ R "5r\ R$"5r\R("SS9r\ R,"5r\ R0rS r\ R6r\ R:r\ R>\ R@\ RB4r"\ RFr$\ RJr&\ RNr(\ RRr*\ RVr,\ RZr.\ R^r0\ Rbr2\ Rfr4\ Rjr6\ Rnr8\ Rrr:\ Rfr;\ Rjr<\ Rzr>\ R~r@\ RrB\ RrD\ RrF\ RrH\ RrJ\ RrL\ RrN\ RrP\ RrR\ RrT\ RrV\ RrX\ RrZ\ Rr\\ Rr^\ Rr`\ RrbSrcSrdSreSrfS rg"S S 5rh"S S5ri"SS5rj"SS5rk"SS\l5rmSrnSro"SS\p5rqSrrSrsSrt"SS\p5ru"SS\p5rv"S S!\p5rw"S"S#\p5rx"S$S%\p5ry"S&S'\p5rzS(r{S)r|S*r}"S+S,\p5r~S6S-jr"S.S/\p5r"S0S1\p5r"S2S3\p5r"S4S5\p5rg)7z=Python interface to the Zstandard (zstd) compression library.)absolute_importunicode_literals)F BufferSegmentBufferSegmentsBufferWithSegmentsBufferWithSegmentsCollectionZstdCompressionChunkerZstdCompressionDictZstdCompressionObjZstdCompressionParametersZstdCompressionReaderZstdCompressionWriterZstdCompressorZstdDecompressionObjZstdDecompressionReaderZstdDecompressionWriterZstdDecompressor ZstdErrorFrameParametersbackend_features#estimate_decompression_context_sizeframe_content_sizeframe_header_sizeget_frame_parameterstrain_dictionary FLUSH_BLOCK FLUSH_FRAMECOMPRESSOBJ_FLUSH_FINISHCOMPRESSOBJ_FLUSH_BLOCK ZSTD_VERSION FRAME_HEADERCONTENTSIZE_UNKNOWNCONTENTSIZE_ERRORMAX_COMPRESSION_LEVEL"COMPRESSION_RECOMMENDED_INPUT_SIZE#COMPRESSION_RECOMMENDED_OUTPUT_SIZE$DECOMPRESSION_RECOMMENDED_INPUT_SIZE%DECOMPRESSION_RECOMMENDED_OUTPUT_SIZE MAGIC_NUMBERBLOCKSIZELOG_MAX BLOCKSIZE_MAX WINDOWLOG_MIN WINDOWLOG_MAX CHAINLOG_MIN CHAINLOG_MAX HASHLOG_MIN HASHLOG_MAX MINMATCH_MIN MINMATCH_MAX SEARCHLOG_MIN SEARCHLOG_MAXSEARCHLENGTH_MINSEARCHLENGTH_MAXTARGETLENGTH_MINTARGETLENGTH_MAXLDM_MINMATCH_MINLDM_MINMATCH_MAXLDM_BUCKETSIZELOG_MAX STRATEGY_FASTSTRATEGY_DFASTSTRATEGY_GREEDY STRATEGY_LAZYSTRATEGY_LAZY2STRATEGY_BTLAZY2STRATEGY_BTOPTSTRATEGY_BTULTRASTRATEGY_BTULTRA2DICT_TYPE_AUTODICT_TYPE_RAWCONTENTDICT_TYPE_FULLDICT FORMAT_ZSTD1FORMAT_ZSTD1_MAGICLESSN)ffilibF)should_clear_after_allocs(/c[R"5=(d S$![a Of=f[R"S5$![[4a gf=f)NrSC_NPROCESSORS_ONLN)os cpu_countAttributeErrorsysconf ValueErrorV/var/www/highfloat_scraper/venv/lib/python3.13/site-packages/zstandard/backend_cffi.py _cpu_countrYs[ ||~""     zz/00 J '    s --AAAc4\rSrSrSr\S5rSrSrSr g)rzRepresents a segment within a ``BufferWithSegments``. This type is essentially a reference to N bytes within a ``BufferWithSegments``. The object conforms to the buffer protocol. c[5e)z9The byte offset of this segment within its parent buffer.NotImplementedErrorselfs rXoffsetBufferSegment.offset "##rWc[5e)z+Obtain the length of the segment, in bytes.r]r_s rX__len__BufferSegment.__len__ !##rWc[5e)z"Obtain bytes copy of this segment.r]r_s rXtobytesBufferSegment.tobytesrgrWrVN) __name__ __module__ __qualname____firstlineno____doc__propertyrareri__static_attributes__rVrWrXrrs%$$$$rWrc\rSrSrSrSrg)rzRepresents an array of ``(offset, length)`` integers. This type is effectively an index used by :py:class:`BufferWithSegments`. The array members are 64-bit unsigned integers using host/native bit order. Instances conform to the buffer protocol. rVN)rkrlrmrnrorqrVrWrXrrsrWrc@\rSrSrSr\S5rSrSrSr Sr Sr g ) ra+A memory buffer containing N discrete items of known lengths. This type is essentially a fixed size memory address and an array of 2-tuples of ``(offset, length)`` 64-bit unsigned native-endian integers defining the byte offset and length of each segment within the buffer. Instances behave like containers. Instances also conform to the buffer protocol. So a reference to the backing bytes can be obtained via ``memoryview(o)``. A *copy* of the backing bytes can be obtained via ``.tobytes()``. This type exists to facilitate operations against N>1 items without the overhead of Python object creation and management. Used with APIs like :py:meth:`ZstdDecompressor.multi_decompress_to_buffer`, it is possible to decompress many objects in parallel without the GIL held, leading to even better performance. c[5e)z)Total sizein bytes of the backing buffer.r]r_s rXsizeBufferWithSegments.sizercrWc[5eNr]r_s rXreBufferWithSegments.__len__s !##rWc[5e)zObtains a segment within the buffer. The returned object references memory within this buffer. :param i: Integer index of segment to retrieve. :return: :py:class:`BufferSegment` r]r`is rX __getitem__BufferWithSegments.__getitem__s "##rWc[5e)ziObtain the array of ``(offset, length)`` segments in the buffer. :return: :py:class:`BufferSegments` r]r_s rXsegmentsBufferWithSegments.segmentss "##rWc[5e)z#Obtain bytes copy of this instance.r]r_s rXriBufferWithSegments.tobytesrgrWrVN) rkrlrmrnrorprwrerrrirqrVrWrXrrs/($$$ $$$rWrc$\rSrSrSrSrSrSrg)ri aA virtual spanning view over multiple BufferWithSegments. Instances are constructed from 1 or more :py:class:`BufferWithSegments` instances. The resulting object behaves like an ordered sequence whose members are the segments within each ``BufferWithSegments``. If the object is composed of 2 ``BufferWithSegments`` instances with the first having 2 segments and the second have 3 segments, then ``b[0]`` and ``b[1]`` access segments in the first object and ``b[2]``, ``b[3]``, and ``b[4]`` access segments from the second. c[5e)z9The number of segments within all ``BufferWithSegments``.r]r_s rXre$BufferWithSegmentsCollection.__len__rgrWc[5e)z*Obtain the ``BufferSegment`` at an offset.r]r}s rXr(BufferWithSegmentsCollection.__getitem__rgrWrVN)rkrlrmrnrorerrqrVrWrXrr s $$rWrc\rSrSrSrg)rirVN)rkrlrmrnrqrVrWrXrrsrWrct[R"[R"U55R S5$)Nutf-8)rLstringrMZSTD_getErrorNamedecode)zresults rX _zstd_errorr#s) ::c++G4 5 < UR@4[RBURD4[RFURH4[RJURL4[RNURP4[RRURT4[RVURX4[RZUR\4[R^UR`4/nUHup4[cXU5 M U$rz)2rMZSTD_createCCtxParamsrLNULL MemoryErrorgcZSTD_freeCCtxParams ZSTD_c_formatformatZSTD_c_compressionLevelcompression_levelZSTD_c_windowLog window_logZSTD_c_hashLoghash_logZSTD_c_chainLog chain_logZSTD_c_searchLog search_logZSTD_c_minMatch min_matchZSTD_c_targetLength target_lengthZSTD_c_strategystrategyZSTD_c_contentSizeFlagwrite_content_sizeZSTD_c_checksumFlagwrite_checksumZSTD_c_dictIDFlag write_dict_idZSTD_c_nbWorkersthreadsZSTD_c_jobSizejob_sizeZSTD_c_overlapLog overlap_logZSTD_c_forceMaxWindowforce_max_window!ZSTD_c_enableLongDistanceMatching enable_ldmZSTD_c_ldmHashLog ldm_hash_logZSTD_c_ldmMinMatch ldm_min_matchZSTD_c_ldmBucketSizeLogldm_bucket_size_logZSTD_c_ldmHashRateLogldm_hash_rate_log_set_compression_parameter)paramsresattrsparamvalues rX_make_cctx_paramsr*s # # %C chhm &&c-- .C  FMM* $ $f&>&>?  v001  V__-  f../  v001  f../ &"6"67  foo. # #V%>%>? &"7"78   4 45  v~~.  V__-   2 23 " "F$;$;< . .0A0AB   3 34  !5!56 $ $f&@&@A " "F$<$<=+ E0 "3u5 JrWc\rSrSrSr\SSj5rSSjr\S5r \S5r \S5r \S5r \S 5r \S 5r\S 5r\S 5r\S 5r\S5r\S5r\S5r\S5r\S5r\S5r\S5r\S5r\S5r\S5r\S5r\S5rSrSrg)r iOaLow-level zstd compression parameters. This type represents a collection of parameters to control how zstd compression is performed. Instances can be constructed from raw parameters or derived from a base set of defaults specified from a compression level (recommended) via :py:meth:`ZstdCompressionParameters.from_level`. >>> # Derive compression settings for compression level 7. >>> params = zstandard.ZstdCompressionParameters.from_level(7) >>> # With an input size of 1MB >>> params = zstandard.ZstdCompressionParameters.from_level(7, source_size=1048576) Using ``from_level()``, it is also possible to override individual compression parameters or to define additional settings that aren't automatically derived. e.g.: >>> params = zstandard.ZstdCompressionParameters.from_level(4, window_log=10) >>> params = zstandard.ZstdCompressionParameters.from_level(5, threads=4) Or you can define low-level compression settings directly: >>> params = zstandard.ZstdCompressionParameters(window_log=12, enable_ldm=True) Once a ``ZstdCompressionParameters`` instance is obtained, it can be used to configure a compressor: >>> cctx = zstandard.ZstdCompressor(compression_params=params) Some of these are very low-level settings. It may help to consult the official zstandard documentation for their behavior. Look for the ``ZSTD_p_*`` constants in ``zstd.h`` (https://github.com/facebook/zstd/blob/dev/lib/zstd.h). c [R"XU5nSSSSSSSS.nUR5HupgXc;dM [XG5X6'M [ S 0UD6$) a-Create compression parameters from a compression level. :param level: Integer compression level. :param source_size: Integer size in bytes of source to be compressed. :param dict_size: Integer size in bytes of compression dictionary to use. :return: :py:class:`ZstdCompressionParameters` windowLogchainLoghashLog searchLogminMatch targetLengthr)rrrrrrrrV)rMZSTD_getCParamsitemsgetattrr )level source_size dict_sizekwargsrargsargattrs rX from_level$ZstdCompressionParameters.from_leveltsj$$UC&#!%#+" IC %f3 &)2622rWc[R"5nU[R:Xa [ 5e[R "U[R 5nUUlUS:a [5n[U[RU5 [U[RU5 [U[RU5 [U[RU5 [U[RU5 [U[RU5 [U[R U5 [U[R"U5 [U[R$U5 U S:XaSn [U[R&U 5 [U[R(U 5 [U[R*U 5 [U[R,U 5 [U[R.U 5 US:XaSn[U[R0U5 [U[R2U5 [U[R4U5 [U[R6U5 [U[R8U5 [U[R:U5 US:XaSn[U[R<U5 g)Nr)rMrrLrrrr_paramsrYrrrrrrrrrrrrrrrrrrrrrr)r`rrrrrrrrrrrrrrrrrrrrrrs rX__init__"ZstdCompressionParameters.__init__s$0**, SXX -  7 78 Q; lG #63+?+?I"63+<++> J"63+?+?L"63+>+> J" C++]  r>H"63+>+>I" C..0B  # C++^  #63+@+@-P"63+=+=xH " K"63+@+@+N" C--/?  # C99:  #63+@+@,O" C**M  # C//1D   " ! " C--/@ rWcJ[UR[R5$rz)_get_compression_parameterrrMrr_s rXr ZstdCompressionParameters.formats)$,,8I8IJJrWcJ[UR[R5$rz)rrrMrr_s rXr+ZstdCompressionParameters.compression_level) LL#55  rWcJ[UR[R5$rz)rrrMrr_s rXr$ZstdCompressionParameters.window_log)$,,8L8LMMrWcJ[UR[R5$rz)rrrMrr_s rXr"ZstdCompressionParameters.hash_log)$,,8J8JKKrWcJ[UR[R5$rz)rrrMrr_s rXr#ZstdCompressionParameters.chain_log)$,,8K8KLLrWcJ[UR[R5$rz)rrrMrr_s rXr$ZstdCompressionParameters.search_logrrWcJ[UR[R5$rz)rrrMrr_s rXr#ZstdCompressionParameters.min_matchrrWcJ[UR[R5$rz)rrrMrr_s rXr'ZstdCompressionParameters.target_length )$,,8O8OPPrWcJ[UR[R5$rz)rrrMrr_s rXr"ZstdCompressionParameters.strategyrrWcJ[UR[R5$rz)rrrMrr_s rXr,ZstdCompressionParameters.write_content_sizes) LL#44  rWcJ[UR[R5$rz)rrrMrr_s rXr(ZstdCompressionParameters.write_checksumrrWcJ[UR[R5$rz)rrrMrr_s rXr'ZstdCompressionParameters.write_dict_id)$,,8M8MNNrWcJ[UR[R5$rz)rrrMrr_s rXr"ZstdCompressionParameters.job_size!rrWcJ[UR[R5$rz)rrrMrr_s rXr%ZstdCompressionParameters.overlap_log%rrWcJ[UR[R5$rz)rrrMrr_s rXr*ZstdCompressionParameters.force_max_window)) LL#33  rWcJ[UR[R5$rz)rrrMrr_s rXr$ZstdCompressionParameters.enable_ldm/s) LL#??  rWcJ[UR[R5$rz)rrrMrr_s rXr&ZstdCompressionParameters.ldm_hash_log5rrWcJ[UR[R5$rz)rrrMrr_s rXr'ZstdCompressionParameters.ldm_min_match9s)$,,8N8NOOrWcJ[UR[R5$rz)rrrMrr_s rXr-ZstdCompressionParameters.ldm_bucket_size_log=rrWcJ[UR[R5$rz)rrrMrr_s rXr+ZstdCompressionParameters.ldm_hash_rate_logCrrWcJ[UR[R5$rz)rrrMrr_s rXr!ZstdCompressionParameters.threadsIrrWcB[R"UR5$)zAEstimated size in bytes needed to compress with these parameters.)rM%ZSTD_estimateCCtxSize_usingCCtxParamsrr_s rX"estimated_compression_context_size>> cctx = zstandard.ZstdCompressor(level=10) >>> compressor = cctx.stream_writer(fh) >>> compressor.write(b"chunk 0\n") >>> compressor.write(b"chunk 1\n") >>> compressor.flush() >>> # Receiver will be able to decode ``chunk 0\nchunk 1\n`` at this point. >>> # Receiver is also expecting more data in the zstd *frame*. >>> >>> compressor.write(b"chunk 2\n") >>> compressor.flush(zstandard.FLUSH_FRAME) >>> # Receiver will be able to decode ``chunk 0\nchunk 1\nchunk 2``. >>> # Receiver is expecting no more data, as the zstd frame is closed. >>> # Any future calls to ``write()`` at this point will construct a new >>> # zstd frame. Instances can be used as context managers. Exiting the context manager is the equivalent of calling ``close()``, which is equivalent to calling ``flush(zstandard.FLUSH_FRAME)``: >>> cctx = zstandard.ZstdCompressor(level=10) >>> with cctx.stream_writer(fh) as compressor: ... compressor.write(b'chunk 0') ... compressor.write(b'chunk 1') ... ... .. important:: If ``flush(FLUSH_FRAME)`` is not called, emitted data doesn't constitute a full zstd *frame* and consumers of this data may complain about malformed input. It is recommended to use instances as a context manager to ensure *frames* are properly finished. If the size of the data being fed to this streaming compressor is known, you can declare it before compression begins: >>> cctx = zstandard.ZstdCompressor() >>> with cctx.stream_writer(fh, size=data_len) as compressor: ... compressor.write(chunk0) ... compressor.write(chunk1) ... ... Declaring the size of the source data allows compression parameters to be tuned. And if ``write_content_size`` is used, it also results in the content size being written into the frame header of the output data. The size of chunks being ``write()`` to the destination can be specified: >>> cctx = zstandard.ZstdCompressor() >>> with cctx.stream_writer(fh, write_size=32768) as compressor: ... ... To see how much memory is being used by the streaming compressor: >>> cctx = zstandard.ZstdCompressor() >>> with cctx.stream_writer(fh) as compressor: ... ... ... byte_size = compressor.memory_size() Thte total number of bytes written so far are exposed via ``tell()``: >>> cctx = zstandard.ZstdCompressor() >>> with cctx.stream_writer(fh) as compressor: ... ... ... total_written = compressor.tell() ``stream_writer()`` accepts a ``write_return_read`` boolean argument to control the return value of ``write()``. When ``False`` (the default), ``write()`` returns the number of bytes that were ``write()``'en to the underlying object. When ``True``, ``write()`` returns the number of bytes read from the input that were subsequently written to the compressor. ``True`` is the *proper* behavior for ``write()`` as specified by the ``io.RawIOBase`` interface and will become the default value in a future release. cTXlX lX@l[U5Ul[U5UlSUlSUlSUlSUl [R"SU5Ul [R"S5Ul URURl[UR5URlSURl[$R&"UR(U5n[$R*"U5(a[-S[/U5-5eg)NFrchar[]ZSTD_outBuffer *error setting source size: %s) _compressor_writer _write_sizebool_write_return_read_closefd_entered_closing_closed_bytes_compressedrLr! _dst_buffer _out_bufferdstlenrwposrMZSTD_CCtx_setPledgedSrcSize_cctxrrr)r` compressorwriterr write_sizewrite_return_readclosefdrs rXrZstdCompressionWriter.__init__s& %"&'8"9W     !"778Z877#56#// #D$4$4 5 11*2B2BKP   G $ $/+g2FF  %rWcUR(a [S5eUR(a [S5eSUlU$Nstream is closedcannot __enter__ multiple timesTr1rUr/rr_s rX __enter__ZstdCompressionWriter.__enter__s4 <</0 0 ===> >  rWc@SUlUR5 SUlgNF)r/closer)r`exc_type exc_valueexc_tbs rX__exit__ZstdCompressionWriter.__exit__s  rWc,[R"5erzioUnsupportedOperationr_s rX__iter__ZstdCompressionWriter.__iter__ %%''rWc,[R"5erzrQr_s rX__next__ZstdCompressionWriter.__next__ rVrWcV[R"URR5$rz)rMZSTD_sizeof_CCtxr)r9r_s rX memory_size!ZstdCompressionWriter.memory_sizes##D$4$4$:$:;;rWcb[URSS5nU(aU"5$[S5eNfilenoz)fileno not available on underlying writerrr*OSErrorr`fs rXr`ZstdCompressionWriter.fileno* DLL(D 1 3JEF FrWcUR(agSUlUR[5 SUlSUl[ UR SS5nUR (aU(aU"5 ggg!SUlSUlf=fNTFrI)r1r0flushrrr*r.rcs rXrIZstdCompressionWriter.closesm <<   DM JJ{ #!DMDL DLL'4 0 ==Q C= "DMDLs A77BcUR$rzr1r_s rXclosedZstdCompressionWriter.closed* ||rWcgrHrVr_s rXisattyZstdCompressionWriter.isatty.rWcgrHrVr_s rXreadableZstdCompressionWriter.readable1rsrWc,[R"5erzrQr`rws rXreadlineZstdCompressionWriter.readline4rVrWc,[R"5erzrQr`hints rX readlinesZstdCompressionWriter.readlines7rVrWNc,[R"5erzrQr`rawhences rXseekZstdCompressionWriter.seek:rVrWcgrHrVr_s rXseekableZstdCompressionWriter.seekable=rsrWc,[R"5erzrQrxs rXtruncateZstdCompressionWriter.truncate@rVrWcgNTrVr_s rXwritableZstdCompressionWriter.writableCrWc[S5e)Nz#writelines() is not yet implementedr]r`liness rX writelines ZstdCompressionWriter.writelinesFs!"GHHrWc,[R"5erzrQrxs rXreadZstdCompressionWriter.readIrVrWc,[R"5erzrQr_s rXreadallZstdCompressionWriter.readallLrVrWc,[R"5erzrQr`bs rXreadintoZstdCompressionWriter.readintoOrVrWcUR(a [S5eSn[R"U5n[R"S5nX4l[ U5UlSUlURnSUlURUR:Ga[R"URRUU[R5n[R"U5(a[!S[#U5-5eUR(a{UR$R'[R("UR*UR5SS5 X%R- nU=R,UR- slSUlURUR:aGMUR.(a UR$U$)z=Send data to the compressor and possibly to the inner stream.rBrZSTD_inBuffer *zstd compress error: %sN)r1rUrL from_bufferr!srcr6rwr7r4rMZSTD_compressStream2r)r9ZSTD_e_continuerrrr*writebufferr5r2r-)r`data total_write data_buffer in_buffer out_bufferrs rXrZstdCompressionWriter.writeRsY <</0 0 ood+ GG-. # [)  %%  mminn,..  &&## G ((- G0DD~~ ""JJz~~z~~>qA~~- &&*..8&!" %mminn,(  " "==  rWcU[:Xa[RnO)U[:Xa[RnO[ SU-5eUR (a [ S5eSnURnSUl[R"S5n[RUl SUl SUl[R"URR XEU5n[R""U5(a[%S['U5-5eUR(a{UR(R+[R,"UR.UR5SS5 X4R- nU=R0UR- slSUlU(dOM[3UR(SS5nU(aUR4(dU"5 U$)aEvict data from compressor's internal state and write it to inner stream. Calling this method may result in 0 or more ``write()`` calls to the inner stream. This method will also call ``flush()`` on the inner stream, if such a method exists. :param flush_mode: How to flush the zstd compressor. ``zstandard.FLUSH_BLOCK`` will flush data already sent to the compressor but not emitted to the inner stream. The stream is still writable after calling this. This is the default behavior. See documentation for other ``zstandard.FLUSH_*`` constants for more flushing options. :return: Integer number of bytes written to the inner stream. zunknown flush_mode: %rrBrrrNri)rrM ZSTD_e_flushr ZSTD_e_endrUr1r4r7rLr!rrrwrr)r9rrrr*rrr5r2rr0)r` flush_moderirrrrrds rXriZstdCompressionWriter.flush|sq,  $$$E ; &NNE5 BC C <</0 0 %%  GG-.    ..  && uG((- G0DD~~ ""JJz~~z~~>qA~~- &&*..8&!" %( DLL'4 0 T]] CrWcUR$rzr2r_s rXtellZstdCompressionWriter.tell%%%rW) r2r1r.r0r)r3r/r4r-r+r*Trrz) rkrlrmrnrorrErNrTrXr\r`rIrprmrqruryr~rrrrrrrrrrrirrqrVrWrXrrqsf^>((<G ((((I((((T +BH&rWrc6\rSrSrSr\4SjrSr\4Sjr Sr g)r iaA compressor conforming to the API in Python's standard library. This type implements an API similar to compression types in Python's standard library such as ``zlib.compressobj`` and ``bz2.BZ2Compressor``. This enables existing code targeting the standard library API to swap in this type to achieve zstd compression. .. important:: The design of this API is not ideal for optimal performance. The reason performance is not optimal is because the API is limited to returning a single buffer holding compressed data. When compressing data, we don't know how much data will be emitted. So in order to capture all this data in a single buffer, we need to perform buffer reallocations and/or extra memory copies. This can add significant overhead depending on the size or nature of the compressed data how much your application calls this type. If performance is critical, consider an API like :py:meth:`ZstdCompressor.stream_reader`, :py:meth:`ZstdCompressor.stream_writer`, :py:meth:`ZstdCompressor.chunker`, or :py:meth:`ZstdCompressor.read_to_iter`, which result in less overhead managing buffers. Instances are obtained by calling :py:meth:`ZstdCompressor.compressobj`. Here is how this API should be used: >>> cctx = zstandard.ZstdCompressor() >>> cobj = cctx.compressobj() >>> data = cobj.compress(b"raw input 0") >>> data = cobj.compress(b"raw input 1") >>> data = cobj.flush() Or to flush blocks: >>> cctx.zstandard.ZstdCompressor() >>> cobj = cctx.compressobj() >>> data = cobj.compress(b"chunk in first block") >>> data = cobj.flush(zstandard.COMPRESSOBJ_FLUSH_BLOCK) >>> data = cobj.compress(b"chunk in second block") >>> data = cobj.flush() For best performance results, keep input chunks under 256KB. This avoids extra allocations for a large output object. It is possible to declare the input size of the data that will be fed into the compressor: >>> cctx = zstandard.ZstdCompressor() >>> cobj = cctx.compressobj(size=6) >>> data = cobj.compress(b"foobar") >>> data = cobj.flush() cXl[R"S5Ul[R"SU5UlURURlX RlSURlSUlg)Nr'r&rF) r)rLr!_outr3r5rwr7 _finished)r`r:r<s rXrZstdCompressionObj.__init__sZ&GG./ 778Z8(( #  rWcFUR(a [S5e[R"U5n[R"S5nX#l[ U5UlSUl/nUR[ U5:Ga[R"URRURU[R5n[R"U5(a[S[!U5-5eURR(abUR#[R$"URR&URR5SS5 SURlUR[ U5:aGMSR)U5$)aSend data to the compressor. This method receives bytes to feed to the compressor and returns bytes constituting zstd compressed data. The zstd compressor accumulates bytes and the returned bytes may be substantially smaller or larger than the size of the input data on any given call. The returned value may be the empty byte string (``b""``). :param data: Data to write to the compressor. :return: Compressed data. z0cannot call compress() after compressor finishedrrrNrW)rrrLrr!rr6rwr7rMrr)r9rrrrappendrr5join)r`rrsourcechunksrs rXcompressZstdCompressionObj.compress s$ >>NO Oood+ *+ +&  jj3t9$..  && 63;N;NG((- G0DDyy}} cjj FqIJ ! jj3t9$xxrWcU[[4;a [S5eUR(a [ S5eU[:Xa[ R nO-U[:Xa[ RnSUlO [ S5eURRS:Xde[R"S5n[RUl SUlSUl /n[ R"UR R"URX25n[ R$"U5(a[ S['U5-5eURR(abUR)[R*"URR,URR5SS5 SURl U(dOMS R/U5$) aiEmit data accumulated in the compressor that hasn't been outputted yet. The ``flush_mode`` argument controls how to end the stream. ``zstandard.COMPRESSOBJ_FLUSH_FINISH`` (the default) ends the compression stream and finishes a zstd frame. Once this type of flush is performed, ``compress()`` and ``flush()`` can no longer be called. This type of flush **must** be called to end the compression context. If not called, the emitted data may be incomplete and may not be readable by a decompressor. ``zstandard.COMPRESSOBJ_FLUSH_BLOCK`` will flush a zstd block. This ensures that all data fed to this instance will have been omitted and can be decoded by a decompressor. Flushes of this type can be performed multiple times. The next call to ``compress()`` will begin a new zstd block. :param flush_mode: How to flush the zstd compressor. :return: Compressed data. zflush mode not recognizedz"compressor object already finishedTzunhandled flush moderr#error ending compression stream: %sNrW)rrrUrrrMrrrr7rLr!rrrwrr)r9rrrrr5r)r`r z_flush_moderrrs rXriZstdCompressionObj.flush3sk.  $ #  89 9 >>@A A 0 0++L 3 3>>L!DN23 3yy}}!!!GG-.    ..  && 9G((9K>> cctx = zstandard.ZstdCompressor() >>> chunker = cctx.chunker(chunk_size=32768) >>> >>> with open(path, 'rb') as fh: ... while True: ... in_chunk = fh.read(32768) ... if not in_chunk: ... break ... ... for out_chunk in chunker.compress(in_chunk): ... # Do something with output chunk of size 32768. ... ... for out_chunk in chunker.finish(): ... # Do something with output chunks that finalize the zstd frame. This compressor type is often a better alternative to :py:class:`ZstdCompressor.compressobj` because it has better performance properties. ``compressobj()`` will emit output data as it is available. This results in a *stream* of output chunks of varying sizes. The consistency of the output chunk size with ``chunker()`` is more appropriate for many usages, such as sending compressed data to a socket. ``compressobj()`` may also perform extra memory reallocations in order to dynamically adjust the sizes of the output chunks. Since ``chunker()`` output chunks are all the same size (except for flushed or final chunks), there is less memory allocation/copying overhead. cXl[R"S5Ul[R"SU5UlURURlX RlSURl[R"S5Ul[RURl SURlSURlSUl g)Nr'r&rrF) r)rLr!rr3r5rwr7_inrrr)r`r: chunk_sizes rXrZstdCompressionChunker.__init__s%GG./ 778Z8(( #  77,-xx   rWc## UR(a [S5eURR[R :wa [S5e[R "U5n[U5(dgX Rl[U5URlSURl URRURR:Ga[R"URRURUR[R5nURRURR:XaA[R URlSURlSURl [R "U5(a[S[#U5-5eURRURR:XaU[R$"URR&URR5SSv SURl URRURR:aGMgg7f)zFeed new input data into the compressor. :param data: Data to feed to compressor. :return: Iterator of ``bytes`` representing chunks of compressed data. z1cannot call compress() after compression finishedzHcannot perform operation before consuming output from previous operationNrr)rrrrrLrrr6rwr7rMrr)r9rrrrrr5)r`rrrs rXrZstdCompressionChunker.compresss >>OP P 88<<388 #*  ood+ ; " K(  hhllTXX]]*..  && 488S=P=PGxx||txx}},"xx !   ((- G0DDyy}} .jj >qAA ! #hhllTXX]]**s JJ J c## UR(a [S5eURR[R :wa [S5e[ R"URRURUR[ R5n[ R"U5(a[S[U5-5eURR(aU[R"URR URR5SSv SURlU(dgM7f)zeFlushes all data currently in the compressor. :return: Iterator of ``bytes`` of compressed data. z.cannot call flush() after compression finishedzCcannot call flush() before consuming output from previous operationrNr)rrrrrLrrMrr)r9rrrrr7rr5r`rs rXriZstdCompressionChunker.flushs >>LM M 88<<388 #%  ..  && 488S=M=MG((- G0DDyy}}jj >qAA ! sE Ec## UR(a [S5eURR[R :wa [S5e[ R"URRURUR[ R5n[ R"U5(a[S[U5-5eURR(aU[R"URR URR5SSv SURlU(dSUlgGM7f)zSignals the end of input data. No new data can be compressed after this method is called. This method will flush buffered data and finish the zstd frame. :return: Iterator of ``bytes`` of compressed data. z/cannot call finish() after compression finishedzDcannot call finish() before consuming output from previous operationTrNr)rrrrrLrrMrr)r9rrrrr7rr5rs rXfinishZstdCompressionChunker.finishs >>MN N 88<<388 #%  ..  && 488S^^G((- G0DDyy}}jj >qAA ! !%sEE)r)r3rrrN) rkrlrmrnrorrrirrqrVrWrXr r ws2h +"Z>"rWr c\rSrSrSrSSjrSrSrSrSr Sr S r S r S r S rS rSrSr\S5rSrSrSrSr\rSrSrSSjrSSjrSrSrSrg)r i+ag Readable compressing stream wrapper. ``ZstdCompressionReader`` is a read-only stream interface for obtaining compressed data from a source. This type conforms to the ``io.RawIOBase`` interface and should be usable by any type that operates against a *file-object* (``typing.BinaryIO`` in Python type hinting speak). Instances are neither writable nor seekable (even if the underlying source is seekable). ``readline()`` and ``readlines()`` are not implemented because they don't make sense for compressed data. ``tell()`` returns the number of compressed bytes emitted so far. Instances are obtained by calling :py:meth:`ZstdCompressor.stream_reader`. In this example, we open a file for reading and then wrap that file handle with a stream from which compressed data can be ``read()``. >>> with open(path, 'rb') as fh: ... cctx = zstandard.ZstdCompressor() ... reader = cctx.stream_reader(fh) ... while True: ... chunk = reader.read(16384) ... if not chunk: ... break ... ... # Do something with compressed chunk. Instances can also be used as context managers: >>> with open(path, 'rb') as fh: ... cctx = zstandard.ZstdCompressor() ... with cctx.stream_reader(fh) as reader: ... while True: ... chunk = reader.read(16384) ... if not chunk: ... break ... ... # Do something with compressed chunk. When the context manager exits or ``close()`` is called, the stream is closed, underlying resources are released, and future operations against the compression stream will fail. ``stream_reader()`` accepts a ``size`` argument specifying how large the input stream is. This is used to adjust compression parameters so they are tailored to the source size. e.g. >>> with open(path, 'rb') as fh: ... cctx = zstandard.ZstdCompressor() ... with cctx.stream_reader(fh, size=os.stat(path).st_size) as reader: ... ... If the ``source`` is a stream, you can specify how large ``read()`` requests to that stream should be via the ``read_size`` argument. It defaults to ``zstandard.COMPRESSION_RECOMMENDED_INPUT_SIZE``. e.g. >>> with open(path, 'rb') as fh: ... cctx = zstandard.ZstdCompressor() ... # Will perform fh.read(8192) when obtaining data to feed into the ... # compressor. ... with cctx.stream_reader(fh, read_size=8192) as reader: ... ... cXlX lX0lX@lSUlSUlSUlSUlSUl[R"S5Ul SUl gNFrr) r)_source _read_sizer.r/r1r2_finished_input_finished_outputrLr! _in_buffer_source_buffer)r`r:r read_sizer>s rXrZstdCompressionReader.__init__nsV% #   !"$ %''"34"rWcUR(a [S5eUR(a [S5eSUlU$NrCrBTr/rUr1r_s rXrEZstdCompressionReader.__enter__}4 ==>? ? <</0 0  rWcNSUlSUlUR5 SUlgrH)r/r)rIrrJs rXrNZstdCompressionReader.__exit__s$   rWcgrrVr_s rXruZstdCompressionReader.readablerrWcgrHrVr_s rXrZstdCompressionReader.writablersrWcgrHrVr_s rXrZstdCompressionReader.seekablersrWc,[R"5erzrQr_s rXryZstdCompressionReader.readlinerVrWc,[R"5erzrQr_s rXr~ZstdCompressionReader.readlinesrVrWc[S5eNzstream is not writablerbr`rs rXrZstdCompressionReader.write.//rWc[S5err)r`ignoreds rXr ZstdCompressionReader.writelinesrrWcgrHrVr_s rXrqZstdCompressionReader.isattyrsrWcgrzrVr_s rXriZstdCompressionReader.flushrrWcUR(agSUl[URSS5nUR(aU(aU"5 gggNTrIr1rrr.rcs rXrIZstdCompressionReader.closes: <<  DLL'4 0 ==Q C=rWcUR$rzrlr_s rXrmZstdCompressionReader.closedrorWcUR$rzrr_s rXrZstdCompressionReader.tellrrWc/nURS5nU(dOURU5 M,SRU5$NirWrrrr`rchunks rXrZstdCompressionReader.readall@IIg&E MM% xxrWc,[R"5erzrQr_s rXrTZstdCompressionReader.__iter__rVrWc,[R"5erzrQr_s rXrXZstdCompressionReader.__next__rVrWcUR(ag[URS5(aURRUR5nU(dSUlg[ R "U5UlURURl [UR5URl SURl g[ R "UR5UlURURl [UR5URl SURl gNrTr) rhasattrrrrrLrrrrr6rwr7rs rX _read_input!ZstdCompressionReader._read_inputs     4<< ( (<<$$T__5D'+$"%//$"7D "&"5"5DOO #&t':':#;DOO "#DOO "%//$,,"?D "&"5"5DOO #&t':':#;DOO "#DOO rWc.URRURR:agURn[R"UR R UUR[R5nU=RURU- - slURRURR:Xaj[RURl SURlSURlSUl [URS5(dSUl[R "U5(a[#S[%U55eUR=(a URUR:H$)NrrTr)rr7rwrMrr)r9rr2rLrrrrrrrrr)r`rold_posrs rX_compress_into_buffer+ZstdCompressionReader._compress_into_buffers ??  $//"6"6 6 ..**    " "  OO      *..7":: ??  $//"6"6 6"%((DOO "#DOO #$DOO "&D 4<<00'+$   G $ $5{77KL L~~C*..JOO"CCrWcjUR(a [S5eUS:a [S5eUS:XaUR5$UR(dUS:Xag[R "SU5n[R "S5nX#lXlSUlURU5(a.[R"UR UR5SS$UR(dgUR5 URU5(a.[R"UR UR5SS$UR(dMgURn[R"URR X0R"[R$5nU=R&URU- - sl[R("U5(a[+S[-U55eUS:XaS Ul[R"UR UR5SS$ NrBr)cannot read negative amounts less than -1rrWr&r'rT)r1rUrrrLr!r5rwr7rrrrrMrr)r9rrr2rrrr`rw dst_bufferrrrs rXrZstdCompressionReader.reads <</0 0 "9HI I 2:<<> !  DAIWWXt, WW/0 #   % %j 1 1::jnnjnn=a@ @&&    ))*55zz*..*..A!DD &&&..**    " "J  *..7"::   G $ $5{77K  a<$(D !zz*..*..9!<>::jnnjnn=a@ @&&    ))*55zz*..*..A!DD~~d&:&:zz*..*..A!DD&&& ..**    " "J  *..7"::   G $ $5 G8LL  a<$(D !zz*..*..9!<> !&&    ))*55!~~%&&& ..**    " "J  *..7"::   G $ $5{77K  a<$(D !~~rWcUR(a [S5eUR(ag[R"U5n[R "USS5 [R "S5nX#l[U5Ul SUl URU5 UR(a UR$UR(dsUR5 URU5(a UR$UR(aUR(d UR$UR(dMsURn[R"UR R"X0R$[R&5nU=R(URU- - sl[R*"U5(a[-S[/U5-5eUS:XaSUlUR$r'r(r*s rX readinto1ZstdCompressionReader.readinto1sv <</0 0  ooa(  AsAWW/0 $k*   "":. >>>> !&&    ))*55!~~%~~d&:&:!~~%&&&..**    " "J  *..7"::   G $ $5 G8LL  a<$(D !~~rW) r2r1r.r)r/rrrrrrNrr) rkrlrmrnrorrErNrurrryr~rrrqrirIrprmrrrTrXnextrrrr$rr.rqrVrWrXr r +s@D #((00&  (( D$*D:.=`@=D'R.rWr c\rSrSrSrSSjrSrSrSrSS jr S\ 4S jr S\ \ 4S jr S\ S 4S jrS\ S S 4SjrS\ \ 4SjrSSjrSrSrg)ria Create an object used to perform Zstandard compression. Each instance is essentially a wrapper around a ``ZSTD_CCtx`` from zstd's C API. An instance can compress data various ways. Instances can be used multiple times. Each compression operation will use the compression parameters defined at construction time. .. note: When using a compression dictionary and multiple compression operations are performed, the ``ZstdCompressionParameters`` derived from an integer compression ``level`` and the first compressed data's size will be reused for all subsequent operations. This may not be desirable if source data sizes vary significantly. ``compression_params`` is mutually exclusive with ``level``, ``write_checksum``, ``write_content_size``, ``write_dict_id``, and ``threads``. Assume that each ``ZstdCompressor`` instance can only handle a single logical compression operation at the same time. i.e. if you call a method like ``stream_reader()`` to obtain multiple objects derived from the same ``ZstdCompressor`` instance and attempt to use them simultaneously, errors will likely occur. If you need to perform multiple logical compression operations and you can't guarantee those operations are temporally non-overlapping, you need to obtain multiple ``ZstdCompressor`` instances. Unless specified otherwise, assume that no two methods of ``ZstdCompressor`` instances can be called from multiple Python threads simultaneously. In other words, assume instances are not thread safe unless stated otherwise. :param level: Integer compression level. Valid values are all negative integers through 22. Lower values generally yield faster operations with lower compression ratios. Higher values are generally slower but compress better. The default is 3, which is what the ``zstd`` CLI uses. Negative levels effectively engage ``--fast`` mode from the ``zstd`` CLI. :param dict_data: A ``ZstdCompressionDict`` to be used to compress with dictionary data. :param compression_params: A ``ZstdCompressionParameters`` instance defining low-level compression parameters. If defined, this will overwrite the ``level`` argument. :param write_checksum: If True, a 4 byte content checksum will be written with the compressed data, allowing the decompressor to perform content verification. :param write_content_size: If True (the default), the decompressed content size will be included in the header of the compressed data. This data will only be written if the compressor knows the size of the input data. :param write_dict_id: Determines whether the dictionary ID will be written into the compressed data. Defaults to True. Only adds content to the compressed data if a dictionary is being used. :param threads: Number of threads to use to compress data concurrently. When set, compression operations are performed on multiple threads. The default value (0) disables multi-threaded compression. A value of ``-1`` means to set the number of threads to the number of detected logical CPUs. Nc U[R"5:a![S[R"5-5eUS:a [5nU(aUb [S5eU(aUb [S5eU(aUb [S5eU(aU(a [S5eU(a[ U5UlGO9UcSn[R "5nU[R:Xa [5e[R"U[R5Ul[UR [RU5 [UR [RUbUOS5 [UR [RU(aSOS5 [UR [R U(aSOS5 U(a%[UR [R"U5 [R$"5n U [R:Xa [5eXlX lUR+5 [R"U [R,[R."U 5S 9Ulg![R"U [R,[R."U 5S 9Ulf=f) Nzlevel must be less than %drz3cannot define compression_params and write_checksumz7cannot define compression_params and write_content_sizez2cannot define compression_params and write_dict_idz,cannot define compression_params and threadsTrKrw)rMZSTD_maxCLevelrUrYrrrrLrrrrrrrrrrZSTD_createCCtxr9 _dict_data _setup_cctx ZSTD_freeCCtxr[) r`r dict_datacompression_paramsrrrrrcctxs rXrZstdCompressor.__init__s$ 3%%' ',s/A/A/CC  Q; lG ."<H  "4"@L  -";G  'KL L ,-?@DL$ $ ..0F!!m#66&#*A*ABDL & c995  ' **&8&D"!  ' ''#  ' c33-QQ *LL#"6"6""$ 388 -  #     c''c.B.B4.HDJc''c.B.B4.HDJs 5J?Kcj[R"URUR5n[R"U5(a[ S[ U5-5eURnU(aUR(a,[R"URUR5nOS[R"URUR5[U5[RUR5n[R"U5(a[ S[ U5-5egg)Nz(could not set compression parameters: %sz)could not load compression dictionary: %s)rM&ZSTD_CCtx_setParametersUsingCCtxParamsr9rrrrr6_cdictZSTD_CCtx_refCDict!ZSTD_CCtx_loadDictionary_advancedas_bytesr6ZSTD_dlm_byRef _dict_type)r`rr9s rXr7ZstdCompressor._setup_cctxjs<< JJ     G $ $:g&'  OO 00Y=M=MN??JJ&&( N&&(( ((?!'*+) rWcB[R"UR5$)z}Obtain the memory usage of this compressor, in bytes. >>> cctx = zstandard.ZstdCompressor() >>> memory = cctx.memory_size() )rMr[r9r_s rXr\ZstdCompressor.memory_size ##DJJ//rWc[R"UR[R5 [R "U5n[R "[U55n[SU5n[R"UR[U55n[R"U5(a[S[U5-5e[R"S5n[R"S5nXFlX6lSUlX'l[U5UlSUl[R$"URXg[R&5n[R"U5(a[S[U5-5eU(a [S5e[R("XFR 5SS$) a$ Compress data in a single operation. This is the simplest mechanism to perform compression: simply pass in a value and get a compressed value back. It is almost the most prone to abuse. The input and output values must fit in memory, so passing in very large values can result in excessive memory usage. For this reason, one of the streaming based APIs is preferred for larger values. :param data: Source data to compress :return: Compressed data >>> cctx = zstandard.ZstdCompressor() >>> compressed = cctx.compress(b"data to compress") r&r(r'rrzcannot compress: %szunexpected partial frame flushN)rMZSTD_CCtx_resetr9ZSTD_reset_session_onlyrLrZSTD_compressBoundr6 new_nonzeror8rrrr!r5rwr7rrrr)r`rr dest_sizeoutrrrs rXrZstdCompressor.compresssK( DJJ(C(CDood+ **3{+;< (I.11$**c+>NO   G $ $/+g2FF WW/0 GG-. # # [)  ** JJ s~~    G $ $1K4HHI I <= =zz#~~.q11rWrcV[R"UR[R5 US:a[Rn[R "URU5n[R "U5(a[S[U5-5e[U[5nU$)z Obtain a compressor exposing the Python standard library compression API. See :py:class:`ZstdCompressionObj` for the full documentation. :param size: Size in bytes of data that will be compressed. :return: :py:class:`ZstdCompressionObj` rr() rMrJr9rKZSTD_CONTENTSIZE_UNKNOWNr8rrrr r&)r`rwrcobjs rX compressobjZstdCompressor.compressobjs DJJ(C(CD !8//D11$**dC   G $ $/+g2FF "$(KL rWcD[R"UR[R5 US:a[Rn[R "URU5n[R "U5(a[S[U5-5e[XS9$)aG Create an object for iterative compressing to same-sized chunks. This API is similar to :py:meth:`ZstdCompressor.compressobj` but has better performance properties. :param size: Size in bytes of data that will be compressed. :param chunk_size: Size of compressed chunks. :return: :py:class:`ZstdCompressionChunker` rr()r) rMrJr9rKrRr8rrrr )r`rwrrs rXchunkerZstdCompressor.chunkers~ DJJ(C(CD !8//D11$**dC   G $ $/+g2FF &dBBrWc[US5(d [S5e[US5(d [S5e[R"UR[R 5 US:a[R n[R"URU5n[R"U5(a[S[U5-5e[R"S5n[R"S5n[R"S U5n Xl XXlSUlS upUR!U5n U (dGO<[R""U 5n U [%U 5- n Xl[%U 5UlSUlURUR:a[R("URX[R*5n[R"U5(a[S [U5-5eUR(aOUR-[R."URUR55 XR- n SUlURUR:aMGMV[R("URX[R05n[R"U5(a[S [U5-5eUR(aOUR-[R."URUR55 XR- n SUlUS:XaX4$M) a" Copy data between 2 streams while compressing it. Data will be read from ``ifh``, compressed, and written to ``ofh``. ``ifh`` must have a ``read(size)`` method. ``ofh`` must have a ``write(data)`` method. >>> cctx = zstandard.ZstdCompressor() >>> with open(input_path, "rb") as ifh, open(output_path, "wb") as ofh: ... cctx.copy_stream(ifh, ofh) It is also possible to declare the size of the source stream: >>> cctx = zstandard.ZstdCompressor() >>> cctx.copy_stream(ifh, ofh, size=len_of_input) You can also specify how large the chunks that are ``read()`` and ``write()`` from and to the streams: >>> cctx = zstandard.ZstdCompressor() >>> cctx.copy_stream(ifh, ofh, read_size=32768, write_size=16384) The stream copier returns a 2-tuple of bytes read and written: >>> cctx = zstandard.ZstdCompressor() >>> read_count, write_count = cctx.copy_stream(ifh, ofh) :param ifh: Source stream to read from :param ofh: Destination stream to write to :param size: Size in bytes of the source stream. If defined, compression parameters will be tuned for this size. :param read_size: Chunk sizes that source stream should be ``read()`` from. :param write_size: Chunk sizes that destination stream should be ``write()`` to. :return: 2-tuple of ints of bytes read and written, respectively. r(first argument must have a read() methodr*second argument must have a write() methodrr(rr'r&rrr)rrUrMrJr9rKrRr8rrrrLr!r5rwr7rrr6rrrrrr)r`ifhofhrwrr<rrrr! total_readrrrs rX copy_streamZstdCompressor.copy_streamsfsF##GH HsG$$IJ J DJJ(C(CD !8//D11$**dC   G $ $/+g2FF GG-. WW/0 WWXz2 #$ "& 88I&D//$/K #k* *J'M -INIM--)..022JJ s7J7J##G,,#1K4HH>>IIcjjHI>>1K%&JN--)..04.. J3>>G((9K) rMrJr9rKr6 ExceptionrRr8rrrr )r`rrwrr>rs rX stream_readerZstdCompressor.stream_readervs@ DJJ(C(CD v;D !8//D11$**dC   G $ $/+g2FF %T9NN   s B// B<;B<c [US5(d [S5e[R"UR[R 5 US:a[R n[XX#XES9$)a Create a stream that will write compressed data into another stream. The argument to ``stream_writer()`` must have a ``write(data)`` method. As compressed data is available, ``write()`` will be called with the compressed data as its argument. Many common Python types implement ``write()``, including open file handles and ``io.BytesIO``. See :py:class:`ZstdCompressionWriter` for more documentation, including usage examples. :param writer: Stream to write compressed data to. :param size: Size in bytes of data to be compressed. If set, it will be used to influence compression parameter tuning and could result in the size being written into the header of the compressed data. :param write_size: How much data to ``write()`` to ``writer`` at a time. :param write_return_read: Whether ``write()`` should return the number of bytes that were consumed from the input. :param closefd: Whether to ``close`` the ``writer`` when this stream is closed. :return: :py:class:`ZstdCompressionWriter` r)must pass an object with a write() methodrrb)rrUrMrJr9rKrRr)r`r;rwr<r=r>s rX stream_writerZstdCompressor.stream_writers_Fvw''HI I DJJ(C(CD !8//D$ $,=  rWc#@# [US5(aSnO,[US5(aSnSn[U5nO [S5e[R"UR [R 5 US:a[Rn[R"UR U5n[R"U5(a[S[U5-5e[R"S5n[R"S 5n [RUlSUlSUl[R"S U5n XlXIlSU lU R"S:XdeU(aUR'U5n O#[U5W- n [)X5n XXm-n Xm- nU (dGO*[R*"U 5nXl[U5UlSUlUR"UR :a[R,"UR X[R.5n[R"U5(a[S [U5-5eU R"(a9[R0"U R$U R"5S S nSU lUv UR"UR :aMU R"S:XdeGMU R"S:Xde[R,"UR X[R25n[R"U5(a[S [U5-5eU R"(a9[R0"U R$U R"5S S nSU lUv US:Xag M7f)a Read uncompressed data from a reader and return an iterator Returns an iterator of compressed data produced from reading from ``reader``. This method provides a mechanism to stream compressed data out of a source as an iterator of data chunks. Uncompressed data will be obtained from ``reader`` by calling the ``read(size)`` method of it or by reading a slice (if ``reader`` conforms to the *buffer protocol*). The source data will be streamed into a compressor. As compressed data is available, it will be exposed to the iterator. Data is read from the source in chunks of ``read_size``. Compressed chunks are at most ``write_size`` bytes. Both values default to the zstd input and and output defaults, respectively. If reading from the source via ``read()``, ``read()`` will be called until it raises or returns an empty bytes (``b""``). It is perfectly valid for the source to deliver fewer bytes than were what requested by ``read(size)``. The caller is partially in control of how fast data is fed into the compressor by how it consumes the returned iterator. The compressor will not consume from the reader unless the caller consumes from the iterator. >>> cctx = zstandard.ZstdCompressor() >>> for chunk in cctx.read_to_iter(fh): ... # Do something with emitted data. ``read_to_iter()`` accepts a ``size`` argument declaring the size of the input stream: >>> cctx = zstandard.ZstdCompressor() >>> for chunk in cctx.read_to_iter(fh, size=some_int): >>> pass You can also control the size that data is ``read()`` from the source and the ideal size of output chunks: >>> cctx = zstandard.ZstdCompressor() >>> for chunk in cctx.read_to_iter(fh, read_size=16384, write_size=8192): >>> pass ``read_to_iter()`` does not give direct control over the sizes of chunks fed into the compressor. Instead, chunk sizes will be whatever the object being read from delivers. These will often be of a uniform size. :param reader: Stream providing data to be compressed. :param size: Size in bytes of input data. :param read_size: Controls how many bytes are ``read()`` from the source. :param write_size: Controls the output size of emitted chunks. :return: Iterator of ``bytes``. rTrFrGmust pass an object with a read() method or conforms to buffer protocolr(rr'r&rNr)rr6rUrMrJr9rKrRr8rrrrLr!rrrwr7r5rminrrrrr)r`readerrwrr< have_read buffer_offsetrrrr! read_result remaining slice_size read_bufferrs rX read_to_iterZstdCompressor.read_to_itersL 66 " "I V] + +IMv;D.  DJJ(C(CD !8//D11$**dC   G $ $/+g2FF GG-. WW/0    WWXz2 #$ >>Q& &&$kk)4 K-7  6 $]5OP + //+6K'M -INIM--)..022JJ s7J7J##G,,#1K4HH>>::jnnjnnEaHD%&JNJ--)..0>>Q& && >>Q& &&.. J3>>G((9K>> cctx = zstandard.ZstdCompressor() >>> (ingested, consumed, produced) = cctx.frame_progression() )rMZSTD_getFrameProgressionr9ingestedconsumedproduced)r` progressions rXframe_progression ZstdCompressor.frame_progression s7224::> ##[%9%9;;O;OOOrW)r9r6r)NNNNNrr)rkrlrmrnrorr7r\rrTr&rWr%r_rdrhrtrwrrqrVrWrXrrsAJWr<052n4*MC>46 x'z4 0Oj6 - d46 gR/$b PrWrc\rSrSrSrSrSrg)ri aInformation about a zstd frame. Instances have the following attributes: ``content_size`` Integer size of original, uncompressed content. This will be ``0`` if the original content size isn't written to the frame (controlled with the ``write_content_size`` argument to ``ZstdCompressor``) or if the input content size was ``0``. ``window_size`` Integer size of maximum back-reference distance in compressed data. ``dict_id`` Integer of dictionary ID used for compression. ``0`` if no dictionary ID was used or if the dictionary ID was ``0``. ``has_checksum`` Bool indicating whether a 4 byte content checksum is stored at the end of the frame. cURUlURUlURUl[ UR5Ulgrz) frameContentSize content_size windowSize window_sizedictIDdict_idr, checksumFlag has_checksum)r`fparamss rXrFrameParameters.__init__ s<#44"--~~  !5!56rW)rrrrN)rkrlrmrnrorrqrVrWrXrr s ,7rWrc[R"U5n[R"U[ U55nU[R :Xa [ S5eU[R:XagU$)zObtain the decompressed size of a frame. The returned value is usually accurate. But strictly speaking it should not be trusted. :return: ``-1`` if size unknown and a non-negative integer otherwise. z#error when determining content sizer)rLrrMZSTD_getFrameContentSizer6ZSTD_CONTENTSIZE_ERRORrrR)rrrws rXrr sY//$'K  ' ' S5E FD s)))=>> -- - rWc[R"U5n[R"U[ U55n[R "U5(a[ S[U5-5eU$)zGObtain the size of a frame header. :return: Integer size in bytes. z)could not determine frame header size: %s)rLrrMZSTD_frameHeaderSizer6rrr)rrrs rXrr s[ //$'K&&{C 4DEG    7+g:N N   NrWcD[R"S5n[R"U5n[R"X[ U55n[R "U5(a[S[U5-5eU(a[SU-5e[US5$)a Parse a zstd frame header into frame parameters. Depending on which fields are present in the frame and their values, the length of the frame parameters varies. If insufficient bytes are passed in to fully parse the frame parameters, ``ZstdError`` is raised. To ensure frame parameters can be parsed, pass in at least 18 bytes. :param data: Data from which to read frame parameters. :return: :py:class:`FrameParameters` ZSTD_frameHeader *zcannot get frame parameters: %sz3not enough data for frame parameters; need %d bytesr) rLr!rrMZSTD_getFrameHeaderr6rrrr)rrrrs rXrr sWW) *F//$'K%%f3{;KLG    - G0D D   AG K   6!9 %%rWcT\rSrSrSr\SS4SjrSrSrSr S S jr \ S 5r S r g) r i" a Represents a computed compression dictionary. Instances are obtained by calling :py:func:`train_dictionary` or by passing bytes obtained from another source into the constructor. Instances can be constructed from bytes: >>> dict_data = zstandard.ZstdCompressionDict(data) It is possible to construct a dictionary from *any* data. If the data doesn't begin with a magic header, it will be treated as a *prefix* dictionary. *Prefix* dictionaries allow compression operations to reference raw data within the dictionary. It is possible to force the use of *prefix* dictionaries or to require a dictionary header: >>> dict_data = zstandard.ZstdCompressionDict(data, dict_type=zstandard.DICT_TYPE_RAWCONTENT) >>> dict_data = zstandard.ZstdCompressionDict(data, dict_type=zstandard.DICT_TYPE_FULLDICT) You can see how many bytes are in the dictionary by calling ``len()``: >>> dict_data = zstandard.train_dictionary(size, samples) >>> dict_size = len(dict_data) # will not be larger than ``size`` Once you have a dictionary, you can pass it to the objects performing compression and decompression: >>> dict_data = zstandard.train_dictionary(131072, samples) >>> cctx = zstandard.ZstdCompressor(dict_data=dict_data) >>> for source_data in input_data: ... compressed = cctx.compress(source_data) ... # Do something with compressed data. ... >>> dctx = zstandard.ZstdDecompressor(dict_data=dict_data) >>> for compressed_data in input_data: ... buffer = io.BytesIO() ... with dctx.stream_writer(buffer) as decompressor: ... decompressor.write(compressed_data) ... # Do something with raw data in ``buffer``. Dictionaries have unique integer IDs. You can retrieve this ID via: >>> dict_id = zstandard.dictionary_id(dict_data) You can obtain the raw data in the dict (useful for persisting and constructing a ``ZstdCompressionDict`` later) via ``as_bytes()``: >>> dict_data = zstandard.train_dictionary(size, samples) >>> raw_data = dict_data.as_bytes() By default, when a ``ZstdCompressionDict`` is *attached* to a ``ZstdCompressor``, each ``ZstdCompressor`` performs work to prepare the dictionary for use. This is fine if only 1 compression operation is being performed or if the ``ZstdCompressor`` is being reused for multiple operations. But if multiple ``ZstdCompressor`` instances are being used with the dictionary, this can add overhead. It is possible to *precompute* the dictionary so it can readily be consumed by multiple ``ZstdCompressor`` instances: >>> d = zstandard.ZstdCompressionDict(data) >>> # Precompute for compression level 3. >>> d.precompute_compress(level=3) >>> # Precompute with specific compression parameters. >>> params = zstandard.ZstdCompressionParameters(...) >>> d.precompute_compress(compression_params=params) .. note:: When a dictionary is precomputed, the compression parameters used to precompute the dictionary overwrite some of the compression parameters specified to ``ZstdCompressor``. :param data: Dictionary data. :param dict_type: Type of dictionary. One of the ``DICT_TYPE_*`` constants. rc[U[5(deXlX0lX@lU[ [ [4;a [S5eX l SUl g)Nz@invalid dictionary load mode: %d; must use DICT_TYPE_* constants) isinstancebytes_datakdrFrGrHrUrDr?)r`r dict_typerrs rXrZstdCompressionDict.__init__s s\$&&&&      (  $ rWc,[UR5$rz)r6rr_s rXreZstdCompressionDict.__len__ s4::rWc|[[R"UR[ UR555$)z(Obtain the integer ID of the dictionary.)intrMZDICT_getDictIDrr6r_s rXrZstdCompressionDict.dict_id s&3&&tzz3tzz?CDDrWcUR$)z6Obtain the ``bytes`` representation of the dictionary.)rr_s rXrBZstdCompressionDict.as_bytes s zzrWNc\U(aU(a [S5eU(dU(d [S5eU(a,[R"US[UR55nO[ R "S5nURUlURUl URUl URUlURUlUR UlUR$Ul[R("UR[UR5[R*UR,U[R.5nU[ R0:Xa [3S5e[ R4"U[R6[R8"U5S9Ulg)zPrecompute a dictionary os it can be used by multiple compressors. Calling this method on an instance that will be used by multiple :py:class:`ZstdCompressor` instances will improve performance. z4must only specify one of level or compression_paramsz/must specify one of level or compression_paramsrZSTD_compressionParameterszunable to precompute dictionaryr3N)rUrMrr6rrLr!rrrrrrrrrrrrrZSTD_createCDict_advancedrCrDZSTD_defaultCMemrrrZSTD_freeCDictZSTD_sizeof_CDictr?)r`rr:cparamscdicts rXprecompute_compress'ZstdCompressionDict.precompute_compress s= 'I /NO O ))%C ODGgg:;G1;;G 099GO1;;G  2 = =G 1::G #5#C#CG 2 = =G -- JJ  O    OO     CHH => >ff 3%%C,A,A%,H  rWc[R"UR[UR5[RUR [R 5nU[R:Xa [S5e[R"U[R[R"U5S9nXRS'U$)Nz#could not create decompression dictr3_ddict)rMZSTD_createDDict_advancedrr6rCrDrrLrrrZSTD_freeDDictZSTD_sizeof_DDict__dict__)r`ddicts rXrZstdCompressionDict._ddict s-- JJ  O    OO    CHH AB B 3%%C,A,A%,H #( h rW)r?rrDrr)rN)rkrlrmrnrorFrrerrBrrprrqrVrWrXr r " sAN`(6a&E' RrWr c 8[U[5(d [S5eU S:a [5n U (d(U (d!U=(d SnU =(d Sn U =(d Sn [ [ [ U55n [SU 5n [S[ U55nSn[U5HUunn[U[5(d [S5e[ U5n[R"X-UU5 UU- nUUU'MW [SU5n[R"S 5SnUUlUUlUUlU UlU UlUUlUUlUUR*lUUR*lU UR*l[2R4"[R6"U5U[R6"U 5[R6"US5[ U5[R6"U55n[2R8"U5(aG[R:"[2R<"U55R?S 5n[AS U-5e[C[RD"UU5S S [FURURS 9$)a Train a dictionary from sample data using the COVER algorithm. A compression dictionary of size ``dict_size`` will be created from the iterable of ``samples``. The raw dictionary bytes will be returned. The dictionary training mechanism is known as *cover*. More details about it are available in the paper *Effective Construction of Relative Lempel-Ziv Dictionaries* (authors: Liao, Petri, Moffat, Wirth). The cover algorithm takes parameters ``k`` and ``d``. These are the *segment size* and *dmer size*, respectively. The returned dictionary instance created by this function has ``k`` and ``d`` attributes containing the values for these parameters. If a ``ZstdCompressionDict`` is constructed from raw bytes data (a content-only dictionary), the ``k`` and ``d`` attributes will be ``0``. The segment and dmer size parameters to the cover algorithm can either be specified manually or ``train_dictionary()`` can try multiple values and pick the best one, where *best* means the smallest compressed data size. This later mode is called *optimization* mode. Under the hood, this function always calls ``ZDICT_optimizeTrainFromBuffer_fastCover()``. See the corresponding C library documentation for more. If neither ``steps`` nor ``threads`` is defined, defaults for ``d``, ``steps``, and ``level`` will be used that are equivalent with what ``ZDICT_trainFromBuffer()`` would use. :param dict_size: Target size in bytes of the dictionary to generate. :param samples: A list of bytes holding samples the dictionary will be trained from. :param k: Segment size : constraint: 0 < k : Reasonable range [16, 2048+] :param d: dmer size : constraint: 0 < d <= k : Reasonable range [6, 16] :param f: log of size of frequency array : constraint: 0 < f <= 31 : 1 means default(20) :param split_point: Percentage of samples used for training: Only used for optimization. The first # samples * ``split_point`` samples will be used to training. The last # samples * (1 - split_point) samples will be used for testing. 0 means default (0.75), 1.0 when all samples are used for both training and testing. :param accel: Acceleration level: constraint: 0 < accel <= 10. Higher means faster and less accurate, 0 means default(1). :param dict_id: Integer dictionary ID for the produced dictionary. Default is 0, which uses a random value. :param steps: Number of steps through ``k`` values to perform when trying parameter variations. :param threads: Number of threads to use when trying parameter variations. Default is 0, which means to use a single thread. A negative value can be specified to use as many threads as there are detected logical CPUs. :param level: Integer target compression level when trying parameter variations. :param notifications: Controls writing of informational messages to ``stderr``. ``0`` (the default) means to write nothing. ``1`` writes errors. ``2`` writes progression info. ``3`` writes more details. And ``4`` writes all info. zsamples must be a listrrr&zsize_t[]zsamples must be byteszZDICT_fastCover_params_t *rzcannot train dict: %sN)rrr)$rlist TypeErrorrYsummapr6rM enumeraterrUrLr)r!rrrdsteps nbThreads splitPointaccelzParamsnotificationLevelrcompressionLevelrM'ZDICT_optimizeTrainFromBuffer_fastCover addressof ZDICT_isErrorrZDICT_getErrorNamerrr rrH)rsamplesrrrd split_pointr notificationsrrrr total_sizesamples_buffer sample_sizesrar~samplelr9dparamsrmsgs rXrr sd gt $ $011{,  F  Sg&'J :6Nz3w<8L Fw' 6&%((45 5 K N+VQ7!  Q(Hi0Igg23A6GGIGIGIGMG$GGM(5GOO%$GOO',GOO$99 i  n% lA& G  g G !!jj//89@@I/#566  9g&q)$ )) ))  rWc^\rSrSrSrSrSrS Sjr\S5r \S5r \S5r S r g ) ri^ aA standard library API compatible decompressor. This type implements a compressor that conforms to the API by other decompressors in Python's standard library. e.g. ``zlib.decompressobj`` or ``bz2.BZ2Decompressor``. This allows callers to use zstd compression while conforming to a similar API. Compressed data chunks are fed into ``decompress(data)`` and uncompressed output (or an empty bytes) is returned. Output from subsequent calls needs to be concatenated to reassemble the full decompressed byte sequence. If ``read_across_frames=False``, each instance is single use: once an input frame is decoded, ``decompress()`` will raise an exception. If ``read_across_frames=True``, instances can decode multiple frames. >>> dctx = zstandard.ZstdDecompressor() >>> dobj = dctx.decompressobj() >>> data = dobj.decompress(compressed_chunk_0) >>> data = dobj.decompress(compressed_chunk_1) By default, calls to ``decompress()`` write output data in chunks of size ``DECOMPRESSION_RECOMMENDED_OUTPUT_SIZE``. These chunks are concatenated before being returned to the caller. It is possible to define the size of these temporary chunks by passing ``write_size`` to ``decompressobj()``: >>> dctx = zstandard.ZstdDecompressor() >>> dobj = dctx.decompressobj(write_size=1048576) .. note:: Because calls to ``decompress()`` may need to perform multiple memory (re)allocations, this streaming decompression API isn't as efficient as other APIs. cDXlX lSUlX0lSUlg)NFrW) _decompressorr+r_read_across_frames _unused_input)r` decompressorr<read_across_framess rXrZstdDecompressionObj.__init__ s#)%#5  rWcUR(a [S5e[R"S5n[R"S5n[R"U5n[ U5S:XagXBl[ U5UlSUl[R"SUR5nXSl [ U5UlSUl/n[R"URRX25n[R"U5(a[S[!U5-5eUR(a=UR#[R$"URUR5S S 5 US:Xa=UR&(d,SUlS Ul XRURUlOyUS:Xa4UR&(a#URUR:XaOGSUlO>GH HGG-. WW/0 ood+ { q # [)  WWXt'7'78 #j/  //""((*G((1K4HH ~~ cjjHKL !|D$<$<"&%)"%)--)..%I"A$":":==INN2%&JN /NNZ__4!" WZxxrWcg)zEffectively a no-op. Implemented for compatibility with the standard library APIs. Safe to call at any time. :return: Empty bytes. rWrV)r`lengths rXriZstdDecompressionObj.flush srWcUR$)zBytes past the end of compressed data. If ``decompress()`` is fed additional data beyond the end of a zstd frame, this value will be non-empty once ``decompress()`` fully decodes the input frame. )rr_s rX unused_data ZstdDecompressionObj.unused_data s!!!rWcg)z5Data that has not yet been fed into the decompressor.rWrVr_s rXunconsumed_tail$ZstdDecompressionObj.unconsumed_tail srWcUR$)z?Whether the end of the compressed data stream has been reached.)rr_s rXeofZstdDecompressionObj.eof s~~rW)rrrrr+N)r) rkrlrmrnrorrrirprrrrqrVrWrXrr^ sU"H!K Z ""rWrc\rSrSrSrSSjrSrSrSrSr Sr SS jr SS jr S r S rS rSrSr\S5rSrSrSrSr\rSrSrSSjrSrSSjrSr\R@4Sjr!Sr"g) ri a` Read only decompressor that pull uncompressed data from another stream. This type provides a read-only stream interface for performing transparent decompression from another stream or data source. It conforms to the ``io.RawIOBase`` interface. Only methods relevant to reading are implemented. >>> with open(path, 'rb') as fh: >>> dctx = zstandard.ZstdDecompressor() >>> reader = dctx.stream_reader(fh) >>> while True: ... chunk = reader.read(16384) ... if not chunk: ... break ... # Do something with decompressed chunk. The stream can also be used as a context manager: >>> with open(path, 'rb') as fh: ... dctx = zstandard.ZstdDecompressor() ... with dctx.stream_reader(fh) as reader: ... ... When used as a context manager, the stream is closed and the underlying resources are released when the context manager exits. Future operations against the stream will fail. The ``source`` argument to ``stream_reader()`` can be any object with a ``read(size)`` method or any object implementing the *buffer protocol*. If the ``source`` is a stream, you can specify how large ``read()`` requests to that stream should be via the ``read_size`` argument. It defaults to ``zstandard.DECOMPRESSION_RECOMMENDED_INPUT_SIZE``.: >>> with open(path, 'rb') as fh: ... dctx = zstandard.ZstdDecompressor() ... # Will perform fh.read(8192) when obtaining data for the decompressor. ... with dctx.stream_reader(fh, read_size=8192) as reader: ... ... Instances are *partially* seekable. Absolute and relative positions (``SEEK_SET`` and ``SEEK_CUR``) forward of the current position are allowed. Offsets behind the current read position and offsets relative to the end of stream are not allowed and will raise ``ValueError`` if attempted. ``tell()`` returns the number of decompressed bytes read so far. Not all I/O methods are implemented. Notably missing is support for ``readline()``, ``readlines()``, and linewise iteration support. This is because streams operate on binary data - not text data. If you want to convert decompressed output to text, you can chain an ``io.TextIOWrapper`` to the stream: >>> with open(path, 'rb') as fh: ... dctx = zstandard.ZstdDecompressor() ... stream_reader = dctx.stream_reader(fh) ... text_stream = io.TextIOWrapper(stream_reader, encoding='utf-8') ... for line in text_stream: ... ... cXlX lX0l[U5Ul[U5UlSUlSUlSUlSUl SUl [R"S5Ul SUlgr)rrrr,rr.r/r1_bytes_decompressedrrrLr!rr)r`rrrrr>s rXr ZstdDecompressionReader.__init__7 sl* ##'(:#; W    #$ $ %''"34"rWcUR(a [S5eUR(a [S5eSUlU$rrr_s rXrE!ZstdDecompressionReader.__enter__M rrWcNSUlSUlUR5 SUlgrH)r/rrIrrJs rXrN ZstdDecompressionReader.__exit__W s$ !  rWcgrrVr_s rXru ZstdDecompressionReader.readable_ rrWcgrHrVr_s rXr ZstdDecompressionReader.writableb rsrWcgrHrVr_s rXr ZstdDecompressionReader.seekablee rsrWc,[R"5erzrQrxs rXry ZstdDecompressionReader.readlineh rVrWc,[R"5erzrQr|s rXr~!ZstdDecompressionReader.readlinesk rVrWc,[R"5erzrQrs rXrZstdDecompressionReader.writen rVrWc,[R"5erzrQrs rXr"ZstdDecompressionReader.writelinesq rVrWcgrHrVr_s rXrqZstdDecompressionReader.isattyt rsrWcgrzrVr_s rXriZstdDecompressionReader.flushw rrWcUR(agSUl[URSS5nUR(aU(aU"5 gggrrrcs rXrIZstdDecompressionReader.closez s: << DLL'4 0 ==Q C=rWcUR$rzrlr_s rXrmZstdDecompressionReader.closed rorWcUR$rz)rr_s rXrZstdDecompressionReader.tell s'''rWc/nURS5nU(dOURU5 M,SRU5$r r r s rXrZstdDecompressionReader.readall rrWc,[R"5erzrQr_s rXrT ZstdDecompressionReader.__iter__ rVrWc,[R"5erzrQr_s rXrX ZstdDecompressionReader.__next__ rVrWcURRURR:agUR(ag[ UR S5(aUR R UR5nU(dSUlg[R"U5Ul URURl [UR5URlSURlg[R"UR 5Ul URURl [UR5URlSURlgr) rr7rwrrrrrrLrrrr6rs rXr#ZstdDecompressionReader._read_input s ??  !5!5 5      4<< ( (<<$$T__5D'+$"%//$"7D "&"5"5DOO #&t':':#;DOO "#DOO "%//$,,"?D "&"5"5DOO #&t':':#;DOO "#DOO rWc[R"URRXR5nURR URR :Xaj[RURl SURlSURlSUl [URS5(dSUl [R"U5(a[S[!U5-5eUR =(a= UR UR :H=(d US:H=(a UR"(+$)zlDecompress available input into an output buffer. Returns True if data in output buffer should be emitted. rNrTzstd decompress error: %s)rMrrrrr7rwrLrrrrrrrrrr)r`rrs rX_decompress_into_buffer/ZstdDecompressionReader._decompress_into_buffer s ++    $ $j//  ??  $//"6"6 6"%((DOO "#DOO #$DOO "&D 4<<00'+$   G $ $7+g:NNO O ~~ NNjoo - -!|-,,, rWcUR(a [S5eUS:a [S5eUS:XaUR5$UR(dUS:Xag[R "SU5n[R "S5nX#lXlSUlUR5 URU5(aMU=RUR- sl [R"UR UR5SS$UR(dUR5 URU5(aMU=RUR- sl [R"UR UR5SS$UR(dMU=RUR- sl [R"UR UR5SS$NrBrrrrWr&r')r1rUrrrLr!r5rwr7rrrrrr`rwr!rs rXrZstdDecompressionReader.read sk <</0 0 "9HI I 2:<<> !  DAIWWXt, WW/0 #    ' ' 3 3  $ $  6 $::jnnjnn=a@ @&&    ++J77((JNN:(zz*..*..A!DD &&&   JNN2 zz*..*..9!<> !&&    ++J77((JNN:(!~~% &&&   JNN2 ~~rWclUR(a [S5eUS:a [S5eUR(dUS:XagUS:Xa[n[R "SU5n[R "S5nX#lXlSUlUR(dFUR5 URU5 UR(aOUR(dMFU=RUR- sl [R"UR UR5SS$r)r1rUrr(rLr!r5rwr7rrrrrrs rXr$ZstdDecompressionReader.read1 s <</0 0 "9HI I  DAI 2:8DWWXt, WW/0 #  &&      ( ( 4~~ &&&   JNN2 zz*..*..9!<>> dctx = zstandard.ZstdDecompressor() >>> decompressor = dctx.stream_writer(fh) >>> # Will call fh.write() with uncompressed data. >>> decompressor.write(compressed_data) Instances can be used as context managers. However, context managers add no extra special behavior other than automatically calling ``close()`` when they exit. Calling ``close()`` will mark the stream as closed and subsequent I/O operations will raise ``ValueError`` (per the documented behavior of ``io.RawIOBase``). ``close()`` will also call ``close()`` on the underlying stream if such a method exists and the instance was created with ``closefd=True``. The size of chunks to ``write()`` to the destination can be specified: >>> dctx = zstandard.ZstdDecompressor() >>> with dctx.stream_writer(fh, write_size=16384) as decompressor: >>> pass You can see how much memory is being used by the decompressor: >>> dctx = zstandard.ZstdDecompressor() >>> with dctx.stream_writer(fh) as decompressor: >>> byte_size = decompressor.memory_size() ``stream_writer()`` accepts a ``write_return_read`` boolean argument to control the return value of ``write()``. When ``True`` (the default)``, ``write()`` returns the number of bytes that were read from the input. When ``False``, ``write()`` returns the number of bytes that were ``write()`` to the inner stream. cUR5 XlX lX0l[ U5Ul[ U5UlSUlSUlSUl grH) _ensure_dctxrr*r+r,r-r.r/r0r1)r`rr;r<r=r>s rXr ZstdDecompressionWriter.__init__ sO !!#) %"&'8"9W     rWcUR(a [S5eUR(a [S5eSUlU$rArDr_s rXrE!ZstdDecompressionWriter.__enter__ s4 <</0 0 ===> >  rWc2SUlUR5 grH)r/rIrJs rXrN ZstdDecompressionWriter.__exit__ s  rWc,[R"5erzrQr_s rXrT ZstdDecompressionWriter.__iter__ rVrWc,[R"5erzrQr_s rXrX ZstdDecompressionWriter.__next__ rVrWcV[R"URR5$rz)rMZSTD_sizeof_DCtxrrr_s rXr\#ZstdDecompressionWriter.memory_size s##D$6$6$<$<==rWc UR(agSUlUR5 SUlSUl[URSS5nUR (aU(aU"5 ggg!SUlSUlf=frh)r1r0rirr*r.rcs rXrIZstdDecompressionWriter.close sh <<   DM JJL!DMDL DLL'4 0 ==Q C= "DMDLs A22BcUR$rzrlr_s rXrmZstdDecompressionWriter.closed rorWcb[URSS5nU(aU"5$[S5er_rarcs rXr`ZstdDecompressionWriter.fileno rfrWcUR(a [S5e[URSS5nU(aUR(dU"5$gg)NrBri)r1rUrr*r0rcs rXriZstdDecompressionWriter.flush s> <</0 0 DLL'4 0 T]]3J#1rWcgrHrVr_s rXrqZstdDecompressionWriter.isatty rsrWcgrHrVr_s rXru ZstdDecompressionWriter.readablersrWc,[R"5erzrQrxs rXry ZstdDecompressionWriter.readlinerVrWc,[R"5erzrQr|s rXr~!ZstdDecompressionWriter.readlinesrVrWNc,[R"5erzrQrs rXrZstdDecompressionWriter.seek rVrWcgrHrVr_s rXr ZstdDecompressionWriter.seekablersrWc,[R"5erzrQr_s rXrZstdDecompressionWriter.tellrVrWc,[R"5erzrQrxs rXr ZstdDecompressionWriter.truncaterVrWcgrrVr_s rXr ZstdDecompressionWriter.writablerrWc,[R"5erzrQrs rXr"ZstdDecompressionWriter.writelinesrVrWc,[R"5erzrQrxs rXrZstdDecompressionWriter.readrVrWc,[R"5erzrQr_s rXrZstdDecompressionWriter.readall rVrWc,[R"5erzrQrs rXr ZstdDecompressionWriter.readinto#rVrWcUR(a [S5eSn[R"S5n[R"S5n[R"U5nXSl[ U5UlSUl[R"SUR5nXdl [ U5UlSUlURRnURUR:a[R"XtU5n[R"U5(a[!S[#U5-5eUR(a\UR$R'[R("URUR5SS5 X$R- nSUlURUR:aMUR*(a UR$U$)NrBrrr'r&r)r1rUrLr!rrr6rwr7r+r5rrrMrrrrr*rrr-) r`rrrrrr!dctxrs rXrZstdDecompressionWriter.write&sa <</0 0 GG-. WW/0 ood+ # [)  WWXt'7'78 #j/  !!''mminn,//)LG((/+g2FF~~ ""JJz~~z~~>qA~~- !" mminn,  " "==  rW)r1r.r0rr/r-r+r*rrrz)rkrlrmrnrorrErNrTrXr\rIrprmr`rirqruryr~rrrrrrrrrrrqrVrWrXrr s.l &  ((> G(((((((((&rWrc\rSrSrSrSS\4SjrSrSS jr\ SS4S jr \ S4S jr \ \ S4S jr \ SS4S jr\ \ 4SjrSrSSjrSSjrSrg)riOaA Context for performing zstandard decompression. Each instance is essentially a wrapper around a ``ZSTD_DCtx`` from zstd's C API. An instance can compress data various ways. Instances can be used multiple times. The interface of this class is very similar to :py:class:`zstandard.ZstdCompressor` (by design). Assume that each ``ZstdDecompressor`` instance can only handle a single logical compression operation at the same time. i.e. if you call a method like ``decompressobj()`` to obtain multiple objects derived from the same ``ZstdDecompressor`` instance and attempt to use them simultaneously, errors will likely occur. If you need to perform multiple logical decompression operations and you can't guarantee those operations are temporally non-overlapping, you need to obtain multiple ``ZstdDecompressor`` instances. Unless specified otherwise, assume that no two methods of ``ZstdDecompressor`` instances can be called from multiple Python threads simultaneously. In other words, assume instances are not thread safe unless stated otherwise. :param dict_data: Compression dictionary to use. :param max_window_size: Sets an upper limit on the window size for decompression operations in kibibytes. This setting can be used to prevent large memory allocations for inputs using large compression windows. :param format: Set the format of data for the decoder. By default this is ``zstandard.FORMAT_ZSTD1``. It can be set to ``zstandard.FORMAT_ZSTD1_MAGICLESS`` to allow decoding frames without the 4 byte magic header. Not all decompression APIs support this mode. Nrc XlX lX0l[R"5nU[ R :Xa [5eX@lUR5 [ R"U[R[R"U5S9Ulg![ R"U[R[R"U5S9Ulf=f)Nr3) r6_max_window_size_formatrMZSTD_createDCtxrLrrrr0r ZSTD_freeDCtxr;)r`r9max_window_sizerr`s rXrZstdDecompressor.__init__ys# / ""$ 388 -       c''c.B.B4.HDJc''c.B.B4.HDJs B?CcB[R"UR5$)zpSize of decompression context, in bytes. >>> dctx = zstandard.ZstdDecompressor() >>> size = dctx.memory_size() )rMr;rr_s rXr\ZstdDecompressor.memory_sizerHrWFTcXU(a [S5eUR5 [R"U5n[R "U[ U55nU[R:Xa [S5eUS:XagU[R:Xa.U(d [S5e[R"SU5nUnSnO[R"SU5nUn[R"S5n Xyl Xl SU l [R"S5n XZl [ U5U l SU l [R"URX5n [R "U 5(a[S [#U 5-5eU (a [S 5eU(aU RU:wa[S X4-5eU(dAU RU R:a'U RU R- n [S U -5e[R$"XyR5S S $)a Decompress data in a single operation. This method will decompress the input data in a single operation and return the decompressed data. The input bytes are expected to contain at least 1 full Zstandard frame (something compressed with :py:meth:`ZstdCompressor.compress` or similar). If the input does not contain a full frame, an exception will be raised. ``read_across_frames`` controls whether to read multiple zstandard frames in the input. When False, decompression stops after reading the first frame. This feature is not yet implemented but the argument is provided for forward API compatibility when the default is changed to True in a future release. For now, if you need to decompress multiple frames, use an API like :py:meth:`ZstdCompressor.stream_reader` with ``read_across_frames=True``. ``allow_extra_data`` controls how to handle extra input data after a fully decoded frame. If False, any extra data (which could be a valid zstd frame) will result in ``ZstdError`` being raised. If True, extra data is silently ignored. The default will likely change to False in a future release when ``read_across_frames`` defaults to True. If the input contains extra data after a full frame, that extra input data is silently ignored. This behavior is undesirable in many scenarios and will likely be changed or controllable in a future release (see #181). If the frame header of the compressed data does not contain the content size, ``max_output_size`` must be specified or ``ZstdError`` will be raised. An allocation of size ``max_output_size`` will be performed and an attempt will be made to perform decompression into that buffer. If the buffer is too small or cannot be allocated, ``ZstdError`` will be raised. The buffer will be resized if it is too large. Uncompressed data could be much larger than compressed data. As a result, calling this function could result in a very large memory allocation being performed to hold the uncompressed data. This could potentially result in ``MemoryError`` or system memory swapping. If you don't need the full output data in a single contiguous array in memory, consider using streaming decompression for more resilient memory behavior. Usage: >>> dctx = zstandard.ZstdDecompressor() >>> decompressed = dctx.decompress(data) If the compressed data doesn't have its content size embedded within it, decompression can be attempted by specifying the ``max_output_size`` argument: >>> dctx = zstandard.ZstdDecompressor() >>> uncompressed = dctx.decompress(data, max_output_size=1048576) Ideally, ``max_output_size`` will be identical to the decompressed output size. .. important:: If the exact size of decompressed data is unknown (not passed in explicitly and not stored in the zstd frame), for performance reasons it is encouraged to use a streaming API. :param data: Compressed data to decompress. :param max_output_size: Integer max size of response. If ``0``, there is no limit and we can attempt to allocate an output buffer of infinite size. :return: ``bytes`` representing decompressed output. z?ZstdDecompressor.read_across_frames=True is not yet implementedz0error determining content size from frame headerrrWz0could not determine content size in frame headerr&r'rzdecompression error: %sz2decompression error: did not decompress full framez7decompression error: decompressed %d bytes; expected %dzFcompressed input contains %d bytes of unused data, which is disallowedN)rr0rLrrMrr6rrRr!r5rwr7rrrrrr) r`rmax_output_sizerallow_extra_datar output_size result_buffer result_sizerrrcounts rXrZstdDecompressor.decompresssf Q  ood+ 22 [)  #44 4NO O A  C88 8"F GGHo>M)KKGGHk:M%KWW/0 &% GG-. # [)  ++DJJ N   G $ $5 G8LLM M D Z^^{:I() "imminn&DNNY]]2EX  zz-8;;rWc8UR5 [XX#US9$)a Read-only stream wrapper that performs decompression. This method obtains an object that conforms to the ``io.RawIOBase`` interface and performs transparent decompression via ``read()`` operations. Source data is obtained by calling ``read()`` on a source stream or object implementing the buffer protocol. See :py:class:`zstandard.ZstdDecompressionReader` for more documentation and usage examples. :param source: Source of compressed data to decompress. Can be any object with a ``read(size)`` method or that conforms to the buffer protocol. :param read_size: Integer number of bytes to read from the source and feed into the compressor at a time. :param read_across_frames: Whether to read data across multiple zstd frames. If False, decompression is stopped at frame boundaries. :param closefd: Whether to close the source stream when this instance is closed. :return: :py:class:`zstandard.ZstdDecompressionReader`. rb)r0r)r`rrrr>s rXrdZstdDecompressor.stream_reader&s%@ & )  rWcXUS:a [S5eUR5 [XUS9$)aObtain a standard library compatible incremental decompressor. See :py:class:`ZstdDecompressionObj` for more documentation and usage examples. :param write_size: size of internal output buffer to collect decompressed chunks in. :param read_across_frames: whether to read across multiple zstd frames. If False, reading stops after 1 frame and subsequent decompress attempts will raise an exception. :return: :py:class:`zstandard.ZstdDecompressionObj` rKzwrite_size must be positive)r<r)rUr0r)r`r<rs rX decompressobjZstdDecompressor.decompressobjKs6$ >:; ; # >> dctx = zstandard.ZstdDecompressor() >>> for chunk in dctx.read_to_iter(fh): ... # Do something with original data. ``read_to_iter()`` accepts an object with a ``read(size)`` method that will return compressed bytes or an object conforming to the buffer protocol. ``read_to_iter()`` returns an iterator whose elements are chunks of the decompressed data. The size of requested ``read()`` from the source can be specified: >>> dctx = zstandard.ZstdDecompressor() >>> for chunk in dctx.read_to_iter(fh, read_size=16384): ... pass It is also possible to skip leading bytes in the input data: >>> dctx = zstandard.ZstdDecompressor() >>> for chunk in dctx.read_to_iter(fh, skip_bytes=1): ... pass .. tip:: Skipping leading bytes is useful if the source data contains extra *header* data. Traditionally, you would need to create a slice or ``memoryview`` of the data you want to decompress. This would create overhead. It is more efficient to pass the offset into this API. Similarly to :py:meth:`ZstdCompressor.read_to_iter`, the consumer of the iterator controls when data is decompressed. If the iterator isn't consumed, decompression is put on hold. When ``read_to_iter()`` is passed an object conforming to the buffer protocol, the behavior may seem similar to what occurs when the simple decompression API is used. However, this API works when the decompressed size is unknown. Furthermore, if feeding large inputs, the decompressor will work in chunks instead of performing a single operation. :param reader: Source of compressed data. Can be any object with a ``read(size)`` method or any object conforming to the buffer protocol. :param read_size: Integer size of data chunks to read from ``reader`` and feed into the decompressor. :param write_size: Integer size of data chunks to emit from iterator. :param skip_bytes: Integer number of bytes to skip over before sending data into the decompressor. :return: Iterator of ``bytes`` representing uncompressed data. z)skip_bytes must be smaller than read_sizerTrFrrkz(skip_bytes larger than first input chunkrr'r&rN)rUrr6rr0rLr!r5rwr7rlrrrMrrrrrr)r`rmrr< skip_bytesrnrorwrrr!rprqrrrsrrs rXrtZstdDecompressor.read_to_iteresH  "HI I 66 " "I V] + +IMv;D.   J'$$%OPP *  GG-. WW/0 WWXz2 #j/  >>Q& &&$kk)4  =0  6 $]5OP + //+6K'M -INIM--)..0!~~***33JJ ##G,,#3k'6JJ>>::jnnjnnEaHD%&JNJa<#--)..0( s II&#I&cT[US5(d [S5e[UUUUUS9$)a1 Push-based stream wrapper that performs decompression. This method constructs a stream wrapper that conforms to the ``io.RawIOBase`` interface and performs transparent decompression when writing to a wrapper stream. See :py:class:`zstandard.ZstdDecompressionWriter` for more documentation and usage examples. :param writer: Destination for decompressed output. Can be any object with a ``write(data)``. :param write_size: Integer size of chunks to ``write()`` to ``writer``. :param write_return_read: Whether ``write()`` should return the number of bytes of input consumed. If False, ``write()`` returns the number of bytes sent to the inner stream. :param closefd: Whether to ``close()`` the inner stream when this stream is closed. :return: :py:class:`zstandard.ZstdDecompressionWriter` rrgrb)rrUr)r`r;r<r=r>s rXrhZstdDecompressor.stream_writers9>vw''HI I&       rWc[US5(d [S5e[US5(d [S5eUR5 [R"S5n[R"S5n[R"SU5nXvlXFlSUlS upURU5n U (dX4$[R"U 5n U[U 5- nXl [U 5UlSUlURUR :a[R"URXe5n [R"U 5(a[!S [#U 5-5eUR(aOUR%[R&"UR UR55 XR- n SUlURUR :aMGMI) aL Copy data between streams, decompressing in the process. Compressed data will be read from ``ifh``, decompressed, and written to ``ofh``. >>> dctx = zstandard.ZstdDecompressor() >>> dctx.copy_stream(ifh, ofh) e.g. to decompress a file to another file: >>> dctx = zstandard.ZstdDecompressor() >>> with open(input_path, 'rb') as ifh, open(output_path, 'wb') as ofh: ... dctx.copy_stream(ifh, ofh) The size of chunks being ``read()`` and ``write()`` from and to the streams can be specified: >>> dctx = zstandard.ZstdDecompressor() >>> dctx.copy_stream(ifh, ofh, read_size=8192, write_size=16384) :param ifh: Source stream to read compressed data from. Must have a ``read()`` method. :param ofh: Destination stream to write uncompressed data to. Must have a ``write()`` method. :param read_size: The number of bytes to ``read()`` from the source in a single operation. :param write_size: The number of bytes to ``write()`` to the destination in a single operation. :return: 2-tuple of integers representing the number of bytes read and written, respectively. rrZrr[rr'r&rrr)rrUr0rLr!r5rwr7rrr6rrMrrrrrrr) r`r\r]rr<rrr!r^rrrrs rXr_ZstdDecompressor.copy_stream#s^sF##GH HsG$$IJ J GG-. WW/0 WWXz2 #$ "& 88I&D2&&///$/K #k* *J'M -INIM--)..033JJ ##G,,#5 G8LL>>IIcjjHI>>1K%&JN--)..0rWc[U[5(d [S5eU(d [S5eUSn[U[5(d [S5e[ R "U5n[ R"S5n[R"XC[U55n[R"U5(a [S5eU(a [S5eUR[R:Xa [S5eURS S 9 [ R"S UR5n[ R"S 5nXgl[U5UlSUl[ R"S 5nX8l[U5UlSUl[R&"UR(Xx5n[R"U5(a[+S[-U5-5eU(a [+S5e[U5S:Xa#[ R."U[U55SS$Sn U [U5:GaXn[U[5(d[SU -5e[ R "U5n[R"XC[U55n[R"U5(a[SU -5eU(a[SU -5eUR[R:Xa[SU -5e[ R"S UR5n Xl[U 5UlSUlX8l[U5UlSUl[R&"UR(Xx5n[R"U5(a[+S[-U5-5eU(a[+SU -5eU nU S- n U [U5:aGM[ R."U[U55SS$)a Decompress a series of frames using the content dictionary chaining technique. Such a list of frames is produced by compressing discrete inputs where each non-initial input is compressed with a *prefix* dictionary consisting of the content of the previous input. For example, say you have the following inputs: >>> inputs = [b"input 1", b"input 2", b"input 3"] The zstd frame chain consists of: 1. ``b"input 1"`` compressed in standalone/discrete mode 2. ``b"input 2"`` compressed using ``b"input 1"`` as a *prefix* dictionary 3. ``b"input 3"`` compressed using ``b"input 2"`` as a *prefix* dictionary Each zstd frame **must** have the content size written. The following Python code can be used to produce a *prefix dictionary chain*: >>> def make_chain(inputs): ... frames = [] ... ... # First frame is compressed in standalone/discrete mode. ... zctx = zstandard.ZstdCompressor() ... frames.append(zctx.compress(inputs[0])) ... ... # Subsequent frames use the previous fulltext as a prefix dictionary ... for i, raw in enumerate(inputs[1:]): ... dict_data = zstandard.ZstdCompressionDict( ... inputs[i], dict_type=zstandard.DICT_TYPE_RAWCONTENT) ... zctx = zstandard.ZstdCompressor(dict_data=dict_data) ... frames.append(zctx.compress(raw)) ... ... return frames ``decompress_content_dict_chain()`` returns the uncompressed data of the last element in the input chain. .. note:: It is possible to implement *prefix dictionary chain* decompression on top of other APIs. However, this function will likely be faster - especially for long input chains - as it avoids the overhead of instantiating and passing around intermediate objects between multiple functions. :param frames: List of ``bytes`` holding compressed zstd frames. :return: zargument must be a listzempty input chainrzchunk 0 must be bytesrz!chunk 0 is not a valid zstd framez,chunk 0 is too small to contain a zstd framez%chunk 0 missing content size in frameF) load_dictr&r'rz could not decompress chunk 0: %sz%chunk 0 did not decompress full framerKNzchunk %d must be bytesz"chunk %d is not a valid zstd framez-chunk %d is too small to contain a zstd framez&chunk %d missing content size in framez!could not decompress chunk %d: %sz&chunk %d did not decompress full frame)rrrrUrrLrr!rMrr6rrrRr0r5rwr7rrrrrr) r`framesr  chunk_bufferrr last_bufferrrr~r+s rXdecompress_content_dict_chain.ZstdDecompressor.decompress_content_dict_chainsdj&$''56 601 1q %''45 5u- -.)) #l"3    G $ $@A A KL L  " "c&B&B BDE E E*ggh(?(?@ WW/0 $k*  GG-. $ \*  ++DJJ N   G $ $2[5II CD D v;! ::k3{+;>??51L--c,&7G(( !E!IJJ CaG&&#*F*FF !IA!MNN''(F,C,CDK(N!+.JOJN(M .INIM// JG((7+g:NN H1 LMM%K FAQ#f+oTzz+s;'78;;rWc[5e)a{ Decompress multiple zstd frames to output buffers as a single operation. (Experimental. Not available in CFFI backend.) Compressed frames can be passed to the function as a ``BufferWithSegments``, a ``BufferWithSegmentsCollection``, or as a list containing objects that conform to the buffer protocol. For best performance, pass a ``BufferWithSegmentsCollection`` or a ``BufferWithSegments``, as minimal input validation will be done for that type. If calling from Python (as opposed to C), constructing one of these instances may add overhead cancelling out the performance overhead of validation for list inputs. Returns a ``BufferWithSegmentsCollection`` containing the decompressed data. All decompressed data is allocated in a single memory buffer. The ``BufferWithSegments`` instance tracks which objects are at which offsets and their respective lengths. >>> dctx = zstandard.ZstdDecompressor() >>> results = dctx.multi_decompress_to_buffer([b'...', b'...']) The decompressed size of each frame MUST be discoverable. It can either be embedded within the zstd frame or passed in via the ``decompressed_sizes`` argument. The ``decompressed_sizes`` argument is an object conforming to the buffer protocol which holds an array of 64-bit unsigned integers in the machine's native format defining the decompressed sizes of each frame. If this argument is passed, it avoids having to scan each frame for its decompressed size. This frame scanning can add noticeable overhead in some scenarios. >>> frames = [...] >>> sizes = struct.pack('=QQQQ', len0, len1, len2, len3) >>> >>> dctx = zstandard.ZstdDecompressor() >>> results = dctx.multi_decompress_to_buffer(frames, decompressed_sizes=sizes) .. note:: It is possible to pass a ``mmap.mmap()`` instance into this function by wrapping it with a ``BufferWithSegments`` instance (which will define the offsets of frames within the memory mapped region). This function is logically equivalent to performing :py:meth:`ZstdCompressor.decompress` on each input frame and returning the result. This function exists to perform decompression on multiple frames as fast as possible by having as little overhead as possible. Since decompression is performed as a single operation and since the decompressed output is stored in a single buffer, extra memory allocations, Python objects, and Python function calls are avoided. This is ideal for scenarios where callers know up front that they need to access data for multiple frames, such as when *delta chains* are being used. Currently, the implementation always spawns multiple threads when requested, even if the amount of work to do is small. In the future, it will be smarter about avoiding threads and their associated overhead when the amount of work to do is small. :param frames: Source defining zstd frames to decompress. :param decompressed_sizes: Array of integers representing sizes of decompressed zstd frames. :param threads: How many threads to use for decompression operations. Negative values will use the same number of threads as logical CPUs on the machine. Values ``0`` or ``1`` use a single thread. :return: ``BufferWithSegmentsCollection`` r])r`rdecompressed_sizesrs rXmulti_decompress_to_buffer+ZstdDecompressor.multi_decompress_to_buffersX"##rWc[R"UR[R5 UR(a][R "URUR5n[R "U5(a[S[U5-5e[R"UR[RUR5n[R "U5(a[S[U5-5eUR(apU(ah[R"URURR5n[R "U5(a[S[U5-5eggg)Nz!unable to set max window size: %sz!unable to set decoding format: %sz+unable to reference prepared dictionary: %s)rMZSTD_DCtx_resetrrKrdZSTD_DCtx_setMaxWindowSizerrrZSTD_DCtx_setParameter ZSTD_d_formatrer6ZSTD_DCtx_refDDictr)r`rrs rXr0ZstdDecompressor._ensure_dctxes* DJJ(C(CD  44 D11G((7+g:NN,, JJ))4<<    G $ $3k'6JJ  ??y,,TZZ9O9OPG((A!'*+) )?rW)rr6rerd)rFT)Nrr)rkrlrmrnrorIrr\rr'rdr(rwrtrhr_rrr0rqrVrWrXrrOs'R"&q(0  O ZSTD_greedyr? ZSTD_lazyr@ ZSTD_lazy2rA ZSTD_btlazy2rB ZSTD_btoptrC ZSTD_btultrarD ZSTD_btultra2rE ZSTD_dct_autorFZSTD_dct_rawContentrGZSTD_dct_fullDictrH ZSTD_f_zstd1rIZSTD_f_zstd1_magiclessrJrrrrrYrrrrrcrrrobjectr rrrrr r r rrrrrr rrrrrrVrWrXrsD8H T  5%(%;%;%="&)&=&=&?#'*'='='?$(+(?(?(A%? **,## " 22.. ,,&& && && $$ $$ "" "" $$ $$ && && ((((,,,,,,,,66 // ####%%""..** 33   "$$.3$3$l$$,  F"J@G@GF'  P&FP&f p p fqVqh_F_D o PVo Pd7f7<*"&>j&j`  L^W6WtD(fD(N MfM`p vp rW