filter.h File Reference

Common filter related types and functions. More...

Go to the source code of this file.

Classes
struct	lzma_filter
	Filter options. More...

Macros
#define	LZMA_FILTERS_MAX   4
	Maximum number of filters in a chain. More...

Functions
	LZMA_API (lzma_bool) lzma_filter_encoder_is_supported(lzma_vli id) lzma_nothrow lzma_attr_const
	Test if the given Filter ID is supported for encoding. More...
	LZMA_API (lzma_ret) lzma_filters_copy(const lzma_filter *src
	Copy the filters array. More...
	LZMA_API (uint64_t) lzma_raw_encoder_memusage(const lzma_filter *filters) lzma_nothrow lzma_attr_pure
	Calculate approximate memory requirements for raw encoder. More...

Variables
lzma_filter *	dest
lzma_filter const lzma_allocator *allocator	lzma_nothrow
const lzma_filter *filters lzma_nothrow	lzma_attr_warn_unused_result
const lzma_allocator *	allocator
const lzma_allocator const uint8_t *	in
const lzma_allocator const uint8_t size_t	in_size
const lzma_allocator const uint8_t size_t uint8_t *	out
const lzma_allocator const uint8_t size_t uint8_t size_t *	out_pos
const lzma_allocator const uint8_t size_t *	in_pos
const lzma_allocator const uint8_t *	props

Detailed Description

Common filter related types and functions.

Definition in file filter.h.

Macro Definition Documentation

LZMA_FILTERS_MAX

#define LZMA_FILTERS_MAX   4

Maximum number of filters in a chain.

A filter chain can have 1-4 filters, of which three are allowed to change the size of the data. Usually only one or two filters are needed.

Definition at line 26 of file filter.h.

Function Documentation

LZMA_API() [1/3]

LZMA_API

(

lzma_bool

)

const

Test if the given Filter ID is supported for encoding.

Test if the given Filter ID is supported for decoding.

Return true if the give Filter ID is supported for encoding by this liblzma build. Otherwise false is returned.

There is no way to list which filters are available in this particular liblzma version and build. It would be useless, because the application couldn't know what kind of options the filter would need.

Return true if the give Filter ID is supported for decoding by this liblzma build. Otherwise false is returned.

Test if the given Filter ID is supported for encoding.

Test if the given Check ID is supported.

Locate a Block.

Parameters

iter	Iterator initialized with lzma_index_iter_init()
mode	Specify what kind of information the caller wants to get. See lzma_index_iter_mode for details.

Returns

If next Block or Stream matching the mode was found, *iter is updated and this function returns false. If no Block or Stream matching the mode is found, *iter is not modified and this function returns true. If mode is set to an unknown value, *iter is not modified and this function returns true.

If it is possible to seek in the .xz file, it is possible to parse the Index field(s) and use lzma_index_iter_locate() to do random-access reading with granularity of Block size.

Parameters

iter	Iterator that was earlier initialized with lzma_index_iter_init().
target	Uncompressed target offset which the caller would like to locate from the Stream

If the target is smaller than the uncompressed size of the Stream (can be checked with lzma_index_uncompressed_size()):

• Information about the Stream and Block containing the requested uncompressed offset is stored into *iter.
• Internal state of the iterator is adjusted so that lzma_index_iter_next() can be used to read subsequent Blocks or Streams.
• This function returns false.

If target is greater than the uncompressed size of the Stream, *iter is not modified, and this function returns true.

Test if the given Filter ID is supported for encoding.

Test if the given Check ID is supported.

Set a compression preset to lzma_options_lzma structure.

Test if given compression mode is supported.

Return true if the given match finder is supported by this liblzma build. Otherwise false is returned. It is safe to call this with a value that isn't listed in lzma_match_finder enumeration; the return value will be false.

There is no way to list which match finders are available in this particular liblzma version and build. It would be useless, because a new match finder, which the application developer wasn't aware, could require giving additional options to the encoder that the older match finders don't need.

Return true if the given compression mode is supported by this liblzma build. Otherwise false is returned. It is safe to call this with a value that isn't listed in lzma_mode enumeration; the return value will be false.

There is no way to list which modes are available in this particular liblzma version and build. It would be useless, because a new compression mode, which the application developer wasn't aware, could require giving additional options to the encoder that the older modes don't need.

0 is the fastest and 9 is the slowest. These match the switches -0 .. -9 of the xz command line tool. In addition, it is possible to bitwise-or flags to the preset. Currently only LZMA_PRESET_EXTREME is supported. The flags are defined in container.h, because the flags are used also with lzma_easy_encoder().

The preset values are subject to changes between liblzma versions.

This function is available only if LZMA1 or LZMA2 encoder has been enabled when building liblzma.

Returns

On success, false is returned. If the preset is not supported, true is returned.

Definition at line 514 of file index.c.

{
    const lzma_index *i = iter->internal[ITER_INDEX].p;
 
    // If the target is past the end of the file, return immediately.
    if (i->uncompressed_size <= target)
        return true;
 
    // Locate the Stream containing the target offset.
    const index_stream *stream = index_tree_locate(&i->streams, target);
    assert(stream != NULL);
    target -= stream->node.uncompressed_base;
 
    // Locate the group containing the target offset.
    const index_group *group = index_tree_locate(&stream->groups, target);
    assert(group != NULL);
 
    // Use binary search to locate the exact Record. It is the first
    // Record whose uncompressed_sum is greater than target.
    // This is because we want the rightmost Record that fullfills the
    // search criterion. It is possible that there are empty Blocks;
    // we don't want to return them.
    size_t left = 0;
    size_t right = group->last;
 
    while (left < right) {
        const size_t pos = left + (right - left) / 2;
        if (group->records[pos].uncompressed_sum <= target)
            left = pos + 1;
        else
            right = pos;
    }
 
    iter->internal[ITER_STREAM].p = stream;
    iter->internal[ITER_GROUP].p = group;
    iter->internal[ITER_RECORD].s = left;
 
    iter_set_info(iter);
 
    return false;
}

LZMA_API() [2/3]

LZMA_API

(

lzma_ret

)

const

Copy the filters array.

Decode Filter Flags from given buffer.

Encode Filter Flags into given buffer.

Calculate encoded size of a Filter Flags field.

Decode the Filter Properties field.

Encode the Filter Properties field.

Get the size of the Filter Properties field.

Single-call raw decoder.

Single-call raw encoder.

Update the filter chain in the encoder.

Initialize raw decoder.

Initialize raw encoder.

Copy the Filter IDs and filter-specific options from src to dest. Up to LZMA_FILTERS_MAX filters are copied, plus the terminating .id == LZMA_VLI_UNKNOWN. Thus, dest should have at least LZMA_FILTERS_MAX + 1 elements space unless the caller knows that src is smaller than that.

Unless the filter-specific options is NULL, the Filter ID has to be supported by liblzma, because liblzma needs to know the size of every filter-specific options structure. The filter-specific options are not validated. If options is NULL, any unsupported Filter IDs are copied without returning an error.

Old filter-specific options in dest are not freed, so dest doesn't need to be initialized by the caller in any way.

If an error occurs, memory possibly already allocated by this function is always freed.

Returns

- LZMA_OK

• LZMA_MEM_ERROR
• LZMA_OPTIONS_ERROR: Unsupported Filter ID and its options is not NULL.
• LZMA_PROG_ERROR: src or dest is NULL.

This function may be useful when implementing custom file formats.

Parameters

strm	Pointer to properly prepared lzma_stream
filters	Array of lzma_filter structures. The end of the array must be marked with .id = LZMA_VLI_UNKNOWN.

The ‘action’ with lzma_code() can be LZMA_RUN, LZMA_SYNC_FLUSH (if the filter chain supports it), or LZMA_FINISH.

Returns

- LZMA_OK

• LZMA_MEM_ERROR
• LZMA_OPTIONS_ERROR
• LZMA_PROG_ERROR

The initialization of raw decoder goes similarly to raw encoder.

The ‘action’ with lzma_code() can be LZMA_RUN or LZMA_FINISH. Using LZMA_FINISH is not required, it is supported just for convenience.

Returns

- LZMA_OK

• LZMA_MEM_ERROR
• LZMA_OPTIONS_ERROR
• LZMA_PROG_ERROR

This function is for advanced users only. This function has two slightly different purposes:

• After LZMA_FULL_FLUSH when using Stream encoder: Set a new filter chain, which will be used starting from the next Block.
• After LZMA_SYNC_FLUSH using Raw, Block, or Stream encoder: Change the filter-specific options in the middle of encoding. The actual filters in the chain (Filter IDs) cannot be changed. In the future, it might become possible to change the filter options without using LZMA_SYNC_FLUSH.

While rarely useful, this function may be called also when no data has been compressed yet. In that case, this function will behave as if LZMA_FULL_FLUSH (Stream encoder) or LZMA_SYNC_FLUSH (Raw or Block encoder) had been used right before calling this function.

Returns

- LZMA_OK

• LZMA_MEM_ERROR
• LZMA_MEMLIMIT_ERROR
• LZMA_OPTIONS_ERROR
• LZMA_PROG_ERROR

Parameters

filters	Array of lzma_filter structures. The end of the array must be marked with .id = LZMA_VLI_UNKNOWN.
allocator	lzma_allocator for custom allocator functions. Set to NULL to use malloc() and free().
in	Beginning of the input buffer
in_size	Size of the input buffer
out	Beginning of the output buffer
out_pos	The next byte will be written to out[*out_pos]. *out_pos is updated only if encoding succeeds.
out_size	Size of the out buffer; the first byte into which no data is written to is out[out_size].

Returns

- LZMA_OK: Encoding was successful.

• LZMA_BUF_ERROR: Not enough output buffer space.
• LZMA_OPTIONS_ERROR
• LZMA_MEM_ERROR
• LZMA_DATA_ERROR
• LZMA_PROG_ERROR

Note

There is no function to calculate how big output buffer would surely be big enough. (lzma_stream_buffer_bound() works only for lzma_stream_buffer_encode(); raw encoder won't necessarily meet that bound.)

Parameters

filters	Array of lzma_filter structures. The end of the array must be marked with .id = LZMA_VLI_UNKNOWN.
allocator	lzma_allocator for custom allocator functions. Set to NULL to use malloc() and free().
in	Beginning of the input buffer
in_pos	The next byte will be read from in[*in_pos]. *in_pos is updated only if decoding succeeds.
in_size	Size of the input buffer; the first byte that won't be read is in[in_size].
out	Beginning of the output buffer
out_pos	The next byte will be written to out[*out_pos]. *out_pos is updated only if encoding succeeds.
out_size	Size of the out buffer; the first byte into which no data is written to is out[out_size].

This function may be useful when implementing custom file formats using the raw encoder and decoder.

Parameters

size	Pointer to uint32_t to hold the size of the properties
filter	Filter ID and options (the size of the properties may vary depending on the options)

Returns

- LZMA_OK

• LZMA_OPTIONS_ERROR
• LZMA_PROG_ERROR

Note

This function validates the Filter ID, but does not necessarily validate the options. Thus, it is possible that this returns LZMA_OK while the following call to lzma_properties_encode() returns LZMA_OPTIONS_ERROR.

Parameters

filter	Filter ID and options
props	Buffer to hold the encoded options. The size of buffer must have been already determined with lzma_properties_size().

Returns

- LZMA_OK

• LZMA_OPTIONS_ERROR
• LZMA_PROG_ERROR

Note

Even this function won't validate more options than actually necessary. Thus, it is possible that encoding the properties succeeds but using the same options to initialize the encoder will fail.

If lzma_properties_size() indicated that the size of the Filter Properties field is zero, calling lzma_properties_encode() is not required, but it won't do any harm either.

Parameters

filter	filter->id must have been set to the correct Filter ID. filter->options doesn't need to be initialized (it's not freed by this function). The decoded options will be stored in filter->options; it's application's responsibility to free it when appropriate. filter->options is set to NULL if there are no properties or if an error occurs.
allocator	Custom memory allocator used to allocate the options. Set to NULL to use the default malloc(), and in case of an error, also free().
props	Input buffer containing the properties.
props_size	Size of the properties. This must be the exact size; giving too much or too little input will return LZMA_OPTIONS_ERROR.

Returns

- LZMA_OK

• LZMA_OPTIONS_ERROR
• LZMA_MEM_ERROR

Knowing the size of Filter Flags is useful to know when allocating memory to hold the encoded Filter Flags.

Parameters

size	Pointer to integer to hold the calculated size
filter	Filter ID and associated options whose encoded size is to be calculated

Returns

- LZMA_OK: *size set successfully. Note that this doesn't guarantee that filter->options is valid, thus lzma_filter_flags_encode() may still fail.

• LZMA_OPTIONS_ERROR: Unknown Filter ID or unsupported options.
• LZMA_PROG_ERROR: Invalid options

Note

If you need to calculate size of List of Filter Flags, you need to loop over every lzma_filter entry.

In contrast to some functions, this doesn't allocate the needed buffer. This is due to how this function is used internally by liblzma.

Parameters

filter	Filter ID and options to be encoded
out	Beginning of the output buffer
out_pos	out[*out_pos] is the next write position. This is updated by the encoder.
out_size	out[out_size] is the first byte to not write.

Returns

- LZMA_OK: Encoding was successful.

• LZMA_OPTIONS_ERROR: Invalid or unsupported options.
• LZMA_PROG_ERROR: Invalid options or not enough output buffer space (you should have checked it with lzma_filter_flags_size()).

The decoded result is stored into *filter. The old value of filter->options is not free()d.

Returns

- LZMA_OK

• LZMA_OPTIONS_ERROR
• LZMA_MEM_ERROR
• LZMA_PROG_ERROR

LZMA_API() [3/3]

LZMA_API

(

uint64_t

)

const

Calculate approximate memory requirements for raw encoder.

Calculate approximate memory requirements for raw decoder.

This function can be used to calculate the memory requirements for Block and Stream encoders too because Block and Stream encoders don't need significantly more memory than raw encoder.

Parameters

filters

Array of filters terminated with .id == LZMA_VLI_UNKNOWN.

Returns

Number of bytes of memory required for the given filter chain when encoding. If an error occurs, for example due to unsupported filter chain, UINT64_MAX is returned.

This function can be used to calculate the memory requirements for Block and Stream decoders too because Block and Stream decoders don't need significantly more memory than raw decoder.

Parameters

filters

Array of filters terminated with .id == LZMA_VLI_UNKNOWN.

Returns

Number of bytes of memory required for the given filter chain when decoding. If an error occurs, for example due to unsupported filter chain, UINT64_MAX is returned.

Calculate approximate memory requirements for raw encoder.

Calculate Unpadded Size.

Get the memory usage of decoder filter chain.

Calculate CRC64 using the polynomial from the ECMA-182 standard.

This function is used similarly to lzma_crc32(). See its documentation.

Definition at line 131 of file block_util.c.

{
    lzma_vli unpadded_size = lzma_block_unpadded_size(block);
 
    if (unpadded_size != LZMA_VLI_UNKNOWN)
        unpadded_size = vli_ceil4(unpadded_size);
 
    return unpadded_size;
}

Variable Documentation

allocator

const lzma_allocator* allocator

Definition at line 260 of file filter.h.

dest

lzma_filter* dest

Definition at line 120 of file filter.h.

in

const lzma_allocator const uint8_t* in

Definition at line 261 of file filter.h.

in_pos

const lzma_allocator const uint8_t size_t* in_pos

Definition at line 285 of file filter.h.

in_size

const lzma_allocator const uint8_t size_t size_t in_size

Definition at line 261 of file filter.h.

lzma_attr_warn_unused_result

const lzma_allocator const uint8_t size_t size_t in_size lzma_nothrow lzma_attr_warn_unused_result

Definition at line 181 of file filter.h.

lzma_nothrow

const lzma_allocator const uint8_t size_t props_size lzma_nothrow

Definition at line 121 of file filter.h.

out

uint8_t* out

Definition at line 261 of file filter.h.

out_pos

uint8_t size_t* out_pos

Definition at line 262 of file filter.h.

props

const lzma_allocator const uint8_t* props

Definition at line 362 of file filter.h.

Referenced by lzma_delta_props_decode(), lzma_lzma_props_decode(), and lzma_simple_props_decode().