block_util.c File Reference

Utility functions to handle lzma_block. More...

#include "common.h"
#include "index.h"

Go to the source code of this file.

Functions
	LZMA_API (lzma_ret)
	LZMA_API (lzma_vli)
	Calculate approximate memory usage of easy encoder. More...

Detailed Description

Utility functions to handle lzma_block.

Definition in file block_util.c.

Function Documentation

LZMA_API() [1/2]

LZMA_API

(

lzma_ret

)

Definition at line 17 of file block_util.c.

{
    // Validate everything but Uncompressed Size and filters.
    if (lzma_block_unpadded_size(block) == 0)
        return LZMA_PROG_ERROR;
 
    const uint32_t container_size = block->header_size
            + lzma_check_size(block->check);
 
    // Validate that Compressed Size will be greater than zero.
    if (unpadded_size <= container_size)
        return LZMA_DATA_ERROR;
 
    // Calculate what Compressed Size is supposed to be.
    // If Compressed Size was present in Block Header,
    // compare that the new value matches it.
    const lzma_vli compressed_size = unpadded_size - container_size;
    if (block->compressed_size != LZMA_VLI_UNKNOWN
            && block->compressed_size != compressed_size)
        return LZMA_DATA_ERROR;
 
    block->compressed_size = compressed_size;
 
    return LZMA_OK;
}

References lzma_block::check, lzma_block::compressed_size, compressed_size, lzma_block::header_size, LZMA_DATA_ERROR, LZMA_OK, LZMA_PROG_ERROR, LZMA_VLI_UNKNOWN, and unpadded_size.

LZMA_API() [2/2]

LZMA_API

(

uint64_t

)

Calculate approximate memory usage of easy encoder.

Get the total amount of physical memory (RAM) in bytes.

Calculate approximate memory usage of multithreaded .xz encoder.

Calculate approximate decoder memory usage of a preset.

This function is a wrapper for lzma_raw_encoder_memusage().

Parameters

preset

Compression preset (level and possible flags)

Returns

Number of bytes of memory required for the given preset when encoding. If an error occurs, for example due to unsupported preset, UINT64_MAX is returned.

This function is a wrapper for lzma_raw_decoder_memusage().

Parameters

preset

Compression preset (level and possible flags)

Returns

Number of bytes of memory required to decompress a file that was compressed using the given preset. If an error occurs, for example due to unsupported preset, UINT64_MAX is returned.

Since doing the encoding in threaded mode doesn't affect the memory requirements of single-threaded decompressor, you can use lzma_easy_decoder_memusage(options->preset) or lzma_raw_decoder_memusage(options->filters) to calculate the decompressor memory requirements.

Parameters

options

Compression options

Returns

Number of bytes of memory required for encoding with the given options. If an error occurs, for example due to unsupported preset or filter chain, UINT64_MAX is returned.

Calculate approximate memory usage of easy encoder.

Get the uncompressed size of the file.

Get the total size of the file.

Get the total size of the Blocks.

Get the total size of the Stream.

Get the size of the Index field as bytes.

Get the number of Blocks.

Get the number of Streams.

Calculate the memory usage of an existing lzma_index.

On disk, the size of the Index field depends on both the number of Records stored and how big values the Records store (due to variable-length integer encoding). When the Index is kept in lzma_index structure, the memory usage depends only on the number of Records/Blocks stored in the Index(es), and in case of concatenated lzma_indexes, the number of Streams. The size in RAM is almost always significantly bigger than in the encoded form on disk.

This function calculates an approximate amount of memory needed hold the given number of Streams and Blocks in lzma_index structure. This value may vary between CPU architectures and also between liblzma versions if the internal implementation is modified.

This is a shorthand for lzma_index_memusage(lzma_index_stream_count(i), lzma_index_block_count(i)).

This returns the total number of Blocks in lzma_index. To get number of Blocks in individual Streams, use lzma_index_iter.

This is needed to verify the Backward Size field in the Stream Footer.

If multiple lzma_indexes have been combined, this works as if the Blocks were in a single Stream. This is useful if you are going to combine Blocks from multiple Streams into a single new Stream.

This doesn't include the Stream Header, Stream Footer, Stream Padding, or Index fields.

When no lzma_indexes have been combined with lzma_index_cat() and there is no Stream Padding, this function is identical to lzma_index_stream_size(). If multiple lzma_indexes have been combined, this includes also the headers of each separate Stream and the possible Stream Padding fields.

Definition at line 45 of file block_util.c.

{
    // Validate the values that we are interested in i.e. all but
    // Uncompressed Size and the filters.
    //
    // NOTE: This function is used for validation too, so it is
    // essential that these checks are always done even if
    // Compressed Size is unknown.
    if (block == NULL || block->version > 1
            || block->header_size < LZMA_BLOCK_HEADER_SIZE_MIN
            || block->header_size > LZMA_BLOCK_HEADER_SIZE_MAX
            || (block->header_size & 3)
            || !lzma_vli_is_valid(block->compressed_size)
            || block->compressed_size == 0
            || (unsigned int)(block->check) > LZMA_CHECK_ID_MAX)
        return 0;
 
    // If Compressed Size is unknown, return that we cannot know
    // size of the Block either.
    if (block->compressed_size == LZMA_VLI_UNKNOWN)
        return LZMA_VLI_UNKNOWN;
 
    // Calculate Unpadded Size and validate it.
    const lzma_vli unpadded_size = block->compressed_size
                + block->header_size
                + lzma_check_size(block->check);
 
    assert(unpadded_size >= UNPADDED_SIZE_MIN);
    if (unpadded_size > UNPADDED_SIZE_MAX)
        return 0;
 
    return unpadded_size;
}

References assert(), lzma_block::check, lzma_block::compressed_size, lzma_block::header_size, LZMA_BLOCK_HEADER_SIZE_MAX, LZMA_BLOCK_HEADER_SIZE_MIN, LZMA_CHECK_ID_MAX, lzma_vli_is_valid, LZMA_VLI_UNKNOWN, NULL, unpadded_size, UNPADDED_SIZE_MAX, UNPADDED_SIZE_MIN, and lzma_block::version.