sqfs_block_processor_t Struct Reference

Abstracts generating of file data and fragment blocks. More...

#include <block_processor.h>

Inheritance diagram for sqfs_block_processor_t:

Collaboration diagram for sqfs_block_processor_t:

Public Member Functions
SQFS_API sqfs_block_processor_t *	sqfs_block_processor_create (size_t max_block_size, sqfs_compressor_t *cmp, unsigned int num_workers, size_t max_backlog, sqfs_block_writer_t *wr, sqfs_frag_table_t *tbl)
	Create a data block processor.
SQFS_API int	sqfs_block_processor_create_ex (const sqfs_block_processor_desc_t *desc, sqfs_block_processor_t **out)
	Create a data block processor.
SQFS_API int	sqfs_block_processor_begin_file (sqfs_block_processor_t *proc, sqfs_inode_generic_t **inode, void *user, sqfs_u32 flags)
	Start writing a file.
SQFS_API int	sqfs_block_processor_append (sqfs_block_processor_t *proc, const void *data, size_t size)
	Append data to the current file.
SQFS_API int	sqfs_block_processor_end_file (sqfs_block_processor_t *proc)
	Stop writing the current file and flush everything that is buffered internally.
SQFS_API int	sqfs_block_processor_submit_block (sqfs_block_processor_t *proc, void *user, sqfs_u32 flags, const void *data, size_t size)
	Submit a raw block for processing.
SQFS_API int	sqfs_block_processor_sync (sqfs_block_processor_t *proc)
	Wait for the in-flight data blocks to finish.
SQFS_API int	sqfs_block_processor_finish (sqfs_block_processor_t *proc)
	Wait for the in-flight data blocks to finish and finally flush the current fragment block.
SQFS_API const sqfs_block_processor_stats_t *	sqfs_block_processor_get_stats (const sqfs_block_processor_t *proc)
	Get accumulated runtime statistics from a block processor.

Additional Inherited Members
Static Public Member Functions inherited from sqfs_object_t
static SQFS_INLINE void	sqfs_destroy (void *obj)
	Destroy an object and free all its memory.
static SQFS_INLINE void *	sqfs_copy (const void *obj)
	Create a deep copy of an object if possible.
Data Fields inherited from sqfs_object_t
void(*	destroy )(struct sqfs_object_t *instance)
struct sqfs_object_t *(*	copy )(const struct sqfs_object_t *orig)

Detailed Description

Abstracts generating of file data and fragment blocks.

This data structure provides a simple begin/append/end interface to generate file data blocks (see sqfs_block_processor_begin_file, sqfs_block_processor_append and sqfs_block_processor_end respectively).

Internally it takes care of partitioning data in the correct block sizes, adding tail-ens to fragment blocks, compressing the data, deduplicating data and finally writing it to disk.

This object is not copyable, i.e. sqfs_copy will always return NULL.

Member Function Documentation

sqfs_block_processor_append()

SQFS_API int sqfs_block_processor_append	(	sqfs_block_processor_t *	proc,
		const void *	data,
		size_t	size
	)

Append data to the current file.

Call this after sqfs_block_processor_begin_file to add data to a file.

Parameters

proc	A pointer to a data writer object.
data	A pointer to a buffer to read data from.
size	How many bytes should be copied out of the given buffer and written to disk.

Returns

Zero on success, an SQFS_ERROR value on failure.

sqfs_block_processor_begin_file()

SQFS_API int sqfs_block_processor_begin_file	(	sqfs_block_processor_t *	proc,
		sqfs_inode_generic_t **	inode,
		void *	user,
		sqfs_u32	flags
	)

Start writing a file.

After calling this function, call sqfs_block_processor_append repeatedly to add data to the file. Finally call sqfs_block_processor_end_file when you are done. After writing all files, use sqfs_block_processor_finish to wait until all blocks that are still in flight are done and written to disk.

The specified pointer to an inode pointer is kept internally and updated with the compressed block sizes and final destinations of the file and possible fragment. Since the inode may have to be grown to larger size, the actual inode pointer can change over time. Furthermore, since there can still be blocks in-flight even after calling sqfs_block_processor_end_file, the data in the inode and the value of the pointer may still change. The only point at which the data writer is guaranteed to not touch them anymore is after sqfs_block_processor_sync or sqfs_block_processor_finish has returned.

Parameters

proc	A pointer to a data writer object.
inode	An optional pointer to a pointer to an inode. If not NULL, the block processor creates a file inode and stores a pointer to it here and keeps updating the inode as the file grows.
user	An optional user data pointer that is passed to the the sqfs_block_writer_t for each file data block.
flags	A combination of SQFS_BLK_FLAGS that can be used to micro manage how the data is processed.

Returns

Zero on success, an SQFS_ERROR value on failure.

sqfs_block_processor_create()

SQFS_API sqfs_block_processor_t * sqfs_block_processor_create	(	size_t	max_block_size,
		sqfs_compressor_t *	cmp,
		unsigned int	num_workers,
		size_t	max_backlog,
		sqfs_block_writer_t *	wr,
		sqfs_frag_table_t *	tbl
	)

Create a data block processor.

Parameters

max_block_size	The maximum size of a data block. Required for the internal scratch buffer used for compressing data.
cmp	A pointer to a compressor. If multiple worker threads are used, the deep copy function of the compressor is used to create several instances that don't interfere with each other.
num_workers	The number of worker threads to create.
max_backlog	The maximum number of blocks currently in flight. When trying to add more, enqueueing blocks until the in-flight block count drops below the threshold.
wr	A block writer to send to finished blocks to.
tbl	A fragment table to use for storing fragment and fragment block locations.

Returns

A pointer to a block processor object on success, NULL on allocation failure or on failure to create and initialize the worker threads.

sqfs_block_processor_create_ex()

SQFS_API int sqfs_block_processor_create_ex	(	const sqfs_block_processor_desc_t *	desc,
		sqfs_block_processor_t **	out
	)

Create a data block processor.

Parameters

desc	A pointer to an extensible structure that holds the description of the block processor.
out	On success, returns the pointer to the newly created block processor object.

Returns

Zero on success, an SQFS_ERROR value on failure.

sqfs_block_processor_end_file()

SQFS_API int sqfs_block_processor_end_file

(

sqfs_block_processor_t *

proc

)

Stop writing the current file and flush everything that is buffered internally.

The counter part to sqfs_block_processor_begin_file.

Even after calling this, there might still be data blocks in-flight. Use sqfs_block_processor_finish when you are done writing files to force the remaining blocks to be processed and written to disk.

Parameters

proc	A pointer to a data writer object.

Returns

Zero on success, an SQFS_ERROR value on failure.

sqfs_block_processor_finish()

SQFS_API int sqfs_block_processor_finish

(

sqfs_block_processor_t *

proc

)

Wait for the in-flight data blocks to finish and finally flush the current fragment block.

This does essentially the same as sqfs_block_processor_sync, but after syncing, it also flushes the current fragment block, even if it isn't full yet and waits for it to be completed as well.

Parameters

proc	A pointer to a block processor object.

Returns

Zero on success, an SQFS_ERROR value on failure. The failure return value can either be an error encountered during enqueueing, processing or writing to disk.

sqfs_block_processor_get_stats()

SQFS_API const sqfs_block_processor_stats_t * sqfs_block_processor_get_stats

(

const sqfs_block_processor_t *

proc

)

Get accumulated runtime statistics from a block processor.

Parameters

proc	A pointer to a data writer object.

Returns

A pointer to a sqfs_block_processor_stats_t structure.

sqfs_block_processor_submit_block()

SQFS_API int sqfs_block_processor_submit_block	(	sqfs_block_processor_t *	proc,
		void *	user,
		sqfs_u32	flags,
		const void *	data,
		size_t	size
	)

Submit a raw block for processing.

This function provides an alternative to the simple file front end to submit raw data blocks to the processor. It will throw an error if called in between sqfs_block_processor_begin_file and sqfs_block_processor_end_file.

The flags aren't sanity checked (besides being a subset of SQFS_BLK_FLAGS_ALL), so in contrast to the simple file API, you can shoot yourself in the foot as hard as you want.

If not specified otherwise through flags, the fragment block/deduplication and fragment consolidation are still in effect and sparse blocks are discarded.

Parameters

proc	A pointer to a block processor object.
user	An optional user data pointer stored with the block and passed on to the underlying sqfs_block_writer_t.
flags	A combination of SQFS_BLK_FLAGS that can be used to micro manage how the data is processed.
data	The data to write to the block.

Returns

Zero on success, an SQFS_ERROR value on failure.

sqfs_block_processor_sync()

SQFS_API int sqfs_block_processor_sync

(

sqfs_block_processor_t *

proc

)

Wait for the in-flight data blocks to finish.

Parameters

proc	A pointer to a block processor object.

Returns

Zero on success, an SQFS_ERROR value on failure. The failure return value can either be an error encountered during enqueueing, processing or writing to disk.

---

The documentation for this struct was generated from the following file:

• include/sqfs/block_processor.h