File Space Management
Documentation
|
|
The purpose of this page is to list and briefly describe the
documentation available to those who will use the
file space management feature found in the
HDF5 library.
The HDF5 library's file space management activities encompass both the allocation of file
space and the management of free space. When an HDF5 object (group, dataset, etc.) is created
and written, file space is allocated for storing its metadata and raw data. When an object is
removed, the space associated with the object becomes free space.
The following are the sections on this page:
|
|
|
|
File Space Management Strategies
|
|
The HDF5 library uses three mechanisms to manage space in an HDF5 file. They
are:
-
Free-space managers that track free-space sections of various sizes in the file that
are not currently allocated.
-
Aggregators, which are contiguous blocks of free space in the file.
-
Virtual file drivers, which use the virtual file driver interface to request
additional space from the file driver associated with the file.
|
|
There are four file space-handling strategies available to users that use these
mechanisms:
|
|
H5F_FSPACE_STRATEGY_FSM_AGGR
This strategy has always been available in HDF5 and is the default.
The mechanisms used for this strategy are free-space managers, aggregators, and virtual
file drivers.
|
|
H5F_FSPACE_STRATEGY_PAGE
The current HDF5 file space allocation accumulates small pieces of metadata and raw data in
aggregator blocks. However, these blocks are not page aligned and vary widely in sizes. The
paged aggregation feature provides efficient paged access of these small pieces of metadata and
raw data. It accumulates metadata and raw data into well-aligned pages called file space pages.
The library defines a default file space page size but a user can set the page size via a new public
routine.
The mechanisms used for this strategy are free-space managers with embedded paged aggregation
and virtual file drivers.
See the RFC on this feature for complete details.
|
|
H5F_FSPACE_STRATEGY_AGGR
With this strategy the library will request space from either the metadata or raw data aggregator
depending on the file space type. If the request is not satisfied, the library will request space
from the virtual file driver.
The mechanisms used for this strategy are aggregators and virtual file drivers.
It does not use the free-space manager.
|
|
H5F_FSPACE_STRATEGY_NONE
This strategy will request space from the virtual file driver. The only mechanism used is the virtual file driver.
It does not use the free-space manager.
|
|
|
|
File Space Management User’s Guide
|
|
(This document is not yet available.)
|
|
|
HDF5 Library APIs
|
|
The APIs listed below from the HDF5 Reference Manual
provide a means for users to directly manage the file space
management feature.
|
|
| |
|
Tools
|
|
The tools listed below have been modified to preserve or modify
file freepace information appropriately when processing files
employing this feature.
|
| |
h5dump
|
When printing the file creation property information for the superblock via the
-B option, h5dump includes the block size obtained via H5Pget_file_space_page_size.
|
|
h5stat
|
When printing the file space information via the -S option, h5stat
includes the block size obtained via H5Pget_file_space_page_size.
|
|
h5repack
|
The following options were added to h5repack:
-G FS_PAGESIZE,--fs_pagesize=FS_PAGESIZE enables the file
space page size to be changed to FS_PAGESIZE.
-P FS_PERSIST,--fs_persist=FS_PERSIST sets the persisting free space to persist
(1) or to not persist (0).
-S FS_STRATEGY, --fs_strategy=FS_STRATEGY sets the file space management strategy.
-T FS_THRESHOLD, --fs_threshold=FS_THRESHOLD sets the free-space section threshold.
|
|
| |
| |
The HDF Group
