Fine-tuning the Metadata Cache Documentation

This page briefly describes the documentation available for those who want to fine-tune how the metadata cache in HDF5 behaves.

HDF5 Library APIs

H5Pget_evict_on_close	Retrieves the property list setting that determines whether an HDF5 object will be evicted from the library’s metadata cache when it is closed.
H5Pset_evict_on_close	Controls the library’s behavior of evicting metadata associated with a closed object.
H5Pget_mdc_image_config	Retrieves the metadata cache image option setting from a file access property list.
H5Pset_mdc_image_config	Sets the metadata cache image option for a file access property list.
H5Fget_mdc_image_info	Gets information about a metadata cache image for a file if it exists.
H5Fget_metadata_read_retry_info	Retrieves the collection of read retries for metadata items with checksum.
H5Pget_metadata_read_attempts	Retrieves the number of read attempts from a file access property list.
H5Pset_metadata_read_attempts	Sets the number of read attempts in a file access property list.
H5Dflush	Causes all buffers associated with a dataset to be immediately written to disk without removing the data from the cache.
H5Drefresh	Causes all buffers associated with a dataset to be cleared and immediately re-loaded with updated contents from disk storage.
H5Gflush	Causes all buffers associated with a group to be immediately flushed to disk without removing the data from the cache.
H5Grefresh	Causes all buffers associated with a group to be cleared and immediately re-loaded with updated contents from disk storage.
H5Oflush	Causes all buffers associated with an object to be immediately flushed to disk without removing the data from the cache.
H5Orefresh	Causes all buffers associated with an object to be cleared and immediately re-loaded with updated contents from disk storage.
H5Tflush	Causes all buffers associated with a committed datatype to be immediately flushed to disk without removing the data from the cache.
H5Trefresh	Causes all buffers associated with a committed datatype to be cleared and immediately re-loaded with updated contents from disk storage.
H5Fget_intent	Determines the read/write or read-only status of a file.

Logging APIs

H5Pset_mdc_log_options	Sets metadata cache logging options.
H5Pget_mdc_log_options	Gets metadata cache logging options.
H5Fstart_mdc_logging	Starts logging metadata cache events if logging was previously enabled.
H5Fstop_mdc_logging	Stops logging metadata cache events if logging was previously enabled and is currently ongoing.
H5Fget_mdc_logging_status	Gets the current metadata cache logging status.

Fine-tuning the Metadata Cache Design Documents and Information

The design documents listed below provide background information for the functions that have been developed to provide better control of the metadata cache and why better control might be useful.

Fine-grained Control of Metadata Cache Flushes (PDF) - This document describes some new functions which allow dynamic, fine-grained flush control of the entire metadata cache and the caches related to individual HDF5 objects.

Read Attempts for Metadata with Checksum (PDF) - The HDF5 Library encounters intermittent checksum failures from reading metadata of an HDF5 file opened with single-writer/multiple-reader (SWMR) access. The failure is from a checksum check that does not reflect the data read. This document describes the modifications to the library that address this problem.

Metadata Cache Image (PDF) - This document describes the metadata cache image feature and how it improves performance by writing the metadata cache in a single block on file close, and then populating the cache with the contents of this block on file open. This avoids the many small I/O operations that would otherwise be required on file open and close.

Metadata Cache Evict on Close - This feature will cause all metadata for an object to be evicted from the cache as long as metadata is not referenced from any other open object. This can relieve memory pressure on an application or system caused by a large amount of metadata in the metadata cache. See the H5Pget_evict_on_close and H5Pset_evict_on_close APIs for more details.

Metadata Cache Logging (PDF) - This document describes the improvements made to the library’s metadata cache logging facility. The changes include APIs that can be used to control logging operations and the format of log messages.

Flush Dependency Testing (PDF) - This document considers how to design tests for flush dependencies of metadata cache items for single-writer/multiple-reader (SWMR) access.

Tools

The following tool was modified to support the Metadata Cache Image feature.

h5clear The “-m” and “–image” flags were added to h5clear to close a cache image if it exists.