HDF5 1.14.5
API Reference
|
Use dataset transfer properties to customize certain aspects of reading and writing datasets such as transformations, MPI-IO I/O mode, error detection, etc. These properties are not persisted with datasets, and can be adjusted at runtime before a dataset is read or written.
C Function | Purpose |
---|---|
H5Pset_btree_ratios/H5Pget_btree_ratios | Sets/gets B-tree split ratios for a dataset transfer property list. |
H5Pset_buffer | Maximum size for the type conversion buffer and the background buffer. May also supply pointers to application-allocated buffers. |
H5Pset_data_transform/H5Pget_data_transform | Sets/gets a data transform expression. |
H5Pset_dataset_io_hyperslab_selection | Sets a hyperslab file selection for a dataset I/O operation. |
H5Pset_edc_check/H5Pget_edc_check | Sets/gets whether to enable error-detection when reading a dataset. |
H5Pset_hyper_vector_size | set the number of "I/O vectors" (offset and length pairs) which are to be accumulated in memory before being issued to the lower levels of the library for reading or writing the actual data. |
H5Pset_filter_callback | Sets user-defined filter callback function. |
H5Pset_hyper_vector_size/H5Pget_hyper_vector_size | Sets/gets number of I/O vectors to be read/written in hyperslab I/O. |
H5Pset_type_conv_cb/H5Pget_type_conv_cb | Sets/gets user-defined datatype conversion callback function. |
H5Pset_vlen_mem_manager/H5Pget_vlen_mem_manager | Sets/gets the memory manager for variable-length datatype allocation in H5Dread and H5Dvlen_reclaim. |
H5Pset_dxpl_mpio/H5Pget_dxpl_mpio | Sets/gets data transfer mode. |
H5Pset_dxpl_mpio_chunk_opt | Sets a flag specifying linked-chunk I/O or multi-chunk I/O. |
H5Pset_dxpl_mpio_chunk_opt_num | Sets a numeric threshold for linked-chunk I/O. |
H5Pset_dxpl_mpio_chunk_opt_ratio | Sets a ratio threshold for collective I/O. |
H5Pset_dxpl_mpio_collective_opt | Sets a flag governing the use of independent versus collective I/O. |
H5Pget_mpio_actual_chunk_opt_mode | Gets the type of chunk optimization that HDF5 actually performed on the last parallel I/O call. |
H5Pget_mpio_actual_io_mode | Gets the type of I/O that HDF5 actually performed on the last parallel I/O call. |
H5Pget_mpio_no_collective_cause | Gets local and global causes that broke collective I/O on the last parallel I/O call. |
H5Pset_selection_io/H5Pget_selection_io | Sets/gets the selection I/O mode. |
H5Pget_no_selection_io_cause | Gets the cause for not performing selection or vector I/O on the last parallel I/O call. |
H5Pget_actual_selection_io_mode | Gets the type(s) (scalar, vector, selection) of raw data I/O performed on the last I/O call. |
H5Pset_modify_write_buf/H5Pget_modify_write_buf | Sets/gets a flag allowing the library to modify the contents of the write buffer. |
H5Pset_preserve/H5Pget_preserve | No longer available, deprecated as it no longer has any effect. |
Functions | |
H5_DLL herr_t | H5Pset_dxpl_mpio (hid_t dxpl_id, H5FD_mpio_xfer_t xfer_mode) |
Sets data transfer mode. | |
H5_DLL herr_t | H5Pget_dxpl_mpio (hid_t dxpl_id, H5FD_mpio_xfer_t *xfer_mode) |
Returns the data transfer mode. | |
H5_DLL herr_t | H5Pset_dxpl_mpio_collective_opt (hid_t dxpl_id, H5FD_mpio_collective_opt_t opt_mode) |
Sets low-level data transfer mode. | |
H5_DLL herr_t | H5Pset_dxpl_mpio_chunk_opt (hid_t dxpl_id, H5FD_mpio_chunk_opt_t opt_mode) |
Sets a flag specifying linked-chunk I/O or multi-chunk I/O. | |
H5_DLL herr_t | H5Pset_dxpl_mpio_chunk_opt_num (hid_t dxpl_id, unsigned num_chunk_per_proc) |
Sets a numeric threshold for linked-chunk I/O. | |
H5_DLL herr_t | H5Pset_dxpl_mpio_chunk_opt_ratio (hid_t dxpl_id, unsigned percent_num_proc_per_chunk) |
Sets a ratio threshold for collective I/O. | |
herr_t | H5Pget_btree_ratios (hid_t plist_id, double *left, double *middle, double *right) |
Gets B-tree split ratios for a dataset transfer property list. | |
size_t | H5Pget_buffer (hid_t plist_id, void **tconv, void **bkg) |
Reads buffer settings. | |
ssize_t | H5Pget_data_transform (hid_t plist_id, char *expression, size_t size) |
Retrieves a data transform expression. | |
H5Z_EDC_t | H5Pget_edc_check (hid_t plist_id) |
Determines whether error-detection is enabled for dataset reads. | |
herr_t | H5Pget_hyper_vector_size (hid_t fapl_id, size_t *size) |
Retrieves number of I/O vectors to be read/written in hyperslab I/O. | |
int | H5Pget_preserve (hid_t plist_id) |
Checks status of the dataset transfer property list (DEPRECATED) | |
herr_t | H5Pget_type_conv_cb (hid_t dxpl_id, H5T_conv_except_func_t *op, void **operate_data) |
Gets user-defined datatype conversion callback function. | |
herr_t | H5Pget_vlen_mem_manager (hid_t plist_id, H5MM_allocate_t *alloc_func, void **alloc_info, H5MM_free_t *free_func, void **free_info) |
Gets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim() | |
herr_t | H5Pset_btree_ratios (hid_t plist_id, double left, double middle, double right) |
Sets B-tree split ratios for a dataset transfer property list. | |
herr_t | H5Pset_buffer (hid_t plist_id, size_t size, void *tconv, void *bkg) |
Sets type conversion and background buffers. | |
herr_t | H5Pset_data_transform (hid_t plist_id, const char *expression) |
Sets a data transform expression. | |
herr_t | H5Pset_edc_check (hid_t plist_id, H5Z_EDC_t check) |
Sets the dataset transfer property list to enable or disable error detection when reading data. | |
herr_t | H5Pset_filter_callback (hid_t plist_id, H5Z_filter_func_t func, void *op_data) |
Sets user-defined filter callback function. | |
herr_t | H5Pset_hyper_vector_size (hid_t plist_id, size_t size) |
Sets number of I/O vectors to be read/written in hyperslab I/O. | |
herr_t | H5Pset_preserve (hid_t plist_id, hbool_t status) |
Sets the dataset transfer property list status . | |
herr_t | H5Pset_type_conv_cb (hid_t dxpl_id, H5T_conv_except_func_t op, void *operate_data) |
Sets user-defined datatype conversion callback function. | |
herr_t | H5Pset_vlen_mem_manager (hid_t plist_id, H5MM_allocate_t alloc_func, void *alloc_info, H5MM_free_t free_func, void *free_info) |
Sets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim() | |
herr_t | H5Pget_mpio_actual_chunk_opt_mode (hid_t plist_id, H5D_mpio_actual_chunk_opt_mode_t *actual_chunk_opt_mode) |
Retrieves the type of chunk optimization that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested) | |
herr_t | H5Pget_mpio_actual_io_mode (hid_t plist_id, H5D_mpio_actual_io_mode_t *actual_io_mode) |
Retrieves the type of I/O that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested) | |
herr_t | H5Pget_mpio_no_collective_cause (hid_t plist_id, uint32_t *local_no_collective_cause, uint32_t *global_no_collective_cause) |
Retrieves local and global causes that broke collective I/O on the last parallel I/O call. | |
herr_t | H5Pset_dataset_io_hyperslab_selection (hid_t plist_id, unsigned rank, H5S_seloper_t op, const hsize_t start[], const hsize_t stride[], const hsize_t count[], const hsize_t block[]) |
Sets a hyperslab file selection for a dataset I/O operation. | |
herr_t | H5Pset_selection_io (hid_t plist_id, H5D_selection_io_mode_t selection_io_mode) |
Sets the selection I/O mode. | |
herr_t | H5Pget_selection_io (hid_t plist_id, H5D_selection_io_mode_t *selection_io_mode) |
Retrieves the selection I/O mode. | |
herr_t | H5Pget_no_selection_io_cause (hid_t plist_id, uint32_t *no_selection_io_cause) |
Retrieves the cause for not performing selection or vector I/O on the last parallel I/O call. | |
herr_t | H5Pget_actual_selection_io_mode (hid_t plist_id, uint32_t *actual_selection_io_mode) |
Retrieves the type(s) of I/O that HDF5 actually performed on raw data during the last I/O call. | |
herr_t | H5Pset_modify_write_buf (hid_t plist_id, hbool_t modify_write_buf) |
Allows the library to modify the contents of the write buffer. | |
herr_t | H5Pget_modify_write_buf (hid_t plist_id, hbool_t *modify_write_buf) |
Retrieves the "modify write buffer" property. | |
Retrieves the type(s) of I/O that HDF5 actually performed on raw data during the last I/O call.
[in] | plist_id | Dataset transfer property list identifier |
[out] | actual_selection_io_mode | A bitwise set value indicating the type(s) of I/O performed |
H5Pget_actual_selection_io_mode() allows the user to determine which type(s) of I/O were actually performed on raw data during the last I/O operation which used plist_id
. This property is set after all I/O is completed; if I/O fails, it will not be set.
H5Pget_no_selection_io_cause() can be used to determine the reason why selection or vector I/O was not performed.
Valid bitflags returned in actual_selection_io_mode
are listed as follows.
0 or more of these can be present in actual_selection_io_mode
in a bitwise fashion, since a single operation can trigger multiple instances of I/O, possibly with different types. A value of 0
indicates no raw data I/O was performed during the operation.
Be aware that this function will only include raw data I/O performed to/from disk as part of the last I/O operation. Any metadata I/O, including attribute and compact dataset I/O, is disregarded. It is also possible that data was cached in the dataset chunk cache or sieve buffer, which may prevent I/O from hitting the disk, and thereby prevent it from being counted by this function.
Gets B-tree split ratios for a dataset transfer property list.
[in] | plist_id | Dataset transfer property list identifier |
[out] | left | The B-tree split ratio for left-most nodes |
[out] | middle | The B-tree split ratio for right-most nodes and lone nodes |
[out] | right | The B-tree split ratio for all other nodes |
H5Pget_btree_ratios() returns the B-tree split ratios for a dataset transfer property list.
The B-tree split ratios are returned through the non-NULL arguments left
, middle
, and right
, as set by the H5Pset_btree_ratios() function.
size_t H5Pget_buffer | ( | hid_t | plist_id, |
void ** | tconv, | ||
void ** | bkg | ||
) |
Reads buffer settings.
[in] | plist_id | Identifier for the dataset transfer property list |
[out] | tconv | Address of the pointer to application-allocated type conversion buffer |
[out] | bkg | Address of the pointer to application-allocated background buffer |
H5Pget_buffer() reads values previously set with H5Pset_buffer().
hsize_t
to size_t
. hsize_t
.Retrieves a data transform expression.
[in] | plist_id | Identifier of the property list or class |
[out] | expression | Pointer to memory where the transform expression will be copied |
[in] | size | Number of bytes of the transform expression to copy to |
H5Pget_data_transform() retrieves the data transform expression previously set in the dataset transfer property list plist_id
by H5Pset_data_transform().
H5Pget_data_transform() can be used to both retrieve the transform expression and query its size.
If expression
is non-NULL, up to size
bytes of the data transform expression are written to the buffer. If expression
is NULL, size
is ignored, and the function does not write anything to the buffer. The function always returns the size of the data transform expression.
If 0 is returned for the size of the expression, no data transform expression exists for the property list.
If an error occurs, the buffer pointed to by expression
is unchanged, and the function returns a negative value.
H5_DLL herr_t H5Pget_dxpl_mpio | ( | hid_t | dxpl_id, |
H5FD_mpio_xfer_t * | xfer_mode | ||
) |
Returns the data transfer mode.
[in] | dxpl_id | Dataset transfer property list identifier |
[out] | xfer_mode | Transfer mode |
H5Pget_dxpl_mpio() queries the data transfer mode currently set in the data transfer property list dxpl_id
.
Upon return, xfer_mode
contains the data transfer mode, if it is non-null.
H5Pget_dxpl_mpio() is not a collective function.
Determines whether error-detection is enabled for dataset reads.
[in] | plist_id | Dataset transfer property list identifier |
H5Z_ENABLE_EDC
or H5Z_DISABLE_EDC
if successful; otherwise returns a negative value.H5Pget_edc_check() queries the dataset transfer property list plist
to determine whether error detection is enabled for data read operations.
Retrieves number of I/O vectors to be read/written in hyperslab I/O.
[in] | fapl_id | Dataset transfer property list identifier |
[out] | size | Number of I/O vectors to accumulate in memory for I/O operations |
H5Pget_hyper_vector_size() retrieves the number of I/O vectors to be accumulated in memory before being issued to the lower levels of the HDF5 library for reading or writing the actual data.
The number of I/O vectors set in the dataset transfer property list fapl_id
is returned in size
. Unless the default value is in use, size
was previously set with a call to H5Pset_hyper_vector_size().
Retrieves the "modify write buffer" property.
[in] | plist_id | Dataset transfer property list identifier |
[out] | modify_write_buf | Whether the library can modify the contents of the write buffer |
H5Pget_modify_write_buf() gets the "modify write buffer" property from the dataset transfer property list plist_id
. This property determines whether the library is allowed to modify the contents of write buffers passed to HDF5 API routines that are passed plist_id
. The default value for modify_write_buf is false.
herr_t H5Pget_mpio_actual_chunk_opt_mode | ( | hid_t | plist_id, |
H5D_mpio_actual_chunk_opt_mode_t * | actual_chunk_opt_mode | ||
) |
Retrieves the type of chunk optimization that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested)
[in] | plist_id | Dataset transfer property list identifier |
[out] | actual_chunk_opt_mode | The type of chunk optimization performed by HDF5 |
H5Pget_mpio_actual_chunk_opt_mode() retrieves the type of chunk optimization performed when collective I/O was requested. This property is set before I/O takes place, and will be set even if I/O fails.
Valid values returned in actual_chunk_opt_mode:
(Click on a enumerator, field, or type for more information.)
herr_t H5Pget_mpio_actual_io_mode | ( | hid_t | plist_id, |
H5D_mpio_actual_io_mode_t * | actual_io_mode | ||
) |
Retrieves the type of I/O that HDF5 actually performed on the last parallel I/O call (not necessarily the type requested)
[in] | plist_id | Dataset transfer property list identifier |
[out] | actual_io_mode | The type of I/O performed by this process |
H5Pget_mpio_actual_io_mode() retrieves the type of I/O performed on the selection of the current process. This property is set after all I/O is completed; if I/O fails, it will not be set.
Valid values returned in actual_io_mode:
(Click on a enumerator, field, or type for more information.)
herr_t H5Pget_mpio_no_collective_cause | ( | hid_t | plist_id, |
uint32_t * | local_no_collective_cause, | ||
uint32_t * | global_no_collective_cause | ||
) |
Retrieves local and global causes that broke collective I/O on the last parallel I/O call.
[in] | plist_id | Dataset transfer property list identifier |
[out] | local_no_collective_cause | An enumerated set value indicating the causes that prevented collective I/O in the local process |
[out] | global_no_collective_cause | An enumerated set value indicating the causes across all processes that prevented collective I/O |
H5Pget_mpio_no_collective_cause() serves two purposes. It can be used to determine whether collective I/O was used for the last preceding parallel I/O call. If collective I/O was not used, the function retrieves the local and global causes that broke collective I/O on that parallel I/O call. The properties retrieved by this function are set before I/O takes place and are retained even when I/O fails.
Valid values returned in local_no_collective_cause
and global_no_collective_cause
are as follows or, if there are multiple causes, a bitwise OR of the relevant causes; the numbers in the center column are the bitmask values:
(Click on a enumerator, field, or type for more information.)
Retrieves the cause for not performing selection or vector I/O on the last parallel I/O call.
[in] | plist_id | Dataset transfer property list identifier |
[out] | no_selection_io_cause | A bitwise set value indicating the relevant causes that prevented selection I/O from being performed |
H5Pget_no_selection_io_cause() can be used to determine whether selection or vector I/O was applied for the last preceding I/O call. If selection or vector I/O was not used, this function retrieves the cause(s) that prevent selection or vector I/O to be performed on that I/O call. The properties retrieved by this function are set before I/O takes place and are retained even when I/O fails.
If a selection I/O request falls back to vector I/O, that is not considered "breaking" selection I/O by this function, since vector I/O still passes all information to the file driver in a single callback.
Valid values returned in no_selection_io_cause
are listed as follows. If there are multiple causes, it is a bitwise OR of the relevant causes.
int H5Pget_preserve | ( | hid_t | plist_id | ) |
Checks status of the dataset transfer property list (DEPRECATED)
[in] | plist_id | Identifier for the dataset transfer property list |
H5Pget_preserve() checks the status of the dataset transfer property list.
herr_t H5Pget_selection_io | ( | hid_t | plist_id, |
H5D_selection_io_mode_t * | selection_io_mode | ||
) |
Retrieves the selection I/O mode.
[in] | plist_id | Dataset transfer property list identifier |
[out] | selection_io_mode | The selection I/O mode |
H5Pget_selection_io() queries the selection I/O mode set in in the dataset transfer property list plist_id
.
Values returned in selection_io_mode:
(Click on a enumerator, field, or type for more information.)
herr_t H5Pget_type_conv_cb | ( | hid_t | dxpl_id, |
H5T_conv_except_func_t * | op, | ||
void ** | operate_data | ||
) |
Gets user-defined datatype conversion callback function.
[in] | dxpl_id | Dataset transfer property list identifier |
[out] | op | User-defined type conversion callback function |
[out] | operate_data | User-defined input data for the callback function |
H5Pget_type_conv_cb() gets the user-defined datatype conversion callback function op
in the dataset transfer property list dxpl_id
.
The parameter operate_data
is a pointer to user-defined input data for the callback function.
The callback function op
defines the actions an application is to take when there is an exception during datatype conversion.
Please refer to the function H5Pset_type_conv_cb() for more details.
herr_t H5Pget_vlen_mem_manager | ( | hid_t | plist_id, |
H5MM_allocate_t * | alloc_func, | ||
void ** | alloc_info, | ||
H5MM_free_t * | free_func, | ||
void ** | free_info | ||
) |
Gets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim()
[in] | plist_id | Identifier for the dataset transfer property list |
[out] | alloc_func | User's allocate routine, or NULL for system malloc |
[out] | alloc_info | Extra parameter for user's allocation routine. Contents are ignored if preceding parameter is NULL |
[out] | free_func | User's free routine, or NULL for system free |
[out] | free_info | Extra parameter for user's free routine. Contents are ignored if preceding parameter is NULL |
H5Pget_vlen_mem_manager() is the companion function to H5Pset_vlen_mem_manager(), returning the parameters set by that function.
Sets B-tree split ratios for a dataset transfer property list.
[in] | plist_id | The dataset transfer property list identifier |
[in] | left | The B-tree split ratio for left-most nodes |
[in] | middle | The B-tree split ratio for all other nodes |
[in] | right | The B-tree split ratio for right-most nodes and lone nodes |
H5Pset_btree_ratios() sets the B-tree split ratios for a dataset transfer property list. The split ratios determine what percent of children go in the first node when a node splits.
The ratio left
is used when the splitting node is the left-most node at its level in the tree; the ratio right
is used when the splitting node is the right-most node at its level; and the ratio middle
is used for all other cases.
A node that is the only node at its level in the tree uses the ratio right
when it splits.
All ratios are real numbers between 0 and 1, inclusive.
Sets type conversion and background buffers.
[in] | plist_id | Dataset transfer property list identifier |
[in] | size | Size, in bytes, of the type conversion and background buffers |
[in] | tconv | Pointer to application-allocated type conversion buffer |
[in] | bkg | Pointer to application-allocated background buffer |
Given a dataset transfer property list, H5Pset_buffer() sets the maximum size for the type conversion buffer and background buffer and optionally supplies pointers to application-allocated buffers. If the buffer size is smaller than the entire amount of data being transferred between the application and the file, and a type conversion buffer or background buffer is required, then strip mining will be used.
Note that there are minimum size requirements for the buffer. Strip mining can only break the data up along the first dimension, so the buffer must be large enough to accommodate a complete slice that encompasses all of the remaining dimensions. For example, when strip mining a 100x200x300
hyperslab of a simple data space, the buffer must be large enough to hold 1x200x300
data elements. When strip mining a 100x200x300x150
hyperslab of a simple data space, the buffer must be large enough to hold 1x200x300x150
data elements.
If tconv
and/or bkg
are null pointers, then buffers will be allocated and freed during the data transfer.
The default value for the maximum buffer is 1 MiB.
size
parameter has changed from type hsize_t to size_t
. size
parameter has changed to type hsize_t.Sets a data transform expression.
[in] | plist_id | Dataset transfer property list identifier |
[in] | expression | Pointer to the null-terminated data transform expression |
H5Pset_data_transform() sets the data transform to be used for reading and writing data. This function operates on the dataset transfer property list plist_id
.
The expression
parameter is a string containing an algebraic expression, such as (5/9.0)*(x-32)
or x*(x-5)
. When a dataset is read or written with this property list, the transform expression is applied with the x
being replaced by the values in the dataset. When reading data, the values in the file are not changed and the transformed data is returned to the user.
Data transforms can only be applied to integer or floating-point datasets. Order of operations is obeyed and the only supported operations are +, -, *, and /. Parentheses can be nested arbitrarily and can be used to change precedence. When writing data back to the dataset, the transformed data is written to the file and there is no way to recover the original values to which the transform was applied.
herr_t H5Pset_dataset_io_hyperslab_selection | ( | hid_t | plist_id, |
unsigned | rank, | ||
H5S_seloper_t | op, | ||
const hsize_t | start[], | ||
const hsize_t | stride[], | ||
const hsize_t | count[], | ||
const hsize_t | block[] | ||
) |
Sets a hyperslab file selection for a dataset I/O operation.
[in] | plist_id | Property list identifier |
[in] | rank | Number of dimensions of selection |
[in] | op | Operation to perform on current selection |
[in] | start | Offset of start of hyperslab |
[in] | stride | Hyperslab stride |
[in] | count | Number of blocks included in hyperslab |
[in] | block | Size of block in hyperslab |
H5Pset_dataset_io_hyperslab_selection() is designed to be used in conjunction with using H5S_PLIST for the file dataspace ID when making a call to H5Dread() or H5Dwrite(). When used with H5S_PLIST, the selection created by one or more calls to this routine is used for determining which dataset elements to access.
rank
is the dimensionality of the selection and determines the size of the start
, stride
, count
, and block
arrays. rank
must be between 1 and H5S_MAX_RANK, inclusive.
The op
, start
, stride
, count
, and block
parameters behave identically to their behavior for H5Sselect_hyperslab(), please see the documentation for that routine for details about their use.
H5_DLL herr_t H5Pset_dxpl_mpio | ( | hid_t | dxpl_id, |
H5FD_mpio_xfer_t | xfer_mode | ||
) |
Sets data transfer mode.
[in] | dxpl_id | Dataset transfer property list identifier |
[in] | xfer_mode | Transfer mode |
H5Pset_dxpl_mpio() sets the data transfer property list dxpl_id
to use transfer mode xfer_mode
. The property list can then be used to control the I/O transfer mode during data I/O operations.
Valid transfer modes are H5FD_MPIO_INDEPENDENT (default) and H5FD_MPIO_COLLECTIVE.
H5_DLL herr_t H5Pset_dxpl_mpio_chunk_opt | ( | hid_t | dxpl_id, |
H5FD_mpio_chunk_opt_t | opt_mode | ||
) |
Sets a flag specifying linked-chunk I/O or multi-chunk I/O.
[in] | dxpl_id | Dataset transfer property list identifier |
[in] | opt_mode | Transfer mode |
H5Pset_dxpl_mpio_chunk_opt() specifies whether I/O is to be performed as linked-chunk I/O or as multi-chunk I/O. This function overrides the HDF5 library's internal algorithm for determining which mechanism to use.
When an application uses collective I/O with chunked storage, the HDF5 library normally uses an internal algorithm to determine whether that I/O activity should be conducted as one linked-chunk I/O or as multi-chunk I/O. H5Pset_dxpl_mpio_chunk_opt() is provided so that an application can override the library's algorithm in circumstances where the library might lack the information needed to make an optimal decision.
H5Pset_dxpl_mpio_chunk_opt() works by setting one of the following flags in the parameter opt_mode:
This function works by setting a corresponding property in the dataset transfer property list dxpl_id
.
The library performs I/O in the specified manner unless it determines that the low-level MPI IO package does not support the requested behavior; in such cases, the HDF5 library will internally use independent I/O.
Use of this function is optional.
Sets a numeric threshold for linked-chunk I/O.
[in] | dxpl_id | Dataset transfer property list identifier |
[in] | num_chunk_per_proc |
H5Pset_dxpl_mpio_chunk_opt_num() sets a numeric threshold for the use of linked-chunk I/O.
The library will calculate the average number of chunks selected by each process when doing collective access with chunked storage. If the number is greater than the threshold set in num_chunk_per_proc
, the library will use linked-chunk I/O; otherwise, a separate I/O process will be invoked for each chunk (multi-chunk I/O).
H5_DLL herr_t H5Pset_dxpl_mpio_chunk_opt_ratio | ( | hid_t | dxpl_id, |
unsigned | percent_num_proc_per_chunk | ||
) |
Sets a ratio threshold for collective I/O.
[in] | dxpl_id | Dataset transfer property list identifier |
[in] | percent_num_proc_per_chunk |
H5Pset_dxpl_mpio_chunk_opt_ratio() sets a threshold for the use of collective I/O based on the ratio of processes with collective access to a dataset with chunked storage. The decision whether to use collective I/O is made on a per-chunk basis.
The library will calculate the percentage of the total number of processes, the ratio, that hold selections in each chunk. If that percentage is greater than the threshold set in percent_proc_per_chunk
, the library will do collective I/O for this chunk; otherwise, independent I/O will be done for the chunk.
H5_DLL herr_t H5Pset_dxpl_mpio_collective_opt | ( | hid_t | dxpl_id, |
H5FD_mpio_collective_opt_t | opt_mode | ||
) |
Sets low-level data transfer mode.
[in] | dxpl_id | Dataset transfer property list identifier |
[in] | opt_mode | Transfer mode |
H5Pset_dxpl_mpio_collective_opt() sets the data transfer property list dxpl_id
to use transfer mode opt_mode
when performing I/O. This allows the application to specify collective I/O at the HDF5 interface level (with the H5Pset_dxpl_mpio() API routine), while controlling whether the actual I/O is performed collectively (e.g., via MPI_File_write_at_all) or independently (e.g., via MPI_File_write_at). If the collectivity setting at the HDF5 interface level (set via H5Pset_dxpl_mpio()) is not set to H5FD_MPIO_COLLECTIVE, this setting will be ignored.
Valid transfer modes are H5FD_MPIO_COLLECTIVE_IO (default) and H5FD_MPIO_INDIVIDUAL_IO.
Sets the dataset transfer property list to enable or disable error detection when reading data.
[in] | plist_id | Dataset transfer property list identifier |
[in] | check | Specifies whether error checking is enabled or disabled for dataset read operations |
H5Pset_edc_check() sets the dataset transfer property list plist
to enable or disable error detection when reading data.
Whether error detection is enabled or disabled is specified in the check
parameter. Valid values are H5Z_ENABLE_EDC (default) and H5Z_DISABLE_EDC.
herr_t H5Pset_filter_callback | ( | hid_t | plist_id, |
H5Z_filter_func_t | func, | ||
void * | op_data | ||
) |
Sets user-defined filter callback function.
[in] | plist_id | Dataset transfer property list identifier |
[in] | func | User-defined filter callback function |
[in] | op_data | User-defined input data for the callback function |
H5Pset_filter_callback() sets the user-defined filter callback function func
in the dataset transfer property list plist_id
.
The parameter op_data
is a pointer to user-defined input data for the callback function and will be passed through to the callback function.
The callback function func
defines the actions an application is to take when a filter fails. The function prototype is as follows:
where filter
indicates which filter has failed, buf
and buf_size
are used to pass in the failed data, and op_data is the required input data for this callback function.
Valid callback function return values are H5Z_CB_FAIL and H5Z_CB_CONT.
Sets number of I/O vectors to be read/written in hyperslab I/O.
[in] | plist_id | Dataset transfer property list identifier |
[in] | size | Number of I/O vectors to accumulate in memory for I/O operations Must be greater than 1 (one) Default value: 1024 |
H5Pset_hyper_vector_size() sets the number of I/O vectors to be accumulated in memory before being issued to the lower levels of the HDF5 library for reading or writing the actual data.
The I/O vectors are hyperslab offset and length pairs and are generated during hyperslab I/O.
The number of I/O vectors is passed in size
to be set in the dataset transfer property list plist_id
. size
must be greater than 1 (one).
H5Pset_hyper_vector_size() is an I/O optimization function; increasing vector_size should provide better performance, but the library will use more memory during hyperslab I/O. The default value of size
is 1024.
Allows the library to modify the contents of the write buffer.
[in] | plist_id | Dataset transfer property list identifier |
[in] | modify_write_buf | Whether the library can modify the contents of the write buffer |
H5Pset_modify_write_buf() sets whether the library is allowed to modify the contents of write buffers passed to HDF5 API routines that are passed the dataset transfer property list plist_id
. The default value for modify_write_buf is false.
This function can be used to allow the library to perform in-place type conversion on write operations to save memory space. After making an API call with this parameter set to true, the contents of the write buffer are undefined.
Sets the dataset transfer property list status
.
[in] | plist_id | Dataset transfer property list identifier |
[in] | status | Status toggle of the dataset transfer property list |
H5Pset_preserve() sets the dataset transfer property list status to 1
or 0
.
When reading or writing compound datatypes and the destination is partially initialized and the read/write is intended to initialize the other members, one must set this property to 1
. Otherwise the I/O pipeline treats the destination datapoints as completely uninitialized.
herr_t H5Pset_selection_io | ( | hid_t | plist_id, |
H5D_selection_io_mode_t | selection_io_mode | ||
) |
Sets the selection I/O mode.
[in] | plist_id | Dataset transfer property list identifier |
[in] | selection_io_mode | The selection I/O mode to be set |
H5Pset_selection_io() sets the selection I/O mode selection_io_mode
in the dataset transfer property list plist_id
.
This can be used to enable collective I/O with type conversion, or with custom VFDs that support vector or selection I/O.
Values that can be set in selection_io_mode:
(Click on a enumerator, field, or type for more information.)
When used with type conversion, selection I/O requires the type conversion buffer (and the background buffer if applicable) be large enough to hold the entirety of the data involved in the I/O. For read operations, the library will use the application's read buffer as the type conversion buffer if the memory type is not smaller than the file type, eliminating the need for a separate type conversion buffer (a background buffer may still be required). For write operations, the library will similarly use the write buffer as a type conversion buffer, but only if H5Pset_modify_write_buf() is used to allow the library to modify the contents of the write buffer.
herr_t H5Pset_type_conv_cb | ( | hid_t | dxpl_id, |
H5T_conv_except_func_t | op, | ||
void * | operate_data | ||
) |
Sets user-defined datatype conversion callback function.
[in] | dxpl_id | Dataset transfer property list identifier |
[in] | op | User-defined type conversion callback function |
[in] | operate_data | User-defined input data for the callback function |
H5Pset_type_conv_cb() sets the user-defined datatype conversion callback function op
in the dataset transfer property list dxpl_id
The parameter operate_data is a pointer to user-defined input data for the callback function and will be passed through to the callback function.
The callback function op
defines the actions an application is to take when there is an exception during datatype conversion. The function prototype is as follows:
herr_t H5Pset_vlen_mem_manager | ( | hid_t | plist_id, |
H5MM_allocate_t | alloc_func, | ||
void * | alloc_info, | ||
H5MM_free_t | free_func, | ||
void * | free_info | ||
) |
Sets the memory manager for variable-length datatype allocation in H5Dread() and H5Dvlen_reclaim()
[in] | plist_id | Dataset transfer property list identifier |
[in] | alloc_func | User's allocate routine, or NULL for system malloc |
[in] | alloc_info | Extra parameter for user's allocation routine. Contents are ignored if preceding parameter is NULL . |
[in] | free_func | User's free routine, or NULL for system free |
[in] | free_info | Extra parameter for user's free routine. Contents are ignored if preceding parameter is NULL |
H5Pset_vlen_mem_manager() sets the memory manager for variable-length datatype allocation in H5Dread() and free in H5Dvlen_reclaim().
The alloc_func
and free_func
parameters identify the memory management routines to be used. If the user has defined custom memory management routines, alloc_func
and/or free_func should be set to make those routine calls (i.e., the name of the routine is used as the value of the parameter); if the user prefers to use the system's malloc
and/or free
, the alloc_func
and free_func
parameters, respectively, should be set to NULL
The prototypes for these user-defined functions are as follows:
The alloc_info
and free_info
parameters can be used to pass along any required information to the user's memory management routines.
In summary, if the user has defined custom memory management routines, the name(s) of the routines are passed in the alloc_func
and free_func
parameters and the custom routines' parameters are passed in the alloc_info
and free_info
parameters. If the user wishes to use the system malloc
and free
functions, the alloc_func
and/or free_func
parameters are set to NULL
and the alloc_info
and free_info
parameters are ignored.