Help us improve by taking our short survey: https://www.hdfgroup.org/website-survey/

| HDF5 Last Updated on 2026-03-23 The HDF5 Field Guide

Loading...
Searching...
No Matches
H5FDsubfiling.h File Reference
`#include "H5FDpublic.h"`

## Data Structures
struct | H5FD_subfiling_config_t
| Configuration structure for H5Pset_fapl_subfiling() / H5Pget_fapl_subfiling(). More...
struct | H5FD_subfiling_params_t
| Subfiling parameter structure that is shared between the H5FD_SUBFILING and H5FD_IOC drivers. More...

## Macros
#define | H5FD_SUBFILING (H5OPEN H5FD_SUBFILING_id_g)
#define | H5FD_SUBFILING_CONFIG_FILE_PREFIX "H5FD_SUBFILING_CONFIG_FILE_PREFIX"
#define | H5FD_SUBFILING_CONFIG_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 ".config"
#define | H5FD_SUBFILING_CURR_FAPL_VERSION 1
#define | H5FD_SUBFILING_DEFAULT_STRIPE_COUNT -1
#define | H5FD_SUBFILING_DEFAULT_STRIPE_SIZE (32 * 1024 * 1024)
#define | H5FD_SUBFILING_FAPL_MAGIC 0xFED01331
#define | H5FD_SUBFILING_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 "_%0*d_of_%d"
#define | H5FD_SUBFILING_IOC_PER_NODE "H5FD_SUBFILING_IOC_PER_NODE"
#define | H5FD_SUBFILING_IOC_SELECTION_CRITERIA "H5FD_SUBFILING_IOC_SELECTION_CRITERIA"
#define | H5FD_SUBFILING_NAME "subfiling"
#define | H5FD_SUBFILING_STRIPE_SIZE "H5FD_SUBFILING_STRIPE_SIZE"
#define | H5FD_SUBFILING_SUBFILE_PREFIX "H5FD_SUBFILING_SUBFILE_PREFIX"

## Enumerations
enum | H5FD_subfiling_ioc_select_t { SELECT_IOC_ONE_PER_NODE = 0 , SELECT_IOC_EVERY_NTH_RANK , SELECT_IOC_WITH_CONFIG , SELECT_IOC_TOTAL , ioc_selection_options }

## Functions
H5_DLL herr_t | H5FDsubfiling_get_file_mapping (hid_t file_id, char ***filenames, size_t *len)
| Retrieve the subfile names for an HDF5 file using the subfiling VFD.
H5_DLL herr_t | H5Pget_fapl_subfiling (hid_t fapl_id, H5FD_subfiling_config_t *config_out)
| Queries a File Access Property List for H5FD_SUBFILING file driver properties.
H5_DLL herr_t | H5Pset_fapl_subfiling (hid_t fapl_id, const H5FD_subfiling_config_t *vfd_config)
| Modifies the specified File Access Property List to use the H5FD_SUBFILING driver.

## Macro Definition Documentation

## ◆ H5FD_SUBFILING

#define H5FD_SUBFILING (H5OPEN H5FD_SUBFILING_id_g)

Macro that returns the identifier for the H5FD_SUBFILING driver. Returns a file driver identifier if successful; otherwise returns H5I_INVALID_HID.

## ◆ H5FD_SUBFILING_CONFIG_FILE_PREFIX

#define H5FD_SUBFILING_CONFIG_FILE_PREFIX "H5FD_SUBFILING_CONFIG_FILE_PREFIX"

Macro for name of the environment variable that specifies a prefix to apply to the subfiling configuration filename. Useful for cases where the application wants to place the configuration file in a different directory than the default of putting it alongside the generated subfiles. For example, when writing to node-local storage one may wish to place the configuration file on a scratch file system readable by all nodes, while the subfiles are initially written to the node-local storage.

The value set for this environment variable is interpreted as a pathname that must already exist.

NOTE: As this prefix string will be encoded in the driver info message that gets written to the file, there is an upper limit of about ~900 single-byte characters for this string, though possibly less due to other information the driver may encode. Avoid long prefix names where possible.

## ◆ H5FD_SUBFILING_CONFIG_FILENAME_TEMPLATE

#define H5FD_SUBFILING_CONFIG_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 ".config"

The basic printf-style template for a H5FD_SUBFILING driver configuration filename. The format specifiers correspond to:

%s
  base filename, e.g. "file.h5"

%PRIu64
  file inode, e.g. 11273556

yielding a filename such as:

file.h5.subfile_11273556.config

## ◆ H5FD_SUBFILING_CURR_FAPL_VERSION

#define H5FD_SUBFILING_CURR_FAPL_VERSION 1

The version number of the H5FD_subfiling_config_t configuration structure for the H5FD_SUBFILING driver

## ◆ H5FD_SUBFILING_DEFAULT_STRIPE_COUNT

#define H5FD_SUBFILING_DEFAULT_STRIPE_COUNT -1

Macro for the default Subfiling stripe count value. The default is currently to use one subfile per node.

## ◆ H5FD_SUBFILING_DEFAULT_STRIPE_SIZE

#define H5FD_SUBFILING_DEFAULT_STRIPE_SIZE (32 * 1024 * 1024)

The default stripe size (in bytes) for data stripes in subfiles

## ◆ H5FD_SUBFILING_FAPL_MAGIC

#define H5FD_SUBFILING_FAPL_MAGIC 0xFED01331

Unique number used to distinguish the H5FD_SUBFILING driver from other HDF5 file drivers

## ◆ H5FD_SUBFILING_FILENAME_TEMPLATE

#define H5FD_SUBFILING_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 "_%0*d_of_%d"

The basic printf-style template for a H5FD_SUBFILING driver subfile filename. The format specifiers correspond to:

%s
  base filename, e.g. "file.h5"

%PRIu64
  file inode, e.g. 11273556

%0*d
  number (starting at 1) signifying the Nth (out of total number of subfiles) subfile. Zero-padded according to the number of digits in the number of subfiles (calculated by log10(num_subfiles) + 1)

%d
  number of subfiles

yielding filenames such as:

file.h5.subfile_11273556_01_of_10
file.h5.subfile_11273556_02_of_10
file.h5.subfile_11273556_10_of_10

## ◆ H5FD_SUBFILING_IOC_PER_NODE

#define H5FD_SUBFILING_IOC_PER_NODE "H5FD_SUBFILING_IOC_PER_NODE"

Macro for name of the environment variable that specifies the number of MPI ranks per node to use as I/O concentrators

The value set for this environment variable is interpreted as a long value and must be > 0.

## ◆ H5FD_SUBFILING_IOC_SELECTION_CRITERIA

#define H5FD_SUBFILING_IOC_SELECTION_CRITERIA "H5FD_SUBFILING_IOC_SELECTION_CRITERIA"

Macro for name of the environment variable that provides information for selection MPI ranks as I/O concentrators

The value set for this environment variable is interpreted differently, depending on the IOC selection method chosen.

For SELECT_IOC_ONE_PER_NODE, this value is ignored.

For SELECT_IOC_EVERY_NTH_RANK, this value is interpreted as a long value and must be > 0. The value will correspond to the N value when selecting every N-th MPI rank as an I/O concentrator.

For SELECT_IOC_WITH_CONFIG, this value is ignored as that particular IOC selection method is not currently supported.

For SELECT_IOC_TOTAL, this value is interpreted as a long value and must be > 0. The value will correspond to the total number of I/O concentrators to be used.

## ◆ H5FD_SUBFILING_NAME

#define H5FD_SUBFILING_NAME "subfiling"

The canonical name for the H5FD_SUBFILING driver

## ◆ H5FD_SUBFILING_STRIPE_SIZE

#define H5FD_SUBFILING_STRIPE_SIZE "H5FD_SUBFILING_STRIPE_SIZE"

Macro for name of the environment variable that specifies the size (in bytes) for data stripes in subfiles

The value set for this environment variable is interpreted as a long long value and must be > 0.

## ◆ H5FD_SUBFILING_SUBFILE_PREFIX

#define H5FD_SUBFILING_SUBFILE_PREFIX "H5FD_SUBFILING_SUBFILE_PREFIX"

Macro for name of the environment variable that specifies a prefix to apply to the filenames generated for subfiles

The value set for this environment variable is interpreted as a pathname.

## Enumeration Type Documentation

## ◆ H5FD_subfiling_ioc_select_t

enum H5FD_subfiling_ioc_select_t

This enum defines the various constants to allow different allocations of MPI ranks as I/O concentrators.

Enumerator
SELECT_IOC_ONE_PER_NODE | Default selection method. One MPI rank per node is used as an I/O concentrator. If this selection method is used, the number of I/O concentrators per node can be adjusted with the H5FD_SUBFILING_IOC_PER_NODE environment variable.
SELECT_IOC_EVERY_NTH_RANK | Starting with MPI rank 0, a stride of 'N' is applied to the MPI rank values to determine the next I/O concentrator. The H5FD_SUBFILING_IOC_SELECTION_CRITERIA environment variable must be set to the value desired for 'N'.
SELECT_IOC_WITH_CONFIG | Currently unsupported. Use a configuration file to determine the mapping from MPI ranks to I/O concentrators. The H5FD_SUBFILING_IOC_SELECTION_CRITERIA environment variable must be set to the path to the configuration file.
SELECT_IOC_TOTAL | Specifies that a total of 'N' I/O concentrators should be used. Starting with MPI rank 0, a stride of 'MPI comm size' / 'N' is applied to the MPI rank values to determine the next I/O concentrator. The H5FD_SUBFILING_IOC_SELECTION_CRITERIA environment variable must be set to the value desired for 'N'.
ioc_selection_options | Unused. Sentinel value