Help us improve by taking our short survey: https://www.hdfgroup.org/website-survey/

| HDF5 Last Updated on 2026-03-23 The HDF5 Field Guide

Loading...
Searching...
No Matches
HDF5 VOL Data Mapping

Navigate back: Main / HDF5 User Guide

The HDF5 Data Mapping can only be used with the HDF5 VOL connectors that implement map objects. The native HDF5 library does not support this feature.

# HDF5 Map Object

## Introduction

While the HDF5 data model provides a flexible way to store structured data, some applications require a more general mechanism for indexing information with arbitrary keys. HDF5 Map objects address this need by providing application-defined key-value stores where key-value pairs can be added, retrieved by key, and iterated over.

Map objects are available only through VOL connectors that implement the map interface. The native HDF5 file format does not support map objects. VOL connectors with map support include DAOS and other storage systems optimized for key-value operations.

See also
  VOL Mapping (H5M) Reference Manual

## Map Object Life Cycle

Map objects follow a well-defined life cycle from creation through cleanup:

1. Creation

   - Create a new map with H5Mcreate() or H5Mcreate_anon():
     - H5Mcreate() creates a map and links it into the file hierarchy with a specified name
     - H5Mcreate_anon() creates an anonymous map that must be linked with H5Olink before closing
   - Specify the key datatype and value datatype during creation - these define how keys and values are stored in the map
   - Optionally provide creation and access property lists (MCPL and MAPL) to control map behavior
   - The function returns a map identifier (hid_t) used for all subsequent operations

2. Opening

   - Open an existing map with H5Mopen() by providing its location and name
   - Optionally specify a map access property list (MAPL) to control access behavior
   - Multiple opens of the same map are allowed and each returns a separate identifier

3. Data Operations

   - Adding/Updating: Use H5Mput() to add new key-value pairs or update existing values
     - Specify memory datatypes for the key and value being written
     - If the key already exists, its value is updated
     - If the key is new, a new key-value pair is added
   - Retrieving: Use H5Mget() to retrieve a value by its key
     - Provide a buffer to receive the value
     - Specify memory datatypes for key and value
     - Type conversion is performed if memory and stored datatypes differ
   - Checking Existence: Use H5Mexists() to check if a key exists without retrieving its value
   - Deleting: Use H5Mdelete() to remove a key-value pair from the map
   - Iterating: Use H5Miterate() or H5Miterate_by_name() to process all key-value pairs
     - Provide a callback function that is invoked for each key-value pair
     - Iteration order is determined by the VOL connector implementation
     - Can start iteration from a specific index position

4. Query Operations

   - H5Mget_count() retrieves the number of key-value pairs stored in the map
   - H5Mget_key_type() returns the datatype used for keys
   - H5Mget_val_type() returns the datatype used for values
   - H5Mget_create_plist() retrieves the map creation property list (MCPL)
   - H5Mget_access_plist() retrieves the map access property list (MAPL)

5. Closing
   - Close the map with H5Mclose() when finished
   - All map identifiers should be closed before closing the file
   - Closing a map does not delete it from the file
   - The library maintains reference counts and will not actually close the map until all identifiers are closed

## Map Property Lists

Map objects use two types of property lists that fit into the HDF5 property list class hierarchy (see Property List Classes):

- Map Creation Property List (MCPL) - H5P_MAP_CREATE

  - Controls properties that are set when a map is created
  - These properties are permanent characteristics of the map
  - Inherits from the Object Creation Property List (OCPL) class
  - Can be retrieved later with H5Mget_create_plist()
  - Default MCPL: H5P_MAP_CREATE_DEFAULT or H5P_DEFAULT

- Map Access Property List (MAPL) - H5P_MAP_ACCESS
  - Controls properties for accessing an existing map
  - These properties affect how the map is opened and accessed
  - Settings can vary between different opens of the same map
  - Can be retrieved with H5Mget_access_plist()
  - Default MAPL: H5P_MAP_ACCESS_DEFAULT or H5P_DEFAULT

These property list classes follow the same inheritance hierarchy as other HDF5 objects. The MCPL inherits properties relevant to object creation (like character encoding for names), while the MAPL controls access-specific settings.

## Key and Value Datatypes

Map objects require two datatypes:

- Key Datatype: Defines how keys are stored in the map

  - Can be any valid HDF5 datatype (integer, float, string, compound, etc.)
  - Variable-length strings (H5T_C_S1 with H5T_VARIABLE size) are commonly used for keys
  - Fixed at map creation time

- Value Datatype: Defines how values are stored in the map
  - Can be any valid HDF5 datatype
  - All values in a map must have the same datatype
  - Fixed at map creation time

During H5Mput() and H5Mget() operations, memory datatypes can differ from the stored datatypes. The HDF5 library will perform type conversion as needed, similar to dataset I/O operations.

## Example Usage

The following example demonstrates creating a map, adding key-value pairs, and retrieving values. Note that this example requires a VOL connector with map support (e.g., DAOS). See HDF5 Virtual Object Layer (VOL) for VOL connector configuration details.

hid_t file_id, map_id, vls_type_id, fapl_id;
const char *names[2] = {"Alice", "Bob"};
uint64_t IDs[2] = {25385486, 34873275};
uint64_t val_out;
herr_t ret;
// Setup file access property list with VOL connector that supports maps
// (Configuration depends on specific VOL connector - see connector documentation)
fapl_id = H5Pcreate(H5P_FILE_ACCESS);
// ... configure VOL connector in fapl_id ...
// Create variable-length string datatype for keys
vls_type_id = H5Tcopy(H5T_C_S1);
H5Tset_size(vls_type_id, H5T_VARIABLE);
// Create file
file_id = H5Fcreate("file.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl_id);
// Create map with string keys and uint64 values
map_id = H5Mcreate(file_id, "map", vls_type_id, H5T_NATIVE_UINT64,
H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
// Add key-value pairs
H5Mput(map_id, vls_type_id, &names[0], H5T_NATIVE_UINT64, &IDs[0], H5P_DEFAULT);
H5Mput(map_id, vls_type_id, &names[1], H5T_NATIVE_UINT64, &IDs[1], H5P_DEFAULT);
// Retrieve a value by key
ret = H5Mget(map_id, vls_type_id, &names[0], H5T_NATIVE_UINT64, &val_out, H5P_DEFAULT);
if(ret < 0 || val_out != IDs[0]) {
fprintf(stderr, "Failed to retrieve correct value from map\n");
goto error;
}
// Close map and other objects
H5Mclose(map_id);
H5Tclose(vls_type_id);
H5Pclose(fapl_id);
H5Fclose(file_id);
return 0;
error:
H5E_BEGIN_TRY {
H5Mclose(map_id);
H5Tclose(vls_type_id);
H5Pclose(fapl_id);
H5Fclose(file_id);
} H5E_END_TRY;
return -1;
H5E_END_TRY
#define H5E_END_TRY
Definition H5Epublic.h:103
H5E_BEGIN_TRY
#define H5E_BEGIN_TRY
Definition H5Epublic.h:84
H5F_ACC_TRUNC
#define H5F_ACC_TRUNC
Definition H5Fpublic.h:30
hid_t
int64_t hid_t
Definition H5Ipublic.h:60
H5P_FILE_ACCESS
#define H5P_FILE_ACCESS
Definition H5Ppublic.h:56
H5P_DEFAULT
#define H5P_DEFAULT
Definition H5Ppublic.h:220
H5T_VARIABLE
#define H5T_VARIABLE
Definition H5Tpublic.h:209
herr_t
int herr_t
Definition H5public.h:269
H5Fclose
herr_t H5Fclose(hid_t file_id)
Terminates access to an HDF5 file.
H5Fcreate
hid_t H5Fcreate(const char *filename, unsigned flags, hid_t fcpl_id, hid_t fapl_id)
Creates an HDF5 file.
H5Mget
herr_t H5Mget(hid_t map_id, hid_t key_mem_type_id, const void *key, hid_t val_mem_type_id, void *value, hid_t dxpl_id)
Retrieves a key-value pair from a map object.
H5Mput
herr_t H5Mput(hid_t map_id, hid_t key_mem_type_id, const void *key, hid_t val_mem_type_id, const void *value, hid_t dxpl_id)
Adds a key-value pair to a map object.
H5Mcreate
hid_t H5Mcreate(hid_t loc_id, const char *name, hid_t key_type_id, hid_t val_type_id, hid_t lcpl_id, hid_t mcpl_id, hid_t mapl_id)
Creates a map object.
H5Mclose
herr_t H5Mclose(hid_t map_id)
Terminates access to a map object.
H5Tset_size
herr_t H5Tset_size(hid_t type_id, size_t size)
Sets size for a datatype.
H5Tcopy
hid_t H5Tcopy(hid_t type_id)
Copies an existing datatype.
H5Tclose
herr_t H5Tclose(hid_t type_id)
Releases a datatype.
H5T_NATIVE_UINT64
#define H5T_NATIVE_UINT64
Definition H5Tpublic.h:1267
H5T_C_S1
#define H5T_C_S1
Definition H5Tpublic.h:646
H5Pclose
herr_t H5Pclose(hid_t plist_id)
Terminates access to a property list.
H5Pcreate
hid_t H5Pcreate(hid_t cls_id)
Creates a new property list as an instance of a property list class.

## Important Notes

- VOL Connector Requirement: Map functionality is only available through VOL connectors that implement the map interface. The native HDF5 file format does not support maps.

- Experimental API: The H5M interface is experimental and subject to change in future releases. Application code using maps should be prepared for potential API modifications.

- Key Uniqueness: Each key in a map must be unique. Calling H5Mput() with an existing key will update that key's value rather than creating a duplicate entry.

- Iteration Order: The order in which key-value pairs are returned during iteration is determined by the VOL connector implementation and may not be predictable.

- Type Conversion: As with datasets, the library performs datatype conversion between memory and storage representations. Ensure memory buffers are sized appropriately for the memory datatype specified.

- Asynchronous Operations: Several map operations have asynchronous variants (e.g., H5Mcreate_async, H5Mopen_async) for use with the asynchronous I/O interface.

Previous Chapter The HDF5 Event Set Interface - Next Chapter HDF5 References

Navigate back: Main / HDF5 User Guide