Help us improve by taking our short survey: https://www.hdfgroup.org/website-survey/

| HDF5 Last Updated on 2026-03-23 The HDF5 Field Guide

Loading...
Searching...
No Matches
Files

Navigate back: Main / Cookbook

# Files

## Tracking Free Space in HDF5 Files

Problem
  You sometimes delete objects in HDF5 files and don't have new content to use the free space, but would like to reuse it in the future.

Solution
  At file creation time, set the file space management strategy and persistence of free space tracking information via H5Pset_file_space_strategy().

Note
  This feature is only supported in HDF5 1.10.1+.

15 {
16 __label__ fail_fcpl, fail_fapl, fail_file;
17 hid_t fcpl, fapl, file;
18
19 if ((fcpl = H5Pcreate(H5P_FILE_CREATE)) < 0) {
20 ret_val = EXIT_FAILURE;
21 goto fail_fcpl;
22 }
23#if H5_VERSION_GE(1, 10, 1)
24 if (H5Pset_file_space_strategy(fcpl, H5F_FSPACE_STRATEGY_FSM_AGGR, 1, 4096) < 0) {
25#else
26#error HDF5 1.10.1+ required
27#endif
28 ret_val = EXIT_FAILURE;
29 goto fail_fapl;
30 }
31
32 if ((fapl = H5Pcreate(H5P_FILE_ACCESS)) < 0) {
33 ret_val = EXIT_FAILURE;
34 goto fail_fapl;
35 }
36#if H5_VERSION_GE(1, 10, 1)
37 if (H5Pset_libver_bounds(fapl, H5F_LIBVER_V110, H5F_LIBVER_LATEST) < 0) {
38#else
39#error HDF5 1.10.x+ required
40#endif
41 ret_val = EXIT_FAILURE;
42 goto fail_file;
43 }
44
45 if ((file = H5Fcreate("free_space.h5", H5F_ACC_TRUNC, fcpl, fapl)) < 0) {
46 ret_val = EXIT_FAILURE;
47 goto fail_file;
48 }
49 H5Fclose(file);
50
51fail_file:
52 H5Pclose(fapl);
53fail_fapl:
54 H5Pclose(fcpl);
55fail_fcpl:;
56 }

Discussion
  Free space tracking is supported only in HDF5 versions 1.10.x and higher. This has implications for the accessibility of your HDF5 files and should be considered carefully. If compatibility with previous versions of HDF5 must be maintained, space reclamation via h5repack might be an option.
  The file space strategy H5F_FSPACE_STRATEGY_FSM_AGGR is not the only option that supports free-space tracking. H5F_FSPACE_STRATEGY_PAGE is another option, which adds paged allocation and is used most effectively with page buffering.
  For an in-depth account of HDF5 file space management, paged-allocation, and page buffering, see the following documents:
  - HDF5 File Space Management
  - HDF5 File Space Management: Paged Aggregation
  - Page Buffering

See Also
  See Maintaining Compatibility with other HDF5 Library Versions for HDF5 compatibility implications.

## Removing Unused Space from HDF5 Files

Problem
  Based on estimates or h5stat output you know that a large portion of an HDF5 file consists of free or unaccounted space, and you would like to remove it.

## Creating an HDF5 File User Block

Problem
  You would like to include certain ancillary, non-HDF5 content in an HDF5 file such that it can be accessed without the HDF5 library.

Solution
  Use a file creation property list in which the user block size is set via H5Pset_userblock(). In the example below, we create an 8 MiB user block.
  60 {
  61 __label__ fail_fcpl, fail_file;
  62 hid_t fcpl, file;
  63
  64 if ((fcpl = H5Pcreate(H5P_FILE_CREATE)) < 0) {
  65 ret_val = EXIT_FAILURE;
  66 goto fail_fcpl;
  67 }
  68 if (H5Pset_userblock(fcpl, 8192 * 1024) < 0) {
  69 ret_val = EXIT_FAILURE;
  70 goto fail_file;
  71 }
  72 if ((file = H5Fcreate("userblock.h5", H5F_ACC_TRUNC, fcpl, H5P_DEFAULT)) < 0) {
  73 ret_val = EXIT_FAILURE;
  74 goto fail_file;
  75 }
  76 H5Fclose(file);
  77
  78fail_file:
  79 H5Pclose(fcpl);
  80fail_fcpl:;
  81 }

Discussion
  The user block begins at offset 0 and must be at least 512 bytes and a power of 2. The HDF5 library ignores any content between the beginning of the file and the end of the user block.
  You can add or strip a user block to/from an existing HDF5 file with the h5jam/h5unjam tool, respectively.

Warning
  If you try to embed content into the user block for use by other applications, pay close attention to how they handle space beyond the last used byte in the user block or the user block in general. In the worst case, applications might try to truncate the rest of the file and destroy the HDF5 portion of the file.

See Also
  References to related recipes

Navigate back: Main / Cookbook