hdf images hdf images

Improved Compression Flexibility with Dynamically Loaded Filters

About the Project

The problem. Deutsches Elektronen-Synchrotron, abbreviated as DESY, is a particle accelerator research center based in Germany. They conduct synchrotron and x-ray free electron laser experiments. They like HDF5's file format and speed. What they did not like was the need to recompile the HDF5 Library when a different compression filter was needed. Since the experiments generate huge amounts of data, DESY needs specially optimized compression routines to help them handle the data.

The solution: dynamically loaded filters. DESY commissioned The HDF Group to change the HDF5 Library's ability to load filters from static to dynamic. As of version 1.8.11, the HDF5 Library has the ability to load a compression filter when it reads a dataset that was filled using a particular compression filter or when an application wants to write data to a dataset using a particular compression filter. No more waiting for a recompiled version of the HDF5 Library.


For more information, click the links on the left.

- - Last modified: 12 October 2016