public class H5CompoundDS extends CompoundDS
An HDF5 dataset is an object composed of a collection of data elements, or raw data, and metadata that stores a description of the data elements, data layout, and all other information necessary to write, read, and interpret the stored data.
A HDF5 compound datatype is similar to a struct in C or a common block in Fortran: it is a collection of one or more atomic types or small arrays of such types. Each member of a compound type has a name which is unique within that type, and a byte offset that determines the first byte (smallest byte address) of that member in a compound datum.
For more information on HDF5 datasets and datatypes, read the HDF5 User's Guide.
There are two basic types of compound datasets: simple compound data and nested compound data. Members of a simple compound dataset have atomic datatypes. Members of a nested compound dataset are compound or array of compound data.
Since Java does not understand C structures, we cannot directly read/write compound data values as in the following C example.
typedef struct s1_t {
int a;
float b;
double c;
} s1_t;
s1_t s1[LENGTH];
...
H5Dwrite(..., s1);
H5Dread(..., s1);
Values of compound data fields are stored in java.util.Vector object. We read and write compound data by fields
instead of compound structure. As for the example above, the java.util.Vector object has three elements: int[LENGTH],
float[LENGTH] and double[LENGTH]. Since Java understands the primitive datatypes of int, float and double, we will be
able to read/write the compound data by field.isMemberSelected, memberDims, memberNames, memberOrders, memberTypes, numberOfMembers, separatorchunkSize, compression, compression_gzip_txt, convertByteToString, convertedBuf, data, datatype, dimNames, dims, enumConverted, filters, isDataLoaded, maxDims, nPoints, originalBuf, rank, selectedDims, selectedIndex, selectedStride, startDims, storage, storage_layoutfileFormat, linkTargetObjName, oid| Constructor and Description |
|---|
H5CompoundDS(FileFormat theFile,
String theName,
String thePath)
Constructs an instance of a HDF5 compound dataset with given file, dataset name and path.
|
H5CompoundDS(FileFormat theFile,
String theName,
String thePath,
long[] oid)
Deprecated.
Not for public use in the future.
Using H5CompoundDS(FileFormat, String, String) |
| Modifier and Type | Method and Description |
|---|---|
void |
clear()
Clears memory held by the dataset, such as the data buffer.
|
void |
close(int did)
Closes access to the object.
|
static Dataset |
create(String name,
Group pgroup,
long[] dims,
long[] maxdims,
long[] chunks,
int gzip,
String[] memberNames,
Datatype[] memberDatatypes,
int[] memberRanks,
long[][] memberDims,
Object data)
Creates a simple compound dataset in a file with/without chunking and compression.
|
static Dataset |
create(String name,
Group pgroup,
long[] dims,
String[] memberNames,
Datatype[] memberDatatypes,
int[] memberRanks,
long[][] memberDims,
Object data)
Deprecated.
Not for public use in the future.
Using create(String, Group, long[], long[], long[], int, String[], Datatype[], int[], long[][], Object) |
static Dataset |
create(String name,
Group pgroup,
long[] dims,
String[] memberNames,
Datatype[] memberDatatypes,
int[] memberSizes,
Object data)
Deprecated.
Not for public use in the future.
Using create(String, Group, long[], long[], long[], int, String[], Datatype[], int[], long[][], Object) |
Datatype |
getDatatype()
Returns the datatype object of the dataset.
|
List<Attribute> |
getMetadata()
Retrieves the metadata such as attributes from file.
|
List<Attribute> |
getMetadata(int... attrPropList) |
int |
getSize(int tid)
Returns the size in bytes of a given datatype.
|
boolean |
hasAttribute()
Check if the object has any attributes attached.
|
void |
init()
Retrieves datatype and dataspace information from file and sets the
dataset in memory.
|
boolean |
isString(int tid)
Checks if a given datatype is a string.
|
int |
open()
Opens an existing object such as a dataset or group for access.
|
Object |
read()
Reads the data from file.
|
byte[] |
readBytes()
Reads the raw data of the dataset from file to a byte array.
|
void |
removeMetadata(Object info)
Deletes an existing metadata from this data object.
|
void |
setName(String newName)
Sets the name of the object.
|
void |
updateMetadata(Object info)
Updates an existing metadata from this data object.
|
void |
write(Object buf)
Writes the given data buffer into this dataset in a file.
|
void |
writeMetadata(Object info)
Writes a specific metadata (such as attribute) into file.
|
copy, getMemberCount, getMemberDims, getMemberNames, getMemberOrders, getMemberTypes, getSelectedMemberCount, getSelectedMemberOrders, getSelectedMemberTypes, isMemberSelected, selectMember, setMemberSelectionbyteToString, clearData, convertFromUnsignedC, convertFromUnsignedC, convertToUnsignedC, convertToUnsignedC, getChunkSize, getCompression, getConvertByteToString, getData, getDimNames, getDims, getFilters, getHeight, getMaxDims, getOriginalClass, getRank, getSelectedDims, getSelectedIndex, getStartDims, getStorage, getStorageLayout, getStride, getWidth, isEnumConverted, setConvertByteToString, setData, setEnumConverted, stringToByte, writedebug, equals, equalsOID, getFID, getFile, getFileFormat, getFullName, getLinkTargetObjName, getName, getOID, getPath, setLinkTargetObjName, setPath, toStringpublic H5CompoundDS(FileFormat theFile, String theName, String thePath)
The dataset object represents an existing dataset in the file. For example, new H5CompoundDS(file, "dset1", "/g0/") constructs a dataset object that corresponds to the dataset,"dset1", at group "/g0/".
This object is usually constructed at FileFormat.open(), which loads the file structure and object information into tree structure (TreeNode). It is rarely used elsewhere.
theFile - the file that contains the data object.theName - the name of the data object, e.g. "dset".thePath - the full path of the data object, e.g. "/arrays/".@Deprecated public H5CompoundDS(FileFormat theFile, String theName, String thePath, long[] oid)
H5CompoundDS(FileFormat, String, String)theFile - the file that contains the data object.theName - the name of the data object, e.g. "dset".thePath - the full path of the data object, e.g. "/arrays/".oid - the oid of the data object.public int open()
HObjectopen in class HObjectHObject.close(int)public void close(int did)
HObjectSub-classes must implement this interface because different data objects have their own ways of how the data resources are closed.
For example, H5Group.close() calls the hdf.hdf5lib.H5.H5Gclose() method and closes the group resource specified by the group id.
public void init()
DatasetThe init() is designed to support lazy operation in a dataset object. When a data object is retrieved from file, the datatype, dataspace and raw data are not loaded into memory. When it is asked to read the raw data from file, init() is first called to get the datatype and dataspace information, then load the raw data from file.
init() is also used to reset the selection of a dataset (start, stride and count) to the default, which is the entire dataset for 1D or 2D datasets. In the following example, init() at step 1) retrieves datatype and dataspace information from file. getData() at step 3) reads only one data point. init() at step 4) resets the selection to the whole dataset. getData() at step 4) reads the values of whole dataset into memory.
dset = (Dataset) file.get(NAME_DATASET);
// 1) get datatype and dataspace information from file
dset.init();
rank = dset.getRank(); // rank = 2, a 2D dataset
count = dset.getSelectedDims();
start = dset.getStartDims();
dims = dset.getDims();
// 2) select only one data point
for (int i = 0; i < rank; i++) {
start[0] = 0;
count[i] = 1;
}
// 3) read one data point
data = dset.getData();
// 4) reset selection to the whole dataset
dset.init();
// 5) clean the memory data buffer
dset.clearData();
// 6) Read the whole dataset
data = dset.getData();
public boolean hasAttribute()
DataFormatpublic Datatype getDatatype()
DatasetgetDatatype in class Datasetpublic void clear()
Datasetpublic byte[] readBytes() throws hdf.hdf5lib.exceptions.HDF5Exception
DatasetreadBytes() reads raw data to an array of bytes instead of array of its datatype. For example, for a one-dimension 32-bit integer dataset of size 5, readBytes() returns a byte array of size 20 instead of an int array of 5.
readBytes() can be used to copy data from one dataset to another efficiently because the raw data is not converted to its native type, it saves memory space and CPU time.
public Object read() throws Exception
Datasetread() reads the data from file to a memory buffer and returns the memory buffer. The dataset object does not hold the memory buffer. To store the memory buffer in the dataset object, one must call getData().
By default, the whole dataset is read into memory. Users can also select a subset to read. Subsetting is done in an implicit way.
How to Select a Subset
A selection is specified by three arrays: start, stride and count.
The following example shows how to make a subset. In the example, the
dataset is a 4-dimensional array of [200][100][50][10], i.e. dims[0]=200;
dims[1]=100; dims[2]=50; dims[3]=10;
We want to select every other data point in dims[1] and dims[2]
int rank = dataset.getRank(); // number of dimensions of the dataset
long[] dims = dataset.getDims(); // the dimension sizes of the dataset
long[] selected = dataset.getSelectedDims(); // the selected size of the dataset
long[] start = dataset.getStartDims(); // the offset of the selection
long[] stride = dataset.getStride(); // the stride of the dataset
int[] selectedIndex = dataset.getSelectedIndex(); // the selected dimensions for display
// select dim1 and dim2 as 2D data for display, and slice through dim0
selectedIndex[0] = 1;
selectedIndex[1] = 2;
selectedIndex[1] = 0;
// reset the selection arrays
for (int i = 0; i < rank; i++) {
start[i] = 0;
selected[i] = 1;
stride[i] = 1;
}
// set stride to 2 on dim1 and dim2 so that every other data point is
// selected.
stride[1] = 2;
stride[2] = 2;
// set the selection size of dim1 and dim2
selected[1] = dims[1] / stride[1];
selected[2] = dims[1] / stride[2];
// when dataset.getData() is called, the selection above will be used since
// the dimension arrays are passed by reference. Changes of these arrays
// outside the dataset object directly change the values of these array
// in the dataset object.
For ScalarDS, the memory data buffer is a one-dimensional array of byte, short, int, float, double or String type based on the datatype of the dataset.
For CompoundDS, the memory data object is an java.util.List object. Each element of the list is a data array that corresponds to a compound field.
For example, if compound dataset "comp" has the following nested structure, and member datatypes
comp --> m01 (int) comp --> m02 (float) comp --> nest1 --> m11 (char) comp --> nest1 --> m12 (String) comp --> nest1 --> nest2 --> m21 (long) comp --> nest1 --> nest2 --> m22 (double)getData() returns a list of six arrays: {int[], float[], char[], String[], long[] and double[]}.
read in class DatasetException - if object can not be readDataset.getData()public void write(Object buf) throws hdf.hdf5lib.exceptions.HDF5Exception
The data buffer is a vector that contains the data values of compound fields. The data is written into file field by field.
public List<Attribute> getMetadata() throws hdf.hdf5lib.exceptions.HDF5Exception
DataFormatMetadata such as attributes are stored in a List.
hdf.hdf5lib.exceptions.HDF5Exceptionpublic List<Attribute> getMetadata(int... attrPropList) throws hdf.hdf5lib.exceptions.HDF5Exception
hdf.hdf5lib.exceptions.HDF5Exceptionpublic void writeMetadata(Object info) throws Exception
DataFormatinfo - the metadata to write.Exception - if the metadata can not be writtenpublic void removeMetadata(Object info) throws hdf.hdf5lib.exceptions.HDF5Exception
DataFormatinfo - the metadata to delete.hdf.hdf5lib.exceptions.HDF5Exceptionpublic void updateMetadata(Object info) throws hdf.hdf5lib.exceptions.HDF5Exception
DataFormatinfo - the metadata to update.hdf.hdf5lib.exceptions.HDF5Exceptionpublic void setName(String newName) throws Exception
HObject@Deprecated public static Dataset create(String name, Group pgroup, long[] dims, String[] memberNames, Datatype[] memberDatatypes, int[] memberSizes, Object data) throws Exception
create(String, Group, long[], long[], long[], int, String[], Datatype[], int[], long[][], Object)name - the name of the dataset to create.pgroup - parent group where the new dataset is created.dims - the dimension size of the dataset.memberNames - the names of compound datatypememberDatatypes - the datatypes of the compound datatypememberSizes - the dim sizes of the membersdata - list of data arrays written to the new dataset, null if no data is written to the new dataset.Exception - if there is a failure.@Deprecated public static Dataset create(String name, Group pgroup, long[] dims, String[] memberNames, Datatype[] memberDatatypes, int[] memberRanks, long[][] memberDims, Object data) throws Exception
create(String, Group, long[], long[], long[], int, String[], Datatype[], int[], long[][], Object)name - the name of the dataset to create.pgroup - parent group where the new dataset is created.dims - the dimension size of the dataset.memberNames - the names of compound datatypememberDatatypes - the datatypes of the compound datatypememberRanks - the ranks of the membersmemberDims - the dim sizes of the membersdata - list of data arrays written to the new dataset, null if no data is written to the new dataset.Exception - if the dataset can not be created.public static Dataset create(String name, Group pgroup, long[] dims, long[] maxdims, long[] chunks, int gzip, String[] memberNames, Datatype[] memberDatatypes, int[] memberRanks, long[][] memberDims, Object data) throws Exception
This function provides an easy way to create a simple compound dataset in file by hiding tedious details of creating a compound dataset from users.
This function calls H5.H5Dcreate() to create a simple compound dataset in file. Nested compound dataset is not supported. The required information to create a compound dataset includes the name, the parent group and data space of the dataset, the names, datatypes and data spaces of the compound fields. Other information such as chunks, compression and the data buffer is optional.
The following example shows how to use this function to create a compound dataset in file.
H5File file = null;
String message = "";
Group pgroup = null;
int[] DATA_INT = new int[DIM_SIZE];
float[] DATA_FLOAT = new float[DIM_SIZE];
String[] DATA_STR = new String[DIM_SIZE];
long[] DIMs = { 50, 10 };
long[] CHUNKs = { 25, 5 };
try {
file = (H5File) H5FILE.open(fname, H5File.CREATE);
file.open();
pgroup = (Group) file.get("/");
}
catch (Exception ex) {
}
Vector data = new Vector();
data.add(0, DATA_INT);
data.add(1, DATA_FLOAT);
data.add(2, DATA_STR);
// create groups
Datatype[] mdtypes = new H5Datatype[3];
String[] mnames = { "int", "float", "string" };
Dataset dset = null;
try {
mdtypes[0] = new H5Datatype(Datatype.CLASS_INTEGER, 4, -1, -1);
mdtypes[1] = new H5Datatype(Datatype.CLASS_FLOAT, 4, -1, -1);
mdtypes[2] = new H5Datatype(Datatype.CLASS_STRING, STR_LEN, -1, -1);
dset = file.createCompoundDS("/CompoundDS", pgroup, DIMs, null, CHUNKs, 9, mnames, mdtypes, null, data);
}
catch (Exception ex) {
failed(message, ex, file);
return 1;
}
name - the name of the dataset to create.pgroup - parent group where the new dataset is created.dims - the dimension size of the dataset.maxdims - the max dimension size of the dataset. maxdims is set to dims if maxdims = null.chunks - the chunk size of the dataset. No chunking if chunk = null.gzip - GZIP compression level (1 to 9). 0 or negative values if no compression.memberNames - the names of compound datatypememberDatatypes - the datatypes of the compound datatypememberRanks - the ranks of the membersmemberDims - the dim sizes of the membersdata - list of data arrays written to the new dataset, null if no data is written to the new dataset.Exception - if there is a failure.public boolean isString(int tid)
DatasetCopyright © 2017. All Rights Reserved.