[Top] [Prev] [Next] [Bottom]

3.8 Chunked (or Tiled) Scientific Data Sets

NOTE: It is strongly encouraged that HDF users who wish to use the SD chunking routines first read the section on SD chunking in Chapter 13, titled HDF Performance Issues. In that section the concepts of chunking are explained, as well as their use in relation to HDF. As the ability to work with chunked data has been added to HDF functionality for the purpose of addressing specific performance-related issues, the user should first have the necessary background knowledge to correctly determine how chunking will positively or adversely affect their application.

This chapter will refer to both "tiled" and "chunked" SDSs as simply "chunked SDSs", as tiled SDSs are the two-dimensional case of chunked SDSs.

3.8.1 Making a Non-Chunked SDS a Chunked SDS: SDsetchunk

In HDF, an SDS must first be created as a generic SDS through the SDcreate routine, then "promoted" to be a chunked SDS through the use of SDsetchunk. SDsetchunk determines the chunk size and the compression method, if any, to be applied when accessing the chunks.

SDsetchunk receives its information on how to create the chunks, as well as compression information, from an HDF_CHUNK_DEF union passed in as its second argument. This union structure is defined in the HDF library as follows:

	typedef union hdf_chunk_def_u {
		int32 chunk_lengths[MAX_VAR_DIMS]; 
		struct {
			int32 chunk_lengths[MAX_VAR_DIMS]; 
			int32 comp_type;
			comp_info cinfo;
		} comp;
	}HDF_CHUNK_DEF

Note that there is currently no Fortran-77 version of SDsetchunk.

TABLE 3M SDsetchunk Parameter List
Routine Name

Parameter

Data Type

Description

C

SDsetchunk

sds_id

int32

SD identifier.

c_def

HDF_CHUNK_DEF

Union containing information on how the chunks are to be defined.

flags

int32

Flags determining the behavior of the routine.

The flags parameter can either be set to HDF_CHUNK if the SDS is to be uncompressed, or to the bitwise-OR'ed values of HDF_CHUNK and HDF_COMP ( HDF_CHUNK | HDF_COMP) if a compression method is to be applied. In the former case the first definition of the chunk_lengths array should be set to the dimensions of the chunks. For example, given the definition of HDF_CHUNK_DEF stated above, a flags parameter value of HDF_CHUNK and the following definition of a three-dimensional chunked SDS


	HDF_CHUNK_DEF current_chunk_def;

	current_chunk_def.chunk_lengths[0] = 2;
	current_chunk_def.chunk_lengths[1] = 3;
	current_chunk_def.chunk_lengths[2] = 4;

the size of the chunk passed to SDsetchunk will be 2 bytes by 3 bytes by 4 bytes.

In the latter case (where a compression method is specified), the chunk_lengths array definition within the comp structure is initialized in the way described above and the cinfo structure is initialized in the same way as for the SDsetcompress routine. (See Figure 3.5.3 on page 40.)

There are two restrictions that apply to chunked SDSs. The maximum number of chunks in a single HDF file is 65,535, and a chunked SDS cannot contain an unlimited dimension.

SDsetchunk returns either a value of SUCCEED or FAIL.

3.8.2 Setting the Maximum Number of Chunks in the Cache: SDsetchunkcache

To maximize the performance of the HDF library routines when working with chunked SDSs, the library maintains a separate area of memory specifically for cached data chunks. SDsetchunkcache determines the maximum number of chunks of the specified SDS that are cached into this segment of memory.

When the chunk cache has been filled, any additional chunks written to cache memory are cached according to the LRU, or Least-Recently-Used, algorithm. This means that the chunk that has resided in the cache the longest without being reread or rewritten will be written over with the new chunk.

Note that there is currently no Fortran-77 equivalent of SDsetchunkcache.

TABLE 3N SDsetchunkcache Parameter List
Routine Name

Parameter

Data Type

Description

C

SDsetchunkcache

sds_id

int32

SD identifier.

maxcache

int32

Maximum number of chunks to cache.

flags

int32

Flags determining the default caching behavior.

By default, when a generic SDS is promoted to be a chunked SDS, the maxcache parameter is set to the number of chunks along the last dimension.

If the chunk cache is full and the value of the maxcache parameter is larger than the currently allowed maximum number of cached chunks, then the maximum number of cached chunks is reset to the value of maxcache. If the chunk cache is not full, then the size of the chunk cache is reset to the value of maxcache only if it is greater than the current number of chunks in the cache.

Currently the only allowed value of the flag parameter is 0, which designates default operation. In the near future, the value HDF_CACHEALL will be supported to be used to specify that the whole SDS object is to be cached.

SDsetchunkcache returns either the set value of the maximum number of cacheable chunks or FAIL.

3.8.3 Writing Data to Chunked SDSs: SDwritechunk and SDwritedata

If an SDS has been created as a chunked SDS (i.e., has been created through calls to SDcreate and SDsetchunk), both SDwritedata and SDwritechunk can be used to write data to it. There are situations where SDwritechunk may be a more appropriate routine to use than SDwritedata, but both routines essentially achieve the same results.

The location of data in a chunked SDS can be referenced in two ways. The first is the standard method used in the SD routines that access both chunked and non-chunked SDSs, and refers to the starting location, or the origin, as an offset in bytes from the origin of the SD array itself. The second method is used by the SD routines that only access chunked SDSs, and refers to the origin of the chunk as an offset in chunks from the origin of the SD array itself. See the section on SD chunking in Chapter 13, titled HDF Performance Issues, for an illustration of this.

SDwritechunk is used when an entire chunk is to be written and requires that the chunk offset be known. SDwritedata is used when the write operation is to be done regardless of the chunking scheme used in the SDS and the byte offset of the target chunk is known. Also, as SDwritechunk is written specifically for chunked SDSs and doesn't have the overhead of the additional functionality supported by the SDwritedata routine, it is much faster than SDwritedata.

The parameters of SDwritedata are listed above. (See Table 3D on page 30.) The parameters of SDwritechunk are as follows. Note that there currently is no Fortran-77 equivalent of SDwritechunk.

TABLE 3O SDwritechunk Parameter List
Routine Name

Parameter

Data Type

Description

C

SDwritechunk

sds_id

int32

SD identifier.

origin

int32 *

Origin of the chunk to be written.

datap

const VOID *

Buffer containing the data to be written.

The datap parameter must point to an array containing an entire chunk of data - in other words, the size of the array must be the same as the chunk size of the SDS to be written to. An error condition will result if this is not the case.

SDwritechunk returns either a value of SUCCEED or FAIL.

3.8.4 Reading Data From Chunked SDSs: SDreadchunk and SDreaddata

As both SDwritedata and SDwritechunk can be used to write data to chunked SDSs, both SDreaddata and SDreadchunk can be used to read data from chunked SDSs.

SDreadchunk is used when an entire chunk of data is to be read. SDreaddata is used when the read operation is to be done regardless of the chunking scheme used in the SDS. Also, SDreadchunk is written specifically for chunked SDSs and doesn't have the overhead of the additional functionality supported by the SDreaddata routine - therefore, it is much faster than SDreaddata.

SDreadchunk returns either a value of SUCCEED or FAIL.

The parameters of SDreaddata are listed above. (See Table 3I on page 47.) The parameters of SDreadchunk are as follows. Note that there is currently no Fortran-77 equivalent of SDreadchunk.

TABLE 3P SDreadchunk Parameter List
Routine Name

Parameter

Data Type

Description

C

SDreadchunk

sds_id

int32

SD identifier.

origin

int32 *

Origin of the chunk to be read.

datap

VOID *

Buffer for the returned chunk data.

As with SDwritechunk, the datap parameter must point to an array containing enough space for an entire chunk of data. In other words, the size of the array must be the same as the chunk size of the SDS to be written to. An error condition will result if this is not the case.

3.8.5 Obtaining Information About a Chunked SDS: SDgetchunkinfo

SDgetchunkinfo is used to determine how the chunks in a chunked SDS are defined and whether the SDS is chunked or not.

Information about the chunks is returned in the HDF_CHUNK_DEF union provided as the second parameter - therefore, it will return the same information that can be set in the SDsetchunk routine. For example, the flags argument will return either HDF_CHUNK for an uncompressed chunked SDS, HDF_CHUNK bitwise-OR'ed with HDF_COMP for a compressed and chunked SDS or HDF_NONE for a non-chunked SDS. A pointer to a chunk_lengths array containing chunk dimension size information will be returned if the returned flags param value is HDF_CHUNK, a pointer to a comp structure containing chunk dimension size and compression information will be returned if the returned flags param value is HDF_CHUNK bitwise-OR'ed with HDF_COMP.

A NULL value can also be passed in as the c_def param if chunking information is not desired.

Note that there is currently no Fortran-77 equivalent of SDgetchunkinfo.

TABLE 3Q SDgetchunkinfo Parameter List
Routine Name

Parameter

Data Type

Description

C

SDgetchunkinfo

sds_id

int32

SD identifier.

c_def

HDF_CHUNK_DEF *

Union structure containing information about the chunks in the SDS.

flags

int32 *

Flags determining the behavior of the routine.

SDgetchunkinfo returns either a value of SUCCEED or FAIL.

EXAMPLE 13. Writing and Reading Chunked Data Using SDwritechunk and SDreaddata

This example creates a 9-by-4 integer chunked uncompressed SDS in a file named "Example13.hdf", then writes six 3-by-2 byte chunks of 16-bit unsigned integers to it. It then reads one 5-by-2 16-bit unsigned integer subset. It uses SDwritechunk to write the chunks and SDreaddata to read the subset.

C:

#include "hdf.h"
#include "mfhdf.h"

	/* Arrays containing dimension info for datasets. */
	static int32  d_dims[3]     = {2, 3, 4};  /* Data dimensions */
	static int32  edge_dims[3]  = {0, 0, 0};  /* Edge dims */
	static int32  start_dims[3] = {0, 0, 0};  /* Starting dims  */
	static int32  cdims[3]      = {1, 2, 3};  /* Chunk lengths */

	int32 status;

	static uint16  chunk1_2u16[6] = {11, 21, 
	                               12, 22, 
	                               13, 23};
    
	static uint16  chunk2_2u16[6] = {31, 41, 
	                               32, 42, 
	                               33, 43};
    
	static uint16  chunk3_2u16[6] = {14, 24, 
	                               15, 25, 
	                               16, 26};
    
	static uint16  chunk4_2u16[6] = {34, 44, 
	                               35, 45, 
	                               36, 46};
    
	static uint16  chunk5_2u16[6] = {17, 27, 
	                               18, 28, 
	                               19, 29};
    
	static uint16  chunk6_2u16[6] = {37, 47, 
	                               38, 48, 
	                               39, 49};

main( ) 
{

	int32 f1;				/* File handle */
	int32 sdsid;			/* SDS handle */
	uint16  inbuf_2u16[5][2];	/* Data array read */
	uint16  fill_u16 = 0;		/* Fill value */
	int32 c_flags;

	HDF_CHUNK_DEF c_def, r_def; 

	/* Create the HDF file. */
	f1 = SDstart("Example13.hdf", DFACC_CREATE);

	/* Create a  9x4 SDS of uint16 in file 1. */
	d_dims[0] = 9;
	d_dims[1] = 4;

	sdsid = SDcreate(f1, "DataSetChunked_1", DFNT_UINT16, 2, d_dims);

	/* Set the fill value. */
	fill_u16 = 0;
	status = SDsetfillvalue(sdsid, (VOIDP) &fill_u16);

	/* Create chunked SDS chunk with 3x2 chunks which will create 
	6 chunks. */

	c_def.chunk_lengths[0] = 3;
	c_def.chunk_lengths[1] = 2;

	status = SDsetchunk(sdsid, c_def, HDF_CHUNK);

	/* Set chunk cache to hold a maximum of 3 chunks */
	status = SDsetchunkcache(sdsid, 3, 0);

	/* Write the data chunks. */

	/* Write chunk 1. */
	start_dims[0] = 0;
	start_dims[1] = 0;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk1_2u16);

	/* Write chunk 4. */
	start_dims[0] = 1;
	start_dims[1] = 1;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk4_2u16);

	/* Write chunk 2. */
	start_dims[0] = 0;
	start_dims[1] = 1;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk2_2u16);

	/* Write chunk 5. */
	start_dims[0] = 2;
	start_dims[1] = 0;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk5_2u16);

	/* Write chunk 3. */
	start_dims[0] = 1;
	start_dims[1] = 0;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk3_2u16);

	/* Write chunk 6. */
	start_dims[0] = 2;
	start_dims[1] = 1;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk6_2u16);

	/* Read a subset of the data back in using SDreaddata
       i.e  5x2 subset of the whole array. */
	start_dims[0] = 2;
	start_dims[1] = 1;
	edge_dims[0] = 5;
	edge_dims[1] = 2;
	status = SDreaddata(sdsid, start_dims, NULL, edge_dims, \
						(VOIDP) inbuf_2u16);

	/* This 5x2 array should look like this
         {{23, 24, 25, 26, 27},
          {33, 34, 35, 36, 37}} 		*/

	/* Get chunk lengths. */
	status = SDgetchunkinfo(sdsid, &r_def, &c_flags);

	/* Close the current SDS. */    
	status = SDendaccess(sdsid);

	/* Close down the SDS interface. */
	status = SDend(f1);

}

EXAMPLE 14. Writing and Reading Chunked Data Using SDwritedata and SDreaddata

This example creates a chunked SDS with a size of 2-by-3-by-4 16-bit unsigned integers in a file named "Example14.hdf", then writes two 2-by-3-by-2 16-bit unsigned integer chunks to it. It then reads one 2-by-3-by-4 16-bit unsigned integer chunk. It uses SDwritedata to write the chunks and SDreaddata to read the subset.

C:

#include "hdf.h"
#include "mfhdf.h"

	/* Arrays containing dimension info for datasets. */
	static int32  d_dims[3]     = {2, 3, 4};  /* Data dimensions */
	static int32  edge_dims[3]  = {0, 0, 0};  /* Edge dims */
	static int32  start_dims[3] = {0, 0, 0};  /* Starting dims  */
	static int32  cdims[3]      = {1, 2, 3};  /* Chunk lengths */

	int32 status;

	static uint16  u16_3data[2][3][4] =
	{
	{
		{ 0, 1, 2, 3},
 		{ 10, 11, 12, 13},
		{ 20, 21, 22, 23}},
	{
		{ 100, 101, 102, 103},
		{ 110, 111, 112, 113},
		{ 120, 121, 122, 123}}};

main( ) 
{

	int32 f1;	/* File handle */
	int32 sdsid;	/* SDS handle */
	uint16  inbuf_3u16[2][3][4];	/* Data array read */
	uint16  fill_u16 = 0;        /* Fill value */
	int32 c_flags;

	HDF_CHUNK_DEF c_def, r_def; 

	/* Create the HDF file. */
	f1 = SDstart("Example14.hdf", DFACC_CREATE);

	/* Create a new 2x3x4 SDS of uint16 in the file. */
	d_dims[0] = 2;
	d_dims[1] = 3;
	d_dims[2] = 4;
	sdsid = SDcreate(f1, "DataSetChunked_2", DFNT_UINT16, 3, d_dims);

	/* Set the fill value. */
	fill_u16 = 0;
	status = SDsetfillvalue(sdsid, (VOIDP) &fill_u16);

	/* Create chunked SDS - chunk is 2x3x2 which will create 2 chunks */
	c_def.chunk_lengths[0] = 2;
	c_def.chunk_lengths[1] = 3;
	c_def.chunk_lengths[2] = 2;
	status = SDsetchunk(sdsid, c_def, HDF_CHUNK);

	/* Set chunk cache to hold a maximum of 2 chunks. */
	status = SDsetchunkcache(sdsid, 2, 0);

	/* Write data using SDwritedata. */
	start_dims[0] = 0;
	start_dims[1] = 0;
	start_dims[2] = 0;
	edge_dims[0] = 2;
	edge_dims[1] = 3;
	edge_dims[2] = 4;
	status = SDwritedata(sdsid, start_dims, NULL, edge_dims, \
					(VOIDP) u16_3data);

	/* Read data using SDreaddata. */
	start_dims[0] = 0;
	start_dims[1] = 0;
	start_dims[2] = 0;
	edge_dims[0] = 2;
	edge_dims[1] = 3;
	edge_dims[2] = 4;
	status = SDreaddata(sdsid, start_dims, NULL, edge_dims, \
						(VOIDP) inbuf_3u16);

	/* Verify the data in inbuf_3u16 against u16_3data[]. */

	/* Get chunk lengths. */
	status = SDgetchunkinfo(sdsid, &r_def, &c_flags);

	/* Close the current SDS. */    
	status = SDendaccess(sdsid);

	/* Close down SDS interface. */
	status = SDend(f1);

}

EXAMPLE 15. Writing and Reading Chunked Data Using SDwritedata and SDreadchunk

This example creates a chunked SDS with a size of 2-by-3-by-4 16-bit unsigned integers in a file named "Example15.hdf", then writes one 2-by-3-by-4 16-bit unsigned integer chunk to it. It then reads six 1-by-1-by-4 16-bit unsigned integer subsets. It uses SDwritedata to write the chunks and SDreadchunk to read the subsets.

C:

#include "hdf.h"
#include "mfhdf.h"

	/* Arrays containing dimension info for datasets. */
	static int32  d_dims[3]     = {2, 3, 4};  /* Data dimensions */
	static int32  edge_dims[3]  = {0, 0, 0};  /* Edge dims */
	static int32  start_dims[3] = {0, 0, 0};  /* Starting dims  */
	static int32  cdims[3]      = {1, 2, 3};  /* Chunk lengths */

	static uint16  u16_3data[2][3][4] =
	{
	{
		{ 0, 1, 2, 3},
		{ 10, 11, 12, 13},
		{ 20, 21, 22, 23}},
	{
		{ 100, 101, 102, 103},
		{ 110, 111, 112, 113},
		{ 120, 121, 122, 123}}};

main( ) 
{

	int32 f1;				/* File handle */
	int32 sdsid;			/* SDS handle */
	uint16  ru16_3data[4];		/* Whole chunk input buffer */
	int32   rcdims[3];		/* For SDgetchunkinfo() */
	uint16  fill_u16 = 0;		/* Fill value */
	int32 c_flags;
	HDF_CHUNK_DEF c_def, r_def; 

	int32 status;

	/* Create the HDF file. */
	f1 = SDstart("Example15.hdf", DFACC_CREATE);

	/* Create a new 2x3x4 SDS of uint16 in the file. */
	d_dims[0] = 2;
	d_dims[1] = 3;
	d_dims[2] = 4;
	sdsid = SDcreate(f1, "DataSetChunked_4", DFNT_UINT16, 3, d_dims);

	/* Set the fill value. */
	fill_u16 = 0;
	status = SDsetfillvalue(sdsid, (VOIDP) &fill_u16);

	/* Create chunked SDS - chunk is 1x1x4 which will create 6 chunks. */
	c_def.chunk_lengths[0] = 1;
	c_def.chunk_lengths[1] = 1;
	c_def.chunk_lengths[2] = 4;
	status = SDsetchunk(sdsid, c_def, HDF_CHUNK);

	/* Set chunk cache to hold a maximum of 4 chunks. */
	status = SDsetchunkcache(sdsid, 4, 0);

	/* Write data using SDwritedata. */
	start_dims[0] = 0;
	start_dims[1] = 0;
	start_dims[2] = 0;
	edge_dims[0] = 2;
	edge_dims[1] = 3;
	edge_dims[2] = 4;
	status = SDwritedata(sdsid, start_dims, NULL, edge_dims, \
						(VOIDP) u16_3data);

	/* Read data using SDreadchunk and verify against
       the chunk arrays chunk1_3u16[] ... chunk6_3u16[]. */

	/* Read chunk 1. */
	start_dims[0] = 0;
	start_dims[1] = 0;
	start_dims[2] = 0;
	status = SDreadchunk(sdsid, start_dims, (VOIDP) ru16_3data);

	/* Read chunk 2. */
	start_dims[0] = 0;
	start_dims[1] = 1;
	start_dims[2] = 0;
	status = SDreadchunk(sdsid, start_dims, (VOIDP) ru16_3data);

	/* Read chunk 3. */
	start_dims[0] = 0;
	start_dims[1] = 2;
	start_dims[2] = 0;
	status = SDreadchunk(sdsid, start_dims, (VOIDP) ru16_3data);

	/* Read chunk 4. */
	start_dims[0] = 1;
	start_dims[1] = 0;
	start_dims[2] = 0;
	status = SDreadchunk(sdsid, start_dims, (VOIDP) ru16_3data);

	/* Read chunk 5. */
	start_dims[0] = 1;
	start_dims[1] = 1;
	start_dims[2] = 0;
	status = SDreadchunk(sdsid, start_dims, (VOIDP) ru16_3data);

	/* Read chunk 6. */
	start_dims[0] = 1;
	start_dims[1] = 2;
	start_dims[2] = 0;
	status = SDreadchunk(sdsid, start_dims, (VOIDP) ru16_3data);

	/* Get chunk lengths. */
	status = SDgetchunkinfo(sdsid, &r_def, &c_flags);

	/* Close the current SDS. */    
	status = SDendaccess(sdsid);

	/* Close down the SDS interface. */
	status = SDend(f1);

}

EXAMPLE 16. Writing and Reading Compressed Chunked Data Using SDwritechunk and SDreaddata

This example uses SDwritechunk to write the chunks and SDreaddata to read the subset, like Example 13 (See page 62.). However, it also compresses the chunks using the GZIP (skipping Huffman) algorithm.

C:

#include "hdf.h"
#include "mfhdf.h"

	/* Arrays containing dimension info for datasets. */
	static int32  d_dims[3]     = {2, 3, 4};  /* Data dimensions */
	static int32  edge_dims[3]  = {0, 0, 0};  /* Edge dims */
	static int32  start_dims[3] = {0, 0, 0};  /* Starting dims  */
	static int32  cdims[3]      = {1, 2, 3};  /* Chunk lengths */
	
	int32 status;

	static uint16  chunk1_2u16[6] = {11, 21, 
      	                           12, 22, 
            	                     13, 23};

	static uint16  chunk2_2u16[6] = {31, 41, 
      	                           32, 42, 
            	                     33, 43};

	static uint16  chunk3_2u16[6] = {14, 24, 
      	                           15, 25, 
            	                     16, 26};

	static uint16  chunk4_2u16[6] = {34, 44, 
      	                           35, 45, 
            	                     36, 46};

	static uint16  chunk5_2u16[6] = {17, 27, 
      	                           18, 28, 
            	                     19, 29};

	static uint16  chunk6_2u16[6] = {37, 47, 
      	                           38, 48, 
            	                     39, 49};

main( ) 
{

	int32 f1;				/* File handle */
	int32 sdsid;			/* SDS handle */
	uint16  inbuf_2u16[5][2];    /* Data array read */
	uint16  fill_u16 = 0;        /* Fill value */
	int32 c_flags;
	HDF_CHUNK_DEF c_def, r_def; 

	/* Create the HDF file. */
	f1 = SDstart("Example16.hdf", DFACC_CREATE);

	/* Create a 9x4 SDS of uint16 in file 1. */
	d_dims[0] = 9;
	d_dims[1] = 4;
	sdsid = SDcreate(f1, "DataSetChunked_1", DFNT_UINT16, 2, d_dims);

	/* Set the fill value. */
	fill_u16 = 0;
	status = SDsetfillvalue(sdsid, (VOIDP) &fill_u16);

	/* Create chunked SDS chunk with 3x2 chunks which will create 
		6 chunks. */
	c_def.chunk_lengths[0] = c_def.comp.chunk_lengths[0] = 3;
	c_def.chunk_lengths[1] = c_def.comp.chunk_lengths[1] = 2;

	c_def.comp.comp_type = COMP_CODE_DEFLATE; /* GZIP */
	c_def.comp.cinfo.deflate.level = 6; 

	status = SDsetchunk(sdsid, c_def, HDF_CHUNK | HDF_COMP);

	/* Set chunk cache to hold a maximum of 3 chunks */
	status = SDsetchunkcache(sdsid, 3, 0);

	/* Write the data chunks. */

	/* Write chunk 1. */
	start_dims[0] = 0;
	start_dims[1] = 0;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk1_2u16);

	/* Write chunk 4. */
	start_dims[0] = 1;
	start_dims[1] = 1;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk4_2u16);

	/* Write chunk 2. */
	start_dims[0] = 0;
	start_dims[1] = 1;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk2_2u16);

	/* Write chunk 5. */
	start_dims[0] = 2;
	start_dims[1] = 0;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk5_2u16);

	/* Write chunk 3. */
	start_dims[0] = 1;
	start_dims[1] = 0;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk3_2u16);

	/* Write chunk 6. */
	start_dims[0] = 2;
	start_dims[1] = 1;
	status = SDwritechunk(sdsid, start_dims, (VOIDP) chunk6_2u16);
	
	/* Read a subset of the data back in using SDreaddata
       i.e  5x2 subset of the whole array. */
	start_dims[0] = 2;
	start_dims[1] = 1;
	edge_dims[0] = 5;
	edge_dims[1] = 2;
	status = SDreaddata(sdsid, start_dims, NULL, edge_dims, \
						(VOIDP) inbuf_2u16);

	/* This 5x2 array should look like this
         {{23, 24, 25, 26, 27},
          {33, 34, 35, 36, 37}} 		*/

	/* Get chunk lengths. */
	status = SDgetchunkinfo(sdsid, &r_def, &c_flags);

	/* Close the current SDS. */    
	status = SDendaccess(sdsid);

	/* Close down the SDS interface. */
	status = SDend(f1);

}

3.8.6 Ghost Areas

In cases where the size of the SDS array is not an even multiple of the chunk size, regions of excess array space beyond the defined dimensions of the SDS will be created. Refer to the following illustration.

FIGURE 3c Array Locations Created Beyond the Defined Dimensions of an SDS

These regions are called "ghost areas". "Ghost areas" can be accessed only by SDreadchunk and SDwritechunk - they cannot be written to or accessed by either SDreaddata or SDwritedata. If the fill value has been set, the values in these array locations will be initialized to the fill value. It is highly recommended that users set the fill value before writing to chunked SDSs so that garbage values won't be read from these locations.

Also, if SDreadchunk and SDwritechunk are used, it is not recommended that valid data be written to the "ghost areas" as these areas won't be accessible by SDreaddata and SDwritedata. Moreover, the ability to write to these areas may not be supported in future versions of the HDF library.



[Top] [Prev] [Next] [Bottom]

hdfhelp@ncsa.uiuc.edu
HDF User's Guide - 06/04/97, NCSA HDF Development Group.