Lossless compression of very large volume data with fast dynamic access

Rongkai Zhao; Tao Tao; Michael Gabriel; Geneva Belford

doi:10.1117/12.481586

30 August 2002 Lossless compression of very large volume data with fast dynamic access

Rongkai Zhao, Tao Tao, Michael Gabriel, Geneva Belford

Proceedings Volume 4925, Electronic Imaging and Multimedia Technology III; (2002) https://doi.org/10.1117/12.481586
Event: Photonics Asia, 2002, Shanghai, China

Abstract

The volumetric data set is important in many scientific and biomedical fields. Since such sets may be extremely large, a compression method is critical to store and transmit them. To achieve a high compression rate, most of the existing volume compression methods are lossy, which is usually unacceptable in biomedical applications. We developed a new context-based non-linear prediction method to preprocess the volume data set in order to effectively lower the prediction entropy. The prediction error is further encoded using Huffman code. Unlike the conventional methods, the volume is divided into cubical blocks to take advantage of the data’s spatial locality. Instead of building one Huffman tree for each block, we developed a novel binning algorithm that build a Huffman tree for each group (bin) of blocks. Combining all the effects above, we achieved an excellent compression rate compared to other lossless volume compression methods. In addition, an auxiliary data structure, Scalable Hyperspace File (SHSF) is used to index the huge volume so that we can obtain many other benefits including parallel construction, on-the-fly accessing of compressed data without global decompression, fast previewing, efficient background compressing, and scalability etc.

Citation Download Citation

Rongkai Zhao, Tao Tao, Michael Gabriel, and Geneva Belford "Lossless compression of very large volume data with fast dynamic access", Proc. SPIE 4925, Electronic Imaging and Multimedia Technology III, (30 August 2002); https://doi.org/10.1117/12.481586

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available