Histogram-Aware Sorting for Enhanced Word-Aligned Compression in Bitmap Indexes

Kaser, Owen; Lemire, Daniel; Aouiche, Kamel

Histogram-Aware Sorting for Enhanced Word-Aligned Compression in Bitmap Indexes

Authors

Kaser, Owen

Lemire, Daniel

Aouiche, Kamel

Description

Bitmap indexes must be compressed to reduce input/output costs and minimize CPU usage. To accelerate logical operations (AND, OR, XOR) over bitmaps, we use techniques based on run-length encoding (RLE), such as Word-Aligned Hybrid (WAH) compression. These techniques are sensitive to the order of the rows: a simple lexicographical sort can divide the index size by 9 and make indexes several times faster. We investigate reordering heuristics based on computed attribute-value histograms. Simply permuting the columns of the table based on these histograms can increase the sorting efficiency by 40%.
To appear in proceedings of DOLAP 2008