Pongo: Efficient Lossless Floating Point Compression

Abstract

A large amount of time series data is increasingly being collected in different fields. In order to make good use of this large amount of time series data, it is necessary to solve the problems of high storage costs and transmission bandwidth that the data bring. The general compression algorithms effectively reduce the size of data at the cost of a large amount of computation. However, due to the huge time cost and batch processing mode of the general compression algorithms, Time Series Management Systems (TSMSs) often use streaming compression algorithms to replace the general compression algorithms for compressing time series data. For floating-point data, the most prevalent streaming compression algorithms, such as those based on exclusive OR (XOR) operations, offer relatively fast processing and high compression ratios compared to conventional general-purpose compression algorithms. Among them, the Elf algorithm proposes the idea of first erasing and then compressing, achieving the best compression ratio among existing streaming compression algorithms. This paper proposes a new lossless streaming compression algorithm, Pongo, for floating-point numbers, which uses a carefully designed erasing method different from Elf. The Pongo algorithm employs a novel erasing technique that transforms the binary representation of fractional parts to decimal, leveraging a newly proposed algorithm that enhances the efficiency of this conversion process. To demonstrate the superior performance of Pongo, we conducted extensive experiments comparing it with ten leading compression algorithms across twenty-two different datasets. On average, Pongo achieves a compression ratio that is 14% better than Elf and 58% better than Gorilla, making it the top-performing algorithm among all those tested, as shown through both mathematical analysis and practical testing.