Security News > 2020 > August > A new project enables data to be read directly from compressed IoT data

The new technique provides possibility to analyze data directly on compressed files, and it may have a major impact on the so-called "Data tsunami" from massive amounts of IoT devices.

"Today, if you need just 1 Byte of data from a 100 MB compressed file, you usually have to decompress a significant part of the whole file to access to the data. Our technology enables random access to the compressed data. It means that you can access 1 Byte data at the cost of only decompressing less than 100 Bytes, which is several orders of magnitude lower compared to the state-of-the-art technologies. This could have a huge impact on data accessibility, data processing speed and the cloud storage infrastructure," says Associate Professor Qi Zhang from Aarhus University.

The typical data analytics could be carried out directly on the compressed data.

"Today, IoT data is constantly being streamed to the cloud, and as consequence of the massive amounts of IoT devices deployed globally an exponential data growth is expected. Conventionally, to allow fast frequent data retrieval and analysis, it is preferable to store the data in an uncompressed form."

"The drawback here is the use of more storage space. If you keep the data in compressed form; however, it takes time to decompress the data first before you can access and analyze it. Our project outcome has the potential not only to reduce data storage space but also to accelerate data analysis," says Qi Zhang.

News URL

http://feedproxy.google.com/~r/HelpNetSecurity/~3/wigupyXyyPI/