asadbek064/hyparquet-compressors
1
Fork 0
mirror of https://github.com/asadbek064/hyparquet-compressors.git synced 2026-01-11 21:26:38 +00:00
Code Issues Actions Projects Releases Wiki Activity
c73ac7e5b7
hyparquet-compressors/README.md
Kenny Daniel c73ac7e5b7
Publish v0.1.0
2024-05-20 01:06:33 -07:00

1.7 KiB
Raw Blame History

hyparquet decompressors

npm workflow status mit license coverage

This package exports a compressors object intended to be passed into hyparquet.

Apache Parquet is a popular columnar storage format that is widely used in data engineering, data science, and machine learning applications for efficiently storing and processing large datasets. It supports a number of different compression formats, but most parquet files use snappy compression.

The hyparquet library by default only supports uncompressed and snappy compressed files. The hyparquet-compressors package extends support for all legal parquet compression formats.

Usage

import { parquetRead } from 'hyparquet'
import { compressors } from 'hyparquet-compressors'

await parquetRead({ file, compressors, onComplete: console.log })

Supported compression formats

Parquet compression types supported with hyparquet-compressors:

  • Uncompressed
  • Snappy
  • GZip
  • LZO
  • Brotli
  • LZ4
  • ZSTD
  • LZ4_RAW

References