Publish v0.2.0

This commit is contained in:
Kenny Daniel 2024-01-15 15:14:11 -08:00
parent 647056da8b
commit 77f0354599
No known key found for this signature in database
GPG Key ID: 6A3C5E318BE71391
3 changed files with 5 additions and 3 deletions

1
.gitignore vendored

@ -3,3 +3,4 @@ package-lock.json
coverage
dist
*.tgz
example.parquet

@ -16,9 +16,10 @@ Dependency free since 2023!
## Features
- Designed to work with huge ML datasets (things like [starcoder](https://huggingface.co/datasets/bigcode/starcoderdata))
- Loads metadata separately from data
- Can load metadata separately from data
- Data can be filtered by row and column ranges
- Only fetches the data needed
- Written in JavaScript, checked with TypeScript
- Fast data loading for large scale ML applications
- Bring data visualization closer to the user, in the browser
@ -50,7 +51,7 @@ const { parquetMetadata } = await import('hyparquet')
const fs = await import('fs')
const buffer = fs.readFileSync('example.parquet')
const arrayBuffer = buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength)
const arrayBuffer = new Uint8Array(buffer).buffer
const metadata = parquetMetadata(arrayBuffer)
```

@ -1,6 +1,6 @@
{
"name": "hyparquet",
"version": "0.1.6",
"version": "0.2.0",
"description": "parquet file parser for javascript",
"keywords": [
"parquet",