Kenny Daniel
f5274904b7
Add onPage callback to parquetRead
2025-04-10 23:29:58 -07:00
Kenny Daniel
90be536e05
Group selection of a row group into an object
2025-04-10 22:36:10 -07:00
Kenny Daniel
4df7095ab4
Group column decoding params into an object
2025-04-10 19:30:25 -07:00
Kenny Daniel
6c225888c4
Skip unnecessary pages
...
Do this by passing rowGroupStart and rowGroupEnd for the rows to
fetch within a rowgroup. If a page is outside those bounds, we can
skip the page. Replaces rowLimit.
2025-04-07 00:40:17 -07:00
Kenny Daniel
ba74d58dd3
Test for reading the last row of files
2025-04-06 22:05:58 -07:00
Kenny Daniel
b38b65f7c7
Refactor assembleLists to take a schemaPath
2025-04-02 23:39:55 -07:00
Kenny Daniel
f37b2aea9f
for is faster than forEach
2025-03-17 10:18:01 -07:00
Kenny Daniel
d7f8d39de3
Return typed arrays in onChunk. Change readColumn to return DecodedArray[]. ( #67 )
...
Refactored readColumn to avoid `concat` operations.
This avoids extra copying and allocation.
2025-03-10 23:33:47 -07:00
Kenny Daniel
791a847e42
Revert "Simplify relative import paths"
...
This reverts commit e590f4ee03263460a389bdd29678015727cdcd5a.
2025-03-06 08:54:32 -08:00
Kenny Daniel
e590f4ee03
Simplify relative import paths
2025-03-05 14:03:17 -08:00
Kenny Daniel
2456cdc85f
Better error messages
2025-03-04 11:05:22 -08:00
Kenny Daniel
f762dba6a8
Use ParquetReadOptions type for parquetRead options ( #51 )
2024-12-10 16:16:52 -08:00
Sylvain Lesage
09ae9400c5
build types before publishing to npm ( #46 )
...
* build types before publishing to npm
* use prepare instead of prepublishOnly + make it clear that we only build types
doc for prepare vs prepublishOnly is here: https://docs.npmjs.com/cli/v8/using-npm/scripts
* no jsx in this lib
* relative imports from the root, so that it works from types/
* remove unused hyparquet.d.ts + report differences to jsdoc in files
* try to understand if this is the cause of the failing CI check
tsc fails: https://github.com/hyparam/hyparquet/actions/runs/12040954822/job/33571851170?pr=46
* Revert "try to understand if this is the cause of the failing CI check"
This reverts commit 5e2fc8ca179064369de71793ab1cda3facefddc7.
* not sure what happens, but we just need to ensure the types are created correctly
* increment version
* Explicitly export types for use in downstream typescript projects
* Use new typescript jsdoc imports for smaller package
* Combine some files and use @import jsdoc
* use the local typescript
---------
Co-authored-by: Kenny Daniel <platypii@gmail.com>
2024-12-02 17:47:42 +01:00
Kenny Daniel
b1c8a1dd8b
Revert onComplete type signature change from #25
...
The type change caused a lot of downstream type errors.
If you pass rowFormat: 'object' then it will return Record<string, any>[]
instead of any[][]. This means the types are not aligned with behavior.
Will figure out how to fix it later, for now don't want break downstream projects.
2024-08-14 22:00:32 -07:00
ctranstrum
8ace1a47d2
return column names in the order requested ( #27 )
...
* return column names in the order requested
* retain correct ordering of columns in object rows as well
2024-08-14 00:01:47 -07:00
ctranstrum
d13d52b606
Add an option to return each row as an object keyed by column name ( #25 )
...
* Add an option to return each row as an object keyed by column name
* rename option to rowFormat and address feedback
2024-08-13 09:15:59 -07:00
Kenny Daniel
58a6b963a1
Fix out of order columns in onComplete
2024-07-22 21:45:18 -07:00
Kenny Daniel
ddb8b16cd0
Fix handling of multiple pages
2024-06-07 23:16:04 -07:00
Kenny Daniel
6d769a4336
Demo: move to folder, typecheck, and render column indices
2024-05-31 19:40:44 -07:00
Kenny Daniel
c4ad05e580
Convert byte arrays to utf8 by default
2024-05-22 22:40:21 -07:00
Kenny Daniel
70387fa345
repeated_no_annotation.parquet
2024-05-20 23:09:31 -07:00
Kenny
cf4c4ba04d
Assembly of nested column types ( #11 )
2024-05-17 22:44:03 -07:00
Kenny Daniel
034e9cda16
Faster row transpose
2024-05-14 17:13:24 -07:00
Kenny Daniel
797db7b4df
Simplify isListLike and isMapLike
2024-05-06 13:33:39 -07:00
Kenny Daniel
12dc5a47f8
Add path to schemaTree
2024-05-06 13:23:18 -07:00
Kenny Daniel
93ff9a9f99
Refactor isListLike and isMapLike to use schemaPath
2024-04-29 18:45:29 -07:00
Kenny Daniel
2c6a111113
Refactor to use schemaPath
2024-04-29 17:38:26 -07:00
Kenny Daniel
a42cc558d0
Adjust read coalesce size
2024-04-29 16:43:07 -07:00
Kenny Daniel
54ef1e6b0a
Rewrite dremel assembly
2024-04-28 19:03:39 -07:00
Kenny Daniel
09ea11517c
Fix typescript definitions
2024-04-26 14:01:00 -07:00
Kenny Daniel
86273b110c
PageType enum to string
2024-04-18 00:21:13 -07:00
Kenny Daniel
00cbb6a3e6
Fix rowStart in onChunk callback.
...
Also split out readFiles tests.
2024-04-12 23:34:56 -07:00
Kenny Daniel
6ffdeca103
Fast array concat
2024-04-07 09:59:37 -07:00
Kenny Daniel
429fd9e813
Fix max call stack error in browser: concat not spread...
2024-04-06 22:01:42 -07:00
Kenny Daniel
48dc10fd18
Error handling for undefined parquet file
2024-04-05 11:11:39 -07:00
Kenny Daniel
46df1ab454
Rewrite dremel assembly
2024-03-21 16:47:21 -07:00
Kenny Daniel
b0d653f342
Undefined int map
2024-03-19 09:09:15 -07:00
Kenny Daniel
b14809a71a
Fix struct map parsing
2024-03-14 19:53:56 -07:00
Kenny Daniel
76c4278f40
List-like column names
2024-03-14 19:38:04 -07:00
Kenny Daniel
2d061392b9
Column filter by name
2024-03-14 18:04:04 -07:00
Kenny Daniel
e3b5fca883
Custom decompressors
2024-02-27 09:05:02 -08:00
Kenny Daniel
c70b3b2227
Prepare for data page v2
2024-02-26 11:44:28 -08:00
Kenny Daniel
a7e5aef31f
decompressPage for dictionary and data page v1 only
2024-02-24 12:12:38 -08:00
Kenny Daniel
69d2e725dd
Fix plain_dictionary encoding
2024-02-11 20:45:53 -08:00
Kenny Daniel
961b92650c
Never copy data
2024-02-09 14:35:11 -08:00
Kenny Daniel
6d03bd6d86
Fix definition level parsing
2024-01-20 18:18:32 -08:00
Kenny Daniel
647056da8b
Async parquetRead with options
2024-01-15 15:11:03 -08:00
