Kenny Daniel
113fbe3ca8
Move hyparquet.js to index.js ( #84 )
2025-05-30 15:47:02 -07:00
Kenny Daniel
f23b2757ca
Node-specific exports for asyncBufferFromFile ( #80 )
...
* Update README for asyncBufferFromFile
* Simplify asyncBufferFromFile
2025-05-30 13:01:20 -07:00
Kenny Daniel
bf6ac3b644
Simplify error messages
2025-05-25 17:49:39 -07:00
Kenny Daniel
e4504c524d
Fast filter by loading each row group and filtering until rowEnd ( #78 )
2025-05-19 02:13:37 -07:00
Kenny Daniel
0e6d7dee6f
Parquet Query Planner: plan byte ranges, pre-fetch in parallel ( #75 )
...
* Parquet Query Planner: plan byte ranges, pre-fetch in parallel.
- parquetPlan() that returns lists of byte ranges to fetch.
- prefetchAsyncBuffer() pre-fetches all byte ranges in parallel.
throws exception if non-pre-fetched slice is requested later.
2025-04-30 00:49:40 -07:00
Kenny Daniel
b7db4653e7
Add another column to page_indexed test
2025-04-26 17:18:11 -07:00
Kenny Daniel
f5274904b7
Add onPage callback to parquetRead
2025-04-10 23:29:58 -07:00
Kenny Daniel
6c225888c4
Skip unnecessary pages
...
Do this by passing rowGroupStart and rowGroupEnd for the rows to
fetch within a rowgroup. If a page is outside those bounds, we can
skip the page. Replaces rowLimit.
2025-04-07 00:40:17 -07:00
Kenny Daniel
ba74d58dd3
Test for reading the last row of files
2025-04-06 22:05:58 -07:00
Kenny Daniel
d7f8d39de3
Return typed arrays in onChunk. Change readColumn to return DecodedArray[]. ( #67 )
...
Refactored readColumn to avoid `concat` operations.
This avoids extra copying and allocation.
2025-03-10 23:33:47 -07:00
Kenny Daniel
2cd582ea5a
Remove unnecessary toJson in tests
2025-03-10 19:32:31 -07:00
Kenny Daniel
2456cdc85f
Better error messages
2025-03-04 11:05:22 -08:00
Kenny Daniel
df02229407
Promisified parquetReadObjects function
2024-08-20 11:30:39 -07:00
ctranstrum
8ace1a47d2
return column names in the order requested ( #27 )
...
* return column names in the order requested
* retain correct ordering of columns in object rows as well
2024-08-14 00:01:47 -07:00
ctranstrum
d13d52b606
Add an option to return each row as an object keyed by column name ( #25 )
...
* Add an option to return each row as an object keyed by column name
* rename option to rowFormat and address feedback
2024-08-13 09:15:59 -07:00
Kenny Daniel
a5122e61d6
utils: asyncBufferFromFile
2024-07-26 15:07:47 -07:00
Kenny Daniel
ddb8b16cd0
Fix handling of multiple pages
2024-06-07 23:16:04 -07:00
Kenny Daniel
561f06f701
Int_Map test is redundant with nullable.impala.parquet
2024-05-18 18:33:15 -07:00
Kenny Daniel
5412ff9bb7
Tests, assemble
2024-05-09 15:46:38 -07:00
Kenny Daniel
09ea11517c
Fix typescript definitions
2024-04-26 14:01:00 -07:00
Kenny Daniel
4b7d864319
Error handling for missing byteLength
2024-04-17 17:48:35 -07:00
Kenny Daniel
00cbb6a3e6
Fix rowStart in onChunk callback.
...
Also split out readFiles tests.
2024-04-12 23:34:56 -07:00
Kenny Daniel
6ffdeca103
Fast array concat
2024-04-07 09:59:37 -07:00
Kenny Daniel
48dc10fd18
Error handling for undefined parquet file
2024-04-05 11:11:39 -07:00
Kenny Daniel
b0d653f342
Undefined int map
2024-03-19 09:09:15 -07:00
Kenny Daniel
4654c5eddf
Fix tests
2024-03-15 00:17:35 -07:00
Kenny Daniel
b14809a71a
Fix struct map parsing
2024-03-14 19:53:56 -07:00
Kenny Daniel
76c4278f40
List-like column names
2024-03-14 19:38:04 -07:00
Kenny Daniel
2d061392b9
Column filter by name
2024-03-14 18:04:04 -07:00
Kenny Daniel
0f4708b954
Change compressors to return Uint8Array
2024-02-27 19:45:56 -08:00
Kenny Daniel
e3b5fca883
Custom decompressors
2024-02-27 09:05:02 -08:00
Kenny Daniel
87d78ab06e
Oops fix the other tests
2024-02-26 22:51:57 -08:00
Kenny Daniel
a65132b79c
Data Page V2
2024-02-26 18:44:33 -08:00
Kenny Daniel
cfaa0bca90
Test each file instead of all files
2024-02-24 11:05:40 -08:00
Kenny Daniel
d02c68e883
Fix isRequired and add parquet-testing impala test
2024-02-16 16:07:09 -08:00
Kenny Daniel
054431c98e
Test read of all sample files
2024-02-13 21:12:11 -08:00
Kenny Daniel
8484426bc8
Split out schemaTree tests
2024-01-20 12:28:36 -08:00
Kenny Daniel
647056da8b
Async parquetRead with options
2024-01-15 15:11:03 -08:00
