Commit Graph

375 Commits

Author SHA1 Message Date
Kenny Daniel
4df7095ab4
Group column decoding params into an object 2025-04-10 19:30:25 -07:00
Kenny Daniel
4645e34f97
Re-order types.d.ts to put important apis up front 2025-04-10 16:33:50 -07:00
Kenny Daniel
8740f14450
Publish v1.11.1 2025-04-09 17:35:49 -07:00
Kenny Daniel
972402d083
Fix handling of dictionary pages from parquet.net 2025-04-09 17:26:47 -07:00
Kenny Daniel
c2adfba626
Publish v1.11.0 2025-04-07 21:18:17 -07:00
Kenny Daniel
655444bcde
Fix continued data pages
Parquet allows consecutive pages to continue a previously assembled
list. Broke in hyparquet 1.9.0. Added continued_page.parquet test.
2025-04-07 17:40:23 -07:00
Kenny Daniel
6c225888c4
Skip unnecessary pages
Do this by passing rowGroupStart and rowGroupEnd for the rows to
fetch within a rowgroup. If a page is outside those bounds, we can
skip the page. Replaces rowLimit.
2025-04-07 00:40:17 -07:00
Kenny Daniel
ba74d58dd3
Test for reading the last row of files 2025-04-06 22:05:58 -07:00
Kenny Daniel
8c4c7456b4
Publish v1.10.4 2025-04-03 20:03:21 -07:00
Kenny Daniel
f9a10da20b
Type thrift 2025-04-03 19:20:00 -07:00
Kenny Daniel
474cd9053a
Publish v1.10.3 2025-04-02 23:53:25 -07:00
Kenny Daniel
b38b65f7c7
Refactor assembleLists to take a schemaPath 2025-04-02 23:39:55 -07:00
Kenny Daniel
1247f5d606
Split out readPage
Remove dict-page-offset-zero test because it's a malformed parquet file.
2025-04-02 20:27:10 -07:00
Kenny Daniel
a4b5c2a943
Publish v1.10.2 2025-03-31 23:26:35 -07:00
Kenny Daniel
6af6f43f44
Export more constants 2025-03-31 23:20:22 -07:00
Kenny Daniel
73a4f295e1
Publish v1.10.1 2025-03-25 15:50:30 -07:00
Kenny Daniel
85e1af66c1
Fix thrift parsing of crypto_metadata 2025-03-25 15:42:48 -07:00
Kenny Daniel
74e8cc8f41
Publish v1.10.0 2025-03-25 00:44:55 -07:00
Kenny Daniel
b9ad12f7cc
Map src to types with package exports (#70) 2025-03-21 04:55:33 -07:00
Kenny Daniel
9c201e00e5
Use defaultInitialFetchSize for both metadata and cachedAsyncBuffer 2025-03-20 16:05:41 -07:00
Kenny Daniel
4b094178b3
Move toVarInt to tests 2025-03-20 12:37:24 -07:00
Kenny Daniel
855d5be0ea
Publish v1.9.1 2025-03-17 23:56:13 -07:00
Kenny Daniel
95c47f243d
Add minSize parameter to cachedAsyncBuffer 2025-03-17 23:54:20 -07:00
Kenny Daniel
f37b2aea9f
for is faster than forEach 2025-03-17 10:18:01 -07:00
Kenny Daniel
97aef2adef
Publish v1.9.0 2025-03-10 23:36:46 -07:00
Kenny Daniel
d7f8d39de3
Return typed arrays in onChunk. Change readColumn to return DecodedArray[]. (#67)
Refactored readColumn to avoid `concat` operations.
This avoids extra copying and allocation.
2025-03-10 23:33:47 -07:00
Kenny Daniel
a9467f6c3d
Remove selfCopyBytes in favor of copyBytes 2025-03-10 20:56:00 -07:00
Kenny Daniel
2cd582ea5a
Remove unnecessary toJson in tests 2025-03-10 19:32:31 -07:00
Kenny Daniel
613cd0e639
Update README 2025-03-09 23:56:43 -07:00
Kenny Daniel
347cc86270
Publish v1.8.7 2025-03-09 11:22:23 -07:00
Kenny Daniel
4bbc7742e5
Comment out unnecessary length read in readRleBitPackedHybrid 2025-03-09 11:20:58 -07:00
Kenny Daniel
d950e7dfb4
Publish v1.8.6 2025-03-06 08:56:15 -08:00
Kenny Daniel
791a847e42
Revert "Simplify relative import paths"
This reverts commit e590f4ee03263460a389bdd29678015727cdcd5a.
2025-03-06 08:54:32 -08:00
Kenny Daniel
03c25eef7f
Publish v1.8.5 2025-03-05 15:50:04 -08:00
Kenny Daniel
e590f4ee03
Simplify relative import paths 2025-03-05 14:03:17 -08:00
Kenny Daniel
2456cdc85f
Better error messages 2025-03-04 11:05:22 -08:00
Kenny Daniel
c70d5c2732
Publish v1.8.4 2025-02-22 13:33:33 -08:00
Kenny Daniel
2a302702d4
Fix handling of boolean rle 2025-02-22 13:29:29 -08:00
Kenny Daniel
312557a83a
Publish v1.8.3 2025-02-19 11:09:29 -08:00
Johan Levin
bf268e141c
Use prepended length for bit-packed hybrid bool columns (#62) 2025-02-19 11:07:49 -08:00
Kenny Daniel
cad751e546
Publish v1.8.2 2025-02-07 18:54:57 -08:00
Kenny Daniel
36d8ea2e1d
Fix handling of signed decimals (#60) 2025-02-07 18:52:48 -08:00
Kenny Daniel
5675560266
Use bigint literals 2025-02-07 17:50:34 -08:00
Kenny Daniel
3c5dbab649
Publish v1.8.1 2025-01-16 12:04:23 -08:00
Sean Lynch
725545731d
Support endpoints that don't support range requests in asyncBufferFromUrl (#57)
* Support endpoints that don't support range requests in asyncBufferFromUrl

Before this commit asyncBufferFromUrl assumes that the body of whatever
successful response it gets is equivalent to the range it requested. If
the origin server does not support HTTP range requests then this
assumption is usually wrong and will lead to parsing failures.

This commit changes asyncBufferFromUrl to change its behaviour slightly
based on the status code in the response:
- if 200 then we got the whole parquet file as the response. Save it and
  use the resulting ArrayBuffer to serve all future slice calls.
- if 206 then we got a range response and we can just return that.

I have also included some test cases to ensure that such responses are
handled correctly and also tweaked other existing mocks to also include
the relevant status code.

* Fix all lint warnings

* replace switch with if-else
2025-01-16 11:55:05 -08:00
Kenny Daniel
248aeaa5e5
Publish v1.8.0 2024-12-21 15:47:48 -08:00
Kenny Daniel
870187c7de
Update README with Awaitable 2024-12-21 15:31:59 -08:00
Brian Park
c9727a4246
Query filter (#56)
* implement ParquetQueryFilter types

* implement parquetQuery filter tests

* implement parquetQuery filter

* filter before ordering

* apply filters before sorting/slicing

* format types

* add deep equality utility

* document and format equals utility

* use deep equality checks

* update filter tests

* support more types for equality

* make $not unary

* ensure arrays are correctly compared

* support both forms of $not

* add operator tests

* Filter operator tests

---------

Co-authored-by: Brian Park <park-brian@users.noreply.github.com>
Co-authored-by: Kenny Daniel <platypii@gmail.com>
2024-12-21 15:23:57 -08:00
Sylvain Lesage
cb639a0b45
factor tests with it.for() (#55) 2024-12-20 09:53:56 +01:00
Kenny Daniel
90f8a3b775
Publish v1.7.0 2024-12-19 18:17:26 -08:00