Commit Graph

24 Commits

Author SHA1 Message Date
Kenny Daniel
f23b2757ca
Node-specific exports for asyncBufferFromFile (#80)
* Update README for asyncBufferFromFile
* Simplify asyncBufferFromFile
2025-05-30 13:01:20 -07:00
Kenny Daniel
78f19aaf6d
Move readRowGroup to rowgroup.js 2025-05-25 14:55:30 -07:00
Sylvain Lesage
447a58eca4
pass custom fetch function to utils (#73)
* pass custom fetch function to utils

it can be used to implement retry logic.

* Update src/utils.js

Co-authored-by: Kenny Daniel <platypii@gmail.com>

---------

Co-authored-by: Kenny Daniel <platypii@gmail.com>
2025-04-15 00:37:05 +02:00
Kenny Daniel
ba74d58dd3
Test for reading the last row of files 2025-04-06 22:05:58 -07:00
Kenny Daniel
9c201e00e5
Use defaultInitialFetchSize for both metadata and cachedAsyncBuffer 2025-03-20 16:05:41 -07:00
Kenny Daniel
95c47f243d
Add minSize parameter to cachedAsyncBuffer 2025-03-17 23:54:20 -07:00
Kenny Daniel
791a847e42
Revert "Simplify relative import paths"
This reverts commit e590f4ee03263460a389bdd29678015727cdcd5a.
2025-03-06 08:54:32 -08:00
Kenny Daniel
e590f4ee03
Simplify relative import paths 2025-03-05 14:03:17 -08:00
Sean Lynch
725545731d
Support endpoints that don't support range requests in asyncBufferFromUrl (#57)
* Support endpoints that don't support range requests in asyncBufferFromUrl

Before this commit asyncBufferFromUrl assumes that the body of whatever
successful response it gets is equivalent to the range it requested. If
the origin server does not support HTTP range requests then this
assumption is usually wrong and will lead to parsing failures.

This commit changes asyncBufferFromUrl to change its behaviour slightly
based on the status code in the response:
- if 200 then we got the whole parquet file as the response. Save it and
  use the resulting ArrayBuffer to serve all future slice calls.
- if 206 then we got a range response and we can just return that.

I have also included some test cases to ensure that such responses are
handled correctly and also tweaked other existing mocks to also include
the relevant status code.

* Fix all lint warnings

* replace switch with if-else
2025-01-16 11:55:05 -08:00
Brian Park
c9727a4246
Query filter (#56)
* implement ParquetQueryFilter types

* implement parquetQuery filter tests

* implement parquetQuery filter

* filter before ordering

* apply filters before sorting/slicing

* format types

* add deep equality utility

* document and format equals utility

* use deep equality checks

* update filter tests

* support more types for equality

* make $not unary

* ensure arrays are correctly compared

* support both forms of $not

* add operator tests

* Filter operator tests

---------

Co-authored-by: Brian Park <park-brian@users.noreply.github.com>
Co-authored-by: Kenny Daniel <platypii@gmail.com>
2024-12-21 15:23:57 -08:00
Kenny Daniel
7ce11ad844
Validate url for asyncBufferFromUrl 2024-12-17 09:25:54 -08:00
Sylvain Lesage
09ae9400c5
build types before publishing to npm (#46)
* build types before publishing to npm

* use prepare instead of prepublishOnly + make it clear that we only build types

doc for prepare vs prepublishOnly is here: https://docs.npmjs.com/cli/v8/using-npm/scripts

* no jsx in this lib

* relative imports from the root, so that it works from types/

* remove unused hyparquet.d.ts + report differences to jsdoc in files

* try to understand if this is the cause of the failing CI check

tsc fails: https://github.com/hyparam/hyparquet/actions/runs/12040954822/job/33571851170?pr=46

* Revert "try to understand if this is the cause of the failing CI check"

This reverts commit 5e2fc8ca179064369de71793ab1cda3facefddc7.

* not sure what happens, but we just need to ensure the types are created correctly

* increment version

* Explicitly export types for use in downstream typescript projects

* Use new typescript jsdoc imports for smaller package

* Combine some files and use @import jsdoc

* use the local typescript

---------

Co-authored-by: Kenny Daniel <platypii@gmail.com>
2024-12-02 17:47:42 +01:00
Sylvain Lesage
6ec836dac5
pass requestInit to fetch utils (#34)
* pass requestInit to fetch utils

It will allow authentication

* add tests
2024-11-08 22:22:30 +01:00
Kenny Daniel
e6301a8bc8
demo: use web worker for parquet parsing to avoid blocking main thread 2024-09-25 02:22:30 -07:00
Kenny Daniel
0e807587e1
Prevent webpack from trying to include node fs 2024-08-02 16:04:22 -07:00
Kenny Daniel
a5122e61d6
utils: asyncBufferFromFile 2024-07-26 15:07:47 -07:00
Kenny Daniel
5188b3c764
utils: asyncBufferFromUrl 2024-07-26 14:12:35 -07:00
Kenny Daniel
9db378de2f
toJson tests 2024-05-28 14:24:12 -07:00
Kenny Daniel
e064efc66c
Int96 date parsing 2024-05-12 19:24:10 -07:00
Kenny Daniel
eabf62f5a1
Return Uint8Array from thrift 2024-05-04 00:47:24 -07:00
Kenny Daniel
4d5c8324aa
TypedArrays 2024-05-01 23:23:55 -07:00
Kenny Daniel
af908b9b83
No copy readBitPacked 2024-04-30 19:59:58 -07:00
Kenny Daniel
195ee9bc80
Splice into fixed length array 2024-04-30 13:43:01 -07:00
Kenny Daniel
6ffdeca103
Fast array concat 2024-04-07 09:59:37 -07:00