diff --git a/docz/docs/03-demos/06-content.md b/docz/docs/03-demos/06-content.md index 6e3f420..f8feda6 100644 --- a/docz/docs/03-demos/06-content.md +++ b/docz/docs/03-demos/06-content.md @@ -17,7 +17,7 @@ to load data from spreadsheets. :::note -This was tested against `lume v1.12.0` on 2022 October 4. +This was tested against `lume v1.14.2` on 2022 December 27. ::: @@ -28,7 +28,7 @@ This was tested against `lume v1.12.0` on 2022 October 4. ```bash mkdir -p sheetjs-lume cd sheetjs-lume -deno run -A https://deno.land/x/lume/init.ts +deno run -Ar https://deno.land/x/lume/init.ts ``` When prompted, enter the following options: @@ -132,7 +132,7 @@ The following query pulls the `Name` and `Index` fields from each row: allPresXlsxSheet1 { # "all" followed by type edges { node { # each line in this block should be a field in the data - Name + Name Index } } @@ -157,6 +157,8 @@ overridden through a `package.json` override in the latest versions of NodeJS: ::: +#### GatsbyJS Demo +
Complete Example (click to show) :::note @@ -471,7 +473,7 @@ ${csv} :::note -This was tested against `next v12.2.5` on 2022 August 16. +This was tested against `next v13.1.1` on 2022 December 28. ::: @@ -500,7 +502,7 @@ export default function Index({html, type}) { return ( ); } ``` -:::warning +:::warning Reading and writing files during the build process `fs` cannot be statically imported from the top level in NextJS pages. The dynamic import must happen within a lifecycle function. For example: @@ -523,6 +525,20 @@ export async function getServerSideProps() { ::: +:::caution Next 13+ and SWC + +Next 13 switched to the SWC minifier. There are known issues with the minifier. +Until those issues are resolved, SWC should be disabled in `next.config.js`: + +```js title="next.config.js" +module.exports = { +// highlight-next-line + swcMinify: false +}; +``` + +::: + #### Demo
Complete Example (click to show) @@ -530,13 +546,13 @@ export async function getServerSideProps() { 0) Disable NextJS telemetry: ```js -npx next telemetry disable +npx next@13.1.1 telemetry disable ``` Confirm it is disabled by running ```js -npx next telemetry status +npx next@13.1.1 telemetry status ``` 1) Set up folder structure. At the end, a `pages` folder with a `sheets` @@ -556,7 +572,7 @@ curl -LO https://docs.sheetjs.com/next/sheetjs.xlsx 3) Install dependencies: ```bash -npm i --save https://cdn.sheetjs.com/xlsx-latest/xlsx-latest.tgz next@12.2.5 +npm i --save https://cdn.sheetjs.com/xlsx-latest/xlsx-latest.tgz next@13.1.1 ``` 4) Download test scripts: @@ -594,7 +610,7 @@ cd ../.. 5) Test the deployment: ```bash -npx next@12.2.5 +npx next@13.1.1 ``` Open a web browser and access: @@ -613,19 +629,19 @@ The individual worksheets are available at 6) Stop the server and run a production build: ```bash -npx next@12.2.5 build +npx next@13.1.1 build ``` The final output will show a list of the routes and types: ``` Route (pages) Size First Load JS -┌ ○ / 551 B 81.7 kB -├ ○ /404 194 B 77.2 kB -├ λ /getServerSideProps 602 B 81.7 kB -├ ● /getStaticPaths 2.7 kB 83.8 kB -├ ● /getStaticProps 600 B 81.7 kB -└ ● /sheets/[id] (312 ms) 580 B 81.7 kB +┌ ○ / 541 B 77.4 kB +├ ○ /404 181 B 73.7 kB +├ λ /getServerSideProps 594 B 77.4 kB +├ ● /getStaticPaths 2.56 kB 79.4 kB +├ ● /getStaticProps 591 B 77.4 kB +└ ● /sheets/[id] (447 ms) 569 B 77.4 kB ├ /sheets/0 ├ /sheets/1 └ /sheets/2 @@ -638,7 +654,7 @@ worksheets in the file. `/getServerSideProps` is server-rendered. 7) Try to build a static site: ```bash -npx next@12.2.5 export +npx next@13.1.1 export ``` :::note The static export will fail! @@ -648,17 +664,17 @@ is still server-rendered. ::: -8) Remove `pages/getServerSideProps.js` and rebuild with `npx next@12.2.5 build` +8) Remove `pages/getServerSideProps.js` and rebuild with `npx next@13.1.1 build` Inspecting the output, there should be no lines with the `λ` symbol: ``` Route (pages) Size First Load JS -┌ ○ / 551 B 81.7 kB -├ ○ /404 194 B 77.2 kB -├ ● /getStaticPaths 2.7 kB 83.8 kB -├ ● /getStaticProps 600 B 81.7 kB -└ ● /sheets/[id] (312 ms) 580 B 81.7 kB +┌ ○ / 541 B 77.4 kB +├ ○ /404 181 B 73.7 kB +├ ● /getStaticPaths 2.56 kB 79.4 kB +├ ● /getStaticProps 591 B 77.4 kB +└ ● /sheets/[id] (459 ms) 569 B 77.4 kB ├ /sheets/0 ├ /sheets/1 └ /sheets/2 @@ -667,7 +683,7 @@ Route (pages) Size First Load JS 9) Generate the static site: ```bash -npx next@12.2.5 export +npx next@13.1.1 export ``` The static site will be written to the `out` subfolder, which can be hosted with diff --git a/docz/docs/03-demos/07-worker.md b/docz/docs/03-demos/07-worker.md index e0cb600..a68b9a8 100644 --- a/docz/docs/03-demos/07-worker.md +++ b/docz/docs/03-demos/07-worker.md @@ -6,7 +6,60 @@ Parsing and writing large spreadsheets takes time. During the process, if the SheetJS library is running in the web browser, the website may freeze. Workers provide a way to off-load the hard work so that the website does not -freeze during processing. +freeze during processing. The work is still performed locally. No data is sent +to a remote server. + +The following diagrams show the normal and Web Worker flows when exporting a +dataset. The regions with a red background mark when the browser is frozen. + + +
Normal ExportWeb Worker Export
+ +```mermaid +sequenceDiagram + autonumber + actor User + participant Page + User->>Page: click button + rect rgba(255,0,0,.2) + activate Page + Note over Page: collect dataset + Note over Page: generate workbook + Note over Page: create file + Note over Page: setup download + end + Page->>User: download workbook + deactivate Page +``` + + + +```mermaid +sequenceDiagram + autonumber + actor User + participant Page + participant Worker + User->>Page: click button + rect rgba(255,0,0,.2) + activate Page + Note over Page: collect dataset + Page->>Worker: transfer dataset + end + deactivate Page + activate Worker + Note over Worker: generate workbook + Note over Worker: create file + Note over Worker: setup download + Worker->>Page: URL to download + deactivate Worker + activate Page + Page->>User: download workbook + deactivate Page +``` + +
+ :::note Browser Compatibility @@ -17,7 +70,7 @@ added later. Feature testing is highly recommended. :::info Inline Workers -Due to limitations of the live codeblocks, all of the workers in this section +Due to limitations of the live code blocks, all of the workers in this section are in-line. The code is embedded in template literals. For production sites, typically workers are written in separate JS files. @@ -119,7 +172,31 @@ const worker = new Worker( ::: Typically the Web Worker performs the `fetch` operation, processes the workbook, -and sends a final result to the main browser context for processing. +and sends a final result (HTML table or raw data) to the main browser context: + +```mermaid +sequenceDiagram + autonumber + actor User + participant Page + participant Worker + User->>Page: click button + activate Page + Page->>Worker: send URL + deactivate Page + activate Worker + Note over Worker: fetch file + Note over Worker: parse file + Note over Worker: generate table + Worker->>Page: HTML table + deactivate Worker + activate Page + Note over Page: add to DOM + Page->>User: table is visible + deactivate Page +``` + +
Live Demo (click to show) In the following example, the script: @@ -172,6 +249,8 @@ self.addEventListener('message', async(e) => { } ``` +
+ ## Creating a Local File :::caution Writing files from Web Workers @@ -181,8 +260,36 @@ the Web Worker to the main browser context for downloading. ::: +Typically the Web Worker receives an array of JS objects, generates a workbook, +and sends a URL to the main browser context for download: + +```mermaid +sequenceDiagram + autonumber + actor User + participant Page + participant Worker + User->>Page: click button + activate Page + Note over Page: collect dataset + Page->>Worker: transfer dataset + deactivate Page + activate Worker + Note over Worker: generate workbook + Note over Worker: create file + Note over Worker: setup download + Worker->>Page: URL to download + deactivate Worker + activate Page + Page->>User: download workbook + deactivate Page +``` + +
Live Demo (click to show) + In the following example, the script: +- sends a dataset (array of JS objects) to the Web Worker - generates a workbook object in the Web Worker - generates a XLSB file using `XLSX.write` in the Web Worker - generates an object URL in the Web Worker @@ -193,6 +300,11 @@ In the following example, the script: function SheetJSWriteFileWorker() { const [__html, setHTML] = React.useState(""); + const data = [ + { "SheetJS": "வணக்கம்", "in": "สวัสดี", "Web": "你好", "Workers": "가지마" }, + { "SheetJS": 1, "in": 2, "Web": 3, "Workers": 4 }, + ]; + return ( <>
); } ``` +
+ ## User-Submitted File :::note FileReaderSync @@ -253,6 +365,33 @@ synchronous version `FileReaderSync` is more efficient. ::: +Typically the Web Worker receives a file pointer, reads and parses the file, +and sends a final result (HTML table or raw data) to the main browser context: + +```mermaid +sequenceDiagram + autonumber + actor User + participant Page + participant Worker + User->>Page: click button + activate Page + Page->>Worker: send URL + deactivate Page + activate Worker + Note over Worker: fetch file + Note over Worker: parse file + Note over Worker: generate table + Worker->>Page: HTML table + deactivate Worker + activate Page + Note over Page: add to DOM + Page->>User: table is visible + deactivate Page +``` + +
Live Demo (click to show) + In the following example, when a file is dropped over the DIV or when the INPUT element is used to select a file, the script: @@ -312,11 +451,15 @@ self.addEventListener('message', (e) => { } ``` +
+ ## Streaming Write A more general discussion, including row-oriented processing demos, is included in the ["Large Datasets"](/docs/demos/stream#browser) demo. +`XLSX.stream.to_csv` incrementally generates CSV rows. + #### File System Access API :::warning Browser Compatibility @@ -336,6 +479,41 @@ Strings larger than 500M may hit browser length limits. ::: +```mermaid +sequenceDiagram + autonumber + actor User + participant Page + participant Worker + User->>Page: click button + activate Page + Page->>User: seek permission + User->>Page: grant permission + Note over Page: collect dataset + Page->>Worker: transfer dataset + deactivate Page + activate Worker + Note over Worker: setup stream + Worker->>User: start download + loop every 100 rows + Note over Worker: generate rows + Worker->>User: write rows to file (without freezing) + Worker->>Page: progress message + activate Page + Page->>User: display progress + deactivate Page + end + Worker->>User: finish download + Worker->>Page: send competion message + deactivate Worker + activate Page + Page->>User: download complete + deactivate Page +``` + + +
Live Demo (click to show) + The following live demo fetches and parses a file in a Web Worker. The script: - prompts user to save file (`window.showSaveFilePicker` in the main thread) @@ -461,3 +639,5 @@ self.addEventListener('message', async(e) => { ); } ``` + +
diff --git a/docz/docs/09-miscellany/02-errors.md b/docz/docs/09-miscellany/02-errors.md index 1946bd1..d93206a 100644 --- a/docz/docs/09-miscellany/02-errors.md +++ b/docz/docs/09-miscellany/02-errors.md @@ -316,4 +316,16 @@ function add_to_sheet(sheet, cell) { } ``` -
\ No newline at end of file +
+ +#### Corrupt files + +Third-party build tools and frameworks may post-process SheetJS scripts. The +changes may result in corrupt files. + +In the web browser, the standalone scripts from the SheetJS CDN will use proper +encodings and should work in applications. The scripts typically can be added +to an `index.html` file, bypassing any third-party post-processing + +There are known bugs with the SWC minifier (used in Next.js 13+). The original +minifier can be enabled by setting `swcMinify: false` in `next.config.js`. diff --git a/docz/docs/index.md b/docz/docs/index.md index e40ed31..3485297 100644 --- a/docz/docs/index.md +++ b/docz/docs/index.md @@ -7,7 +7,7 @@ title: Overview # SheetJS CE ![License](https://img.shields.io/github/license/SheetJS/sheetjs) -[![Build Status](https://img.shields.io/github/workflow/status/sheetjs/sheetjs/Tests:%20node.js)](https://github.com/SheetJS/sheetjs/actions) +[![Build Status](https://img.shields.io/github/actions/workflow/status/sheetjs/sheetjs/node-4+.yml?branch=master)](https://github.com/SheetJS/sheetjs/actions/workflows/node-4+.yml) [![Vulnerabilities](https://img.shields.io/snyk/vulnerabilities/github/SheetJS/sheetjs)](https://snyk.io/test/github/SheetJS/sheetjs) [![npm Downloads](https://img.shields.io/npm/dm/xlsx.svg)](https://cdn.sheetjs.com/) [![GitHub stars](https://img.shields.io/github/stars/SheetJS/sheetjs?style=social)](https://github.com/SheetJS/sheetjs) diff --git a/docz/docusaurus.config.js b/docz/docusaurus.config.js index 14b319d..d89754a 100644 --- a/docz/docusaurus.config.js +++ b/docz/docusaurus.config.js @@ -149,8 +149,12 @@ const config = { } }), themes: [ - "@docusaurus/theme-live-codeblock" + "@docusaurus/theme-live-codeblock", + "@docusaurus/theme-mermaid" ], + markdown: { + mermaid: true + }, scripts: [ { src: "https://cdn.sheetjs.com/xlsx-latest/package/dist/xlsx.full.min.js", diff --git a/docz/package.json b/docz/package.json index 7f230a7..627b182 100644 --- a/docz/package.json +++ b/docz/package.json @@ -20,6 +20,7 @@ "@docusaurus/preset-classic": "2.2.0", "@docusaurus/theme-common": "2.2.0", "@docusaurus/theme-live-codeblock": "2.2.0", + "@docusaurus/theme-mermaid": "2.2.0", "@mdx-js/react": "1.6.22", "clsx": "1.2.1", "prism-react-renderer": "1.3.5", diff --git a/docz/static/next/getStaticPaths.js b/docz/static/next/getStaticPaths.js index 09affec..69b12bc 100644 --- a/docz/static/next/getStaticPaths.js +++ b/docz/static/next/getStaticPaths.js @@ -16,7 +16,7 @@ export default function Index({type, snames}) { return (
Each worksheet maps to a path: