docs.sheetjs.com/08-nextjs.md at ae138cc32776f2793747ad0dba2d07cc483657a4

Rasmus/docs.sheetjs.com

SheetJS ae138cc327 next-asset-module

2023-05-26 18:54:05 -04:00

17 KiB

Raw Blame History

title	pagination_prev	pagination_next
NextJS	demos/net/index	demos/mobile/index

import current from '/version.js'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock';

:::note

This was tested against next v13.4.4 on 2023 May 26.

:::

The NodeJS module can be imported from pages or loaded in Webpack loaders.

:::warning

import does not load NodeJS native modules. The Installation section includes a note on dynamic import of fs within lifecycle methods.

:::

NextJS best practices have evolved over time, but there are three key parts:

Loading Data: NextJS can read files in lifecycle methods OR custom Webpack loaders can create asset modules.
Lifecycle Methods: NextJS includes strategies for static pages (getStaticProps) as well as dynamic pages (getServerSideProps).
Data Presentation: Pages use React and JSX.

:::caution Next 13+ and SWC

Next 13 switched to the SWC minifier. There are known issues with the minifier. Until those issues are resolved, SWC should be disabled in next.config.js:

module.exports = {
// highlight-next-line
  swcMinify: false
};

:::

Loading Data

At a high level, there are two ways to pull spreadsheet data into NextJS apps: loading an asset module or performing the file read operations from the NextJS lifecycle methods.

Asset modules are appropriate for static sites when the file names are known in advance. Performing file read operations in lifecycle methods is more flexible but does not support live reloading.

Asset Module

:::caution

When the demo was last tested, Turbopack did not support true raw loaders. For development use, the normal npx next dev should be used.

:::

The following diagram depicts the workbook waltz:

flowchart LR
  file[(workbook\nfile)]
  subgraph SheetJS operations
    base64(base64\nstring)
    aoo(array of\nobjects)
  end
  html{{HTML\nTABLE}}
  file --> |base64-loader.js\ncustom plugin| base64
  base64 --> |page\nlifecycle method| aoo
  aoo --> |page\nIndex method| html

In this flow, it is strongly recommended to make a loader return a Base64 string:

function loader(content) {
  /* since `loader.raw` is true, `content` is a Buffer */
  return `export default '${content.toString("base64")}'`;
}
/* ensure the function receives a Buffer */
loader.raw = true;
module.exports = loader;

The webpack configuration is controlled in next.config.js:

module.exports = {
  webpack: (config) => {
    // highlight-start
    /* add to the webpack config module.rules array */
    config.module.rules.push({
      /* `test` matches file extensions */
      test: /\.(numbers|xls|xlsx|xlsb)/,
      /* use the loader script */
      use: [ { loader: './base64-loader' } ]
    });
    // highlight-end
    return config;
  }
};

Module alias directories can be defined in jsconfig.json or tsconfig.json:

{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      // highlight-next-line
      "@/*": ["*"]
    }
  }
}

Pages can import the files directly. It is strongly recommended to store files in a data folder. This example uses getStaticProps to parse sheetjs.xlsx:

import { read, utils } from 'xlsx';
// highlight-next-line
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticProps() {
  /* parse base64 data */
  // highlight-next-line
  const wb = read(base64, { type: "base64" });
  return { props: {
      /* generate array of objects from the first sheet */
      data: utils.sheet_to_json(wb.Sheets[wb.SheetNames[0]])
  } };
}

Raw Operations

Files can be read using readFile in lifecycle methods. The cwd method from the process module will point to the root of the project.

The following diagram depicts the workbook waltz:

flowchart LR
  file[(workbook\nfile)]
  subgraph SheetJS operations
    buffer(NodeJS\nBuffer)
    aoo(array of\nobjects)
  end
  html{{HTML\nTABLE}}
  file --> |page\nlifecycle method| buffer
  buffer --> |page\nlifecycle method| aoo
  aoo --> |page\nIndex method| html

This example reads the file sheetjs.xlsx in the data folder in the project:

import { readFile, utils, set_fs } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getServerSideProps() {
// highlight-start
  set_fs(await import("fs")); // dynamically import 'fs' when needed
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(filename);
  // highlight-end

  /* generate and return the html from the first worksheet */
  const data = utils.sheet_to_json(wb.Sheets[wb.SheetNames[0]]);
  return { props: { data } };
}

:::warning Reading and writing files during the build process

As the NextJS workaround is non-traditional, it bears repeating:

fs cannot be statically imported from the top level in NextJS pages. The dynamic import must happen within a lifecycle function.

:::

NextJS Strategies

NextJS currently provides 3 strategies:

"Static Site Generation" using getStaticProps
"SSG with Dynamic Routes" using getStaticPaths
"Server-Side Rendering" using getServerSideProps

Static Site Generation

When using getStaticProps, the file will be read once during build time. This example reads sheetjs.xlsx from the data folder:

import { read, utils } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticProps() {
  const wb = read(base64, { type: "base64" });

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

import { readFile, set_fs, utils } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getStaticProps() {
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(filename);

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

Dynamic Routes

Typically a static site with dynamic routes has an endpoint /sheets/[id] that implements both getStaticPaths and getStaticProps.

getStaticPaths should return an array of worksheet indices:

import { read } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticPaths() {
  /* read file */
  const wb = read(base64, { type: "base64" });

  /* generate an array of objects that will be used for generating pages */
  const paths = wb.SheetNames.map((name, idx) => ({ params: { id: idx.toString() } }));
  return { paths, fallback: false };
};

import { readFile, set_fs } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getStaticPaths() {
  /* read file */
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(path);

  /* generate an array of objects that will be used for generating pages */
  const paths = wb.SheetNames.map((name, idx) => ({ params: { id: idx.toString() } }));
  return { paths, fallback: false };
};

:::note

For a pure static site, fallback must be set to false!

:::

getStaticProps will generate the actual HTML for each page:

import { read, utils } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getStaticProps(ctx) {
  /* read file */
  const wb = read(base64, { type: "base64" });

  /* get the corresponding worksheet and generate HTML */
  const ws = wb.Sheets[wb.SheetNames[ctx.params.id]]; // id from getStaticPaths
  const html = utils.sheet_to_html(ws);
  return { props: { html } };
};

import { readFile, set_fs, utils } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getStaticProps(ctx) {
  /* read file */
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(path);

  /* get the corresponding worksheet and generate HTML */
  const ws = wb.Sheets[wb.SheetNames[ctx.params.id]]; // id from getStaticPaths
  const html = utils.sheet_to_html(ws);
  return { props: { html } };
};

Server-Side Rendering

:::caution Do not use on a static site

These routes require a NodeJS dynamic server. Static page generation will fail!

getStaticProps and getStaticPaths support static site generation (SSG).

getServerSideProps is suited for NodeJS hosted deployments where the workbook changes frequently and a static site is undesirable.

:::

When using getServerSideProps, the file will be read on each request.

:::caution Consider using a static strategy

When using asset modules, the file names and file paths are processed during the build step. The content is fixed. In this situation, a static approach such as getStaticProps is strongly recommended.

:::

import { read } from 'xlsx';
import base64 from '@/data/sheetjs.xlsx';

export async function getServerSideProps() {
  /* read file */
  const wb = read(base64, { type: "base64" });

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

import { readFile, set_fs, utils } from 'xlsx';
import { join } from 'path';
import { cwd } from 'process';

export async function getServerSideProps() {
  /* read file */
  set_fs(await import("fs"));
  const filename = join(cwd(), "data", "sheetjs.xlsx"); // /data/sheetjs.xlsx
  const wb = readFile(path);

  /* generate and return the html from the first worksheet */
  const html = utils.sheet_to_html(wb.Sheets[wb.SheetNames[0]]);
  return { props: { html } };
};

Data Presentation

The React demo compares common approaches.

HTML

HTML output can be generated using XLSX.utils.sheet_to_html and inserted into the document using the dangerouslySetInnerHTML attribute:

flowchart LR
  subgraph SheetJS operations
    data(File\nData)
    code{{HTML\nTABLE}}
  end
  html{{Rendered\nPage}}
  data --> |lifecycle\nsheet_to_html| code
  code --> |Index\ninnerHTML| html

export default function Index({html, type}) { return (
  <div dangerouslySetInnerHTML={{ __html: html }} />
); }

Arrays of Objects

Arrays of objects can be generated using XLSX.utils.sheet_to_json and inserted into the document using standard JSX:

flowchart LR
  subgraph SheetJS operations
    data(File\nData)
    aoo(array of\nobjects)
  end
  html{{Rendered\nPage}}
  data --> |lifecycle\nsheet_to_json| aoo
  aoo --> |Index\nReact + JSX| html

export default function Index({aoo, type}) { return (
  <table><thead><tr key={0}><th>Name</th><th>Index</th></tr></thead><tbody>
// highlight-start
    {aoo.map(row => ( <tr>
      <td>{row.Name}</td>
      <td>{row.Index}</td>
    </tr>))}
// highlight-end
  </tbody></table>
); }

Demo

:::note

This demo showcases the following SheetJS + NextJS flows:

Page	Loading Data	Lifecycle Method	SheetJS API
`/getStaticProps`	asset module	`getStaticProps`	`sheet_to_json`
`/sheets/[id]`	asset module	`getStaticPaths`	`sheet_to_html`
`/getServerSideProps`	lifecycle	`getServerSideProps`	`sheet_to_html`

:::

Initial Setup

Disable NextJS telemetry:

npx next@13.4.4 telemetry disable

Confirm it is disabled by running

npx next@13.4.4 telemetry status

Set up folder structure. At the end, a pages folder with a sheets subfolder must be created. On Linux or MacOS or WSL:

mkdir sheetjs-next
cd sheetjs-next
mkdir -p pages/sheets/

Download the test file and place in the project root. On Linux or MacOS or WSL:

curl -LO https://docs.sheetjs.com/next/sheetjs.xlsx

Install dependencies:

{\ npm i --save https://cdn.sheetjs.com/xlsx-${current}/xlsx-${current}.tgz next@13.4.4}

Download NextJS config scripts and place in the root folder:

On Linux or MacOS or WSL:

curl -LO https://docs.sheetjs.com/next/base64-loader.js
curl -LO https://docs.sheetjs.com/next/jsconfig.json
curl -LO https://docs.sheetjs.com/next/next.config.js
curl -LO https://docs.sheetjs.com/next/styles.css

Download test scripts:

Download and place the following scripts in the pages subfolder:

Download [id].js and place in the pages/sheets subfolder.

:::caution Percent-Encoding in the script name

The [id].js script must have the literal square brackets in the name. If your browser saved the file to %5Bid%5D.js. rename the file.

:::

On Linux or MacOS or WSL:

cd pages
curl -LO https://docs.sheetjs.com/next/_app.js
curl -LO https://docs.sheetjs.com/next/index.js
curl -LO https://docs.sheetjs.com/next/getServerSideProps.js
curl -LO https://docs.sheetjs.com/next/getStaticPaths.js
curl -LO https://docs.sheetjs.com/next/getStaticProps.js
cd sheets
curl -LOg 'https://docs.sheetjs.com/next/[id].js'
cd ../..

Testing

Test the deployment:

npx next@13.4.4

Open a web browser and access:

http://localhost:3000 landing page
http://localhost:3000/getStaticProps shows data from the first sheet
http://localhost:3000/getServerSideProps shows data from the first sheet
http://localhost:3000/getStaticPaths shows a list (2 sheets)

The individual worksheets are available at

http://localhost:3000/sheets/0
http://localhost:3000/sheets/1

While the development server is running, open the /getStaticProps page and open sheetjs.xlsx with a spreadsheet editor. In the editor, add a row to the bottom of the "Indices" worksheet.

After saving the file, the website should refresh with the new row.

Production Build

Stop the server and run a production build:

npx next@13.4.4 build

The final output will show a list of the routes and types:

Route (pages)                              Size     First Load JS
┌ ○ /                                      563 B          74.4 kB
├   /_app                                  0 B            73.9 kB
├ ○ /404                                   182 B          74.1 kB
├ λ /getServerSideProps                    522 B          74.4 kB
├ ● /getStaticPaths                        2.89 kB        76.8 kB
├ ● /getStaticProps                        586 B          74.5 kB
└ ● /sheets/[id]                           522 B          74.4 kB
    ├ /sheets/0
    └ /sheets/1

As explained in the summary, the /getStaticPaths and /getStaticProps routes are completely static. 2 /sheets/# pages were generated, corresponding to 2 worksheets in the file. /getServerSideProps is server-rendered.

Try to build a static site:

npx next@13.4.4 export

:::note The static export will fail!

A static page cannot be generated at this point because /getServerSideProps is still server-rendered.

:::

Static Site

Delete pages/getServerSideProps.js and rebuild:

rm -f pages/getServerSideProps.js
npx next@13.4.4 build

Inspecting the output, there should be no lines with the λ symbol:

Route (pages)                              Size     First Load JS
┌ ○ /                                      563 B          74.4 kB
├   /_app                                  0 B            73.9 kB
├ ○ /404                                   182 B          74.1 kB
├ ● /getStaticPaths                        2.89 kB        76.8 kB
├ ● /getStaticProps                        586 B          74.5 kB
└ ● /sheets/[id]                           522 B          74.4 kB
    ├ /sheets/0
    └ /sheets/1

Generate the static site:

npx next@13.4.4 export

The static site will be written to the out subfolder

Serve the static site:

npx http-server out

The command will start a local HTTP server for testing the generated site. Note that /getServerSideProps will 404 since the page was removed.

17 KiB Raw Blame History

Loading Data

Asset Module

Raw Operations

NextJS Strategies

Static Site Generation

Dynamic Routes

Server-Side Rendering

Data Presentation

HTML

Arrays of Objects

Demo

Initial Setup

Testing

Production Build

Static Site

17 KiB

Raw Blame History