Set up the backend

In this guide, you will learn how to set up the backend infrastructure for a collaborative application using Weave.js.

To make the backend functional, we need to set up a Next.js custom server to later integrate Weave.js Websockets store.

The store will handle the shared-state management, client networking, persistence, and awareness events.

Prerequisites

Before you begin, ensure that you have completed the manual installation guide up to the "Set up the backend project" step.

Step by step

To set up Weave.js backend over our Next.js project (on a single artifact), follow these steps:

Install the Weave.js backend dependencies

First, let's install the Weave.js websocket store dependencies.

On your root project folder, execute the following command:

npm install ws @inditextech/weave-store-websockets

Set up the Next.js custom server

To have a frontend and backend on the same artifact on top of Next.js we need to implement a custom server on Next.js, and later on, include the necessary Weave.js backend logic.

Not recommended for production

For production environments, you should consider using a separate backend server to handle the Weave.js backend logic.

For more details, refer to the backend showcase implementation.

Dependencies installation

To start, install the necessary dependencies for the custom server with the following commands:

Dependencies needed on runtime:

npm install cross-env express

Dependencies only needed on the local development environment or build time:

npm install -D ts-node nodemon @types/express

TypeScript configuration

Nodemon

This section uses nodemon, a tool that helps to automatically start the underlying Node.js application when a file changes.

Then, set up the TypeScript configuration for the server part:

Create a tsconfig.server.json file on the root of your project with the following content:

tsconfig.server.json

{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "module": "commonjs",
    "outDir": "dist",
    "lib": ["es2019"],
    "target": "es2019",
    "isolatedModules": false,
    "noEmit": false
  },
  "include": ["server.ts"]
}

Set up the transpilation of the server files from TypeScript to JavaScript. Create a nodemon.json file on the root of your project with the following content:

nodemon.json

{
  "watch": ["**/*.{ts,tsx}", "next.config.mjs", "persistence.ts", "server.ts"],
  "ignore": ["node_modules/**/*"],
  "exec": "ts-node --project tsconfig.server.json server.ts",
  "ext": "js ts"
}

Next.js configuration

Then, set up the Next.js configuration:

Create a next.config.mjs file on the root of your project with the following content:

next.config.mjs

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: false,
  experimental: { serverComponentsExternalPackages: ["yjs"] },
};

export default nextConfig;

Custom server logic

Express

This section uses Express, a web framework for Node.js, as the underlying server for the custom server.

Define the custom server logic by creating a file named server.ts on the root of your project with the following content:

server.ts

import express, { Request, Response } from "express";
import next from "next";

const port = parseInt(process.env.PORT || "3000", 10);
const dev = process.env.NODE_ENV !== "production";
const app = next({ dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  const app = express();

  app.get("/{*splat}", (req: Request, res: Response) => {
    return handle(req, res);
  });

  app.listen(3000, (err: Error | undefined) => {
    if (err) throw err;
    console.log(
      `> Server listening at http://localhost:${port} as ${
        dev ? "development" : process.env.NODE_ENV
      }`
    );
  });
});

Custom server tooling

To properly use the custom server both locally and during the build process, we need to customize the dependent scripts in your package.json: specifically the dev, build, and start scripts.

Change the scripts definitions with the following content:

{
  ...
  "scripts": {
    ...
    "dev": "next dev --turbo", 
    "dev": "nodemon", 
    "build": "next build", 
    "build": "next build && tsc --project tsconfig.server.json", 
    "start": "next start", 
    "start": "cross-env NODE_ENV=production node dist/server.js"
    ...
  },
  ...
}

Finally, test that the Next.js project keeps starting with the new custom server configuration.

On the project root folder, run the following command:

npm run dev

Navigate to http://localhost:3000 on a browser. You should still see the Next.js default application running and the console should have no errors at all.

Integrate Weave.js into the backend

Now that the Next.js custom server is completely set up, let's add the necessary Weave.js backend logic. For that, we will use the Websockets store library: (@inditextech/weave-store-websockets).

We need to:

Define how to handle the persistence of the shared-state.
Instantiate the WeaveWebsocketsServer class from the library @inditextech/weave-store-websockets and configure it.

Set up the shared-state persistence handlers

In this step, we will define the persistence logic for the shared-state. In this case, we use the file system as our persistence layer.

Create a persistence.ts file on the project root and set its content to:

persistence.ts

import path from "path"; // (1)
import fs from "fs/promises"; // (1)

export const fetchRoom = async (docName: string) => {
  try {
    const roomsFolder = path.join(__dirname, "rooms");
    const roomsFile = path.join(roomsFolder, docName);
    return await fs.readFile(roomsFile);
  } catch (e) {
    return null;
  }
}; // (2)

export const persistRoom = async (
  docName: string,
  actualState: Uint8Array<ArrayBufferLike>
) => {
  try {
    const roomsFolder = path.join(__dirname, "rooms");

    let folderExists = false;
    try {
      await fs.access(roomsFolder);
      folderExists = true;
      // eslint-disable-next-line @typescript-eslint/no-unused-vars
    } catch (e) {
      folderExists = false;
    }

    if (!folderExists) {
      await fs.mkdir(roomsFolder, { recursive: true });
    }

    const roomsFile = path.join(roomsFolder, docName);
    await fs.writeFile(roomsFile, actualState);
  } catch (ex) {
    console.error(ex);
  }
}; // (3)

File explanation:

(1): Import necessary Node.js dependencies to handle the file system.
(2): fetchRoom function fetches a previously saved shared-state of a room by its ID from the file system.
(3): persistRoom function persist the shared-state of a room by its ID onto the file system.

Set up the Weave.js Websockets store

Finally, set up the Weave.js store handler. In this case, we use the Websockets transport.

On the custom server file server.ts, add the necessary logic to use the @inditextech/weave-store-websockets store handler.

You can replace the contents of server.ts with this content:

server.ts

import { IncomingMessage } from "http"; 
import express, { Request, Response } from "express";
import next from "next";
import { WeaveWebsocketsServer } from "@inditextech/weave-store-websockets/server"; // (1)
import { fetchRoom, persistRoom } from "./persistence"; // (2)

const VALID_ROOM_WEBSOCKET_URL = /\/sync\/rooms\/(.*)/; // (3)

const port = parseInt(process.env.PORT || "3000", 10);
const dev = process.env.NODE_ENV !== "production";
const app = next({ dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  const app = express();

  app.get("/{*splat}", (req: Request, res: Response) => {
    return handle(req, res);
  });

  app.listen(3000, (err: Error | undefined) => { 
  const server = app.listen(3000, (err: Error | undefined) => { 
    if (err) throw err;
    console.log(
      `> Server listening at http://localhost:${port} as ${
        dev ? "development" : process.env.NODE_ENV
      }`
    );
  }); // (4)

  const weaveWebsocketsServerConfig = { 
    performUpgrade: async (request: IncomingMessage) => { 
      return VALID_ROOM_WEBSOCKET_URL.test(request.url ?? ""); 
    }, // (6)
    extractRoomId: (request: IncomingMessage) => { 
      const match = request.url?.match(VALID_ROOM_WEBSOCKET_URL); 
      if (match) { 
        return match[1]; 
      } 
      return undefined; 
    }, // (7)
    fetchRoom, // (8)
    persistRoom, // (9)
  }; // (5)

  const wss = new WeaveWebsocketsServer(weaveWebsocketsServerConfig); // (10)

  wss.handleUpgrade(server); // (11)
});

Changes explanation:

(1): Import Weave.js Websockets store dependency.
(2): Import persistence handlers previously defined.
(3): Define a regex to identify the Weave.js Websockets connection URI.
(4): Extract the underlying HTTP server.
(5): Define the configuration for the WeaveWebsocketsServer class.
(6): Define the property performUpgrade, this indicates which URIs are valid for the Websocket upgrade protocol (upgrade from HTTP to WEBSOCKETS)
(7): Define the property extractRoomId, this indicates how to extract the room ID from the Websocket connection URI.
(8): Use the previous fetchRoom function for persistence management.
(9): Use the previous persistRoom function for persistence management.
(10): Instantiate the WeaveWebsocketsServer class.
(11): Attach the Weave.js Websockets server to the HTTP server.

Run the project

Finally, test that the project keeps starting with the changes made to the custom server.

On the project root folder, run the following command:

npm run dev

Navigate to http://localhost:3000 on a browser.

You should still see the Next.js default application running and the console should have no errors at all.

Next steps

Let's now set up the frontend of the application.

Set up the backend

On this page