Set up the backend

In this guide, you will learn how to set up the backend infrastructure for a collaborative application using Weave.js.

To make the backend functional, you will set up a Next.js custom server to later integrate Weave.js Websockets store.

The store manages shared state, client networking, persistence, and awareness events.

Prerequisites

Before you begin, ensure that you have completed the manual installation guide up to the "Set up the backend project" step.

Step by step

To set up Weave.js backend over our Next.js project (on a single artifact), follow these steps:

Install Weave.js SDK

From the project root folder, install the Weave.js SDK by executing the following command:

npm install @inditextech/weave-sdk

Install Weave.js Store

From the project root folder, install the Weave.js Store by executing the following command:

npm install @inditextech/weave-store-websockets ws

This example uses the WebSockets store.

Set up the Next.js custom server

To have a frontend and backend on the same artifact on top of Next.js you will implement a custom server on Next.js and the necessary Weave.js backend logic.

Not recommended for production

For production environments, you should consider using a separate backend server to handle the Weave.js backend logic.

For more details, refer to the backend showcase implementation.

Dependencies installation

Install the necessary dependencies for the custom server with the following commands:

Dependencies needed on runtime:

npm install cross-env express

Dependencies only needed on the local development environment or build time:

npm install -D tsx nodemon @types/express

TypeScript configuration

Nodemon

This section uses nodemon, a tool that helps to automatically start the underlying Node.js application when a file changes.

Set up the transpilation of the server files from TypeScript

Create a nodemon.json file on the project root with:

nodemon.json

{
  "watch": ["**/*.{ts,tsx}", "next.config.mjs", "persistence.ts", "server.ts"],
  "ignore": ["node_modules/**/*"],
  "exec": "tsx server.ts",
  "ext": "js ts"
}

Next.js configuration

Set up the Next.js configuration:

Create a next.config.mjs file in the project root with:

next.config.mjs

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: false,
  experimental: { serverComponentsExternalPackages: ["yjs"] },
};

export default nextConfig;

Custom server logic

Express

This section uses Express, a web framework for Node.js, as the underlying server for the custom server.

Define the custom server logic by creating a file named server.ts in the project root with:

server.ts

import express, { Request, Response } from "express";
import next from "next";

const port = parseInt(process.env.PORT || "3000", 10);
const dev = process.env.NODE_ENV !== "production";
const app = next({ dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  const app = express();

  app.get("/{*splat}", (req: Request, res: Response) => {
    return handle(req, res);
  });

  const server = app.listen(port, (err: Error | undefined) => {
    if (err) throw err;
    console.log(
      `> Server listening at http://localhost:${port} as ${
        dev ? "development" : process.env.NODE_ENV
      }`
    );
  });
});

Custom server tooling

To properly use the custom server both locally and during the build process, update the dependent scripts in your package.json, specifically dev, build, and start.

Change the scripts definitions with:

{
  ...
  "scripts": {
    ...
    "dev": "next dev --turbo", 
    "dev": "nodemon", 
    "build": "next build", 
    "build": "next build && tsc --project tsconfig.server.json", 
    "start": "next start", 
    "start": "cross-env NODE_ENV=production node dist/server.js", 
    ...
  },
  ...
}

Finally, test that the Next.js project keeps starting with the new custom server configuration.

From the project root folder, run the following command:

npm run dev

Navigate to http://localhost:3000 in a browser. You should still see the Next.js default application running and the console should have no errors at all.

Integrate Weave.js into the backend

Now that the Next.js custom server is completely set up, add the necessary Weave.js backend logic. Use the Websockets store library (@inditextech/weave-store-websockets).

You need to:

Define how to handle the persistence of the shared-state.
Instantiate the WeaveWebsocketsServer class from the library @inditextech/weave-store-websockets and configure it.

Set up the shared-state persistence handlers

Define the persistence logic for the shared-state. For this example, use the file system as our persistence layer.

Create a folder named weave on the project root. Inside create a persistence.ts file with:

persistence.ts

import path from "path"; // (1)
import fs from "fs/promises"; // (1)
import { dirname } from "path";
import { fileURLToPath } from "url";

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

export const fetchRoom = async (docName: string) => {
  try {
    const roomsFolder = path.join(__dirname, "rooms");
    const roomsFile = path.join(roomsFolder, docName);

    const roomsFileCheck = path.resolve(roomsFolder, docName);
    if (!roomsFileCheck.startsWith(path.resolve(roomsFolder))) {
      throw new Error("Path escape detected");
    }

    return await fs.readFile(roomsFile);
  } catch (e) {
    console.error(e);
    return null;
  }
}; // (2)

export const persistRoom = async (
  docName: string,
  actualState: Uint8Array<ArrayBufferLike>
) => {
  try {
    const roomsFolder = path.join(__dirname, "rooms");

    let folderExists = false;
    try {
      await fs.access(roomsFolder);
      folderExists = true;
      // eslint-disable-next-line @typescript-eslint/no-unused-vars
    } catch {
      folderExists = false;
    }

    if (!folderExists) {
      await fs.mkdir(roomsFolder, { recursive: true });
    }

    const roomsFile = path.join(roomsFolder, docName);
    await fs.writeFile(roomsFile, actualState);
  } catch (ex) {
    console.error(ex);
  }
}; // (3)

Changes explanation:

(1): Import necessary Node.js dependencies to handle the file system.
(2): fetchRoom function fetches a previously saved shared-state of a room by its ID from the file system.
(3): persistRoom function persists the shared-state of a room by its ID onto the file system.

Set up the Weave.js Websockets store

Finally, set up the Weave.js store handler using the WebSockets transport.

In server.ts, replace the contents with the following to integrate the @inditextech/weave-store-websockets store handler:

server.ts

import { IncomingMessage } from "http"; 
import express, { Request, Response } from "express";
import next from "next";
import { WeaveWebsocketsServer } from "@inditextech/weave-store-websockets/server"; // (1)
import { fetchRoom, persistRoom } from "./weave/persistence"; // (2)

const VALID_ROOM_WEBSOCKET_URL = /\/rooms\/(.*)\/connect/; // (3)

const port = parseInt(process.env.PORT || "3000", 10);
const dev = process.env.NODE_ENV !== "production";
const app = next({ dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  const app = express();

  app.get("/{*splat}", (req: Request, res: Response) => {
    return handle(req, res);
  });

  app.get("/api/rooms/:roomId", async (req: Request, res: Response) => { // (4)
    const buffer = await fetchRoom(req.params.roomId); 
    if (!buffer) { 
      return res.status(404).send("Room not found"); 
    } 
    res.setHeader("Content-Type", "application/octet-stream"); 
    res.setHeader( 
      "Content-Disposition", 
      `attachment; filename="${req.params.roomId}"`
    ); 
    res.setHeader("Content-Type", "image/png"); 
    res.send(buffer); 
  }); 

  app.listen(port, (err: Error | undefined) => { 
  const server = app.listen(port, (err: Error | undefined) => { 
    if (err) throw err;
    console.log(
      `> Server listening at http://localhost:${port} as ${
        dev ? "development" : process.env.NODE_ENV
      }`
    );
  }); // (5)

  const weaveWebsocketsServerConfig = { 
    performUpgrade: async (request: IncomingMessage) => { 
      return VALID_ROOM_WEBSOCKET_URL.test(request.url ?? ""); 
    }, // (7)
    extractRoomId: (request: IncomingMessage) => { 
      const match = request.url?.match(VALID_ROOM_WEBSOCKET_URL); 
      if (match) { 
        return match[1]; 
      } 
      return undefined; 
    }, // (8)
    fetchRoom, // (9)
    persistRoom, // (10)
  }; // (6)

  const wss = new WeaveWebsocketsServer(weaveWebsocketsServerConfig); // (11)

  wss.handleUpgrade(server); // (12)
});

Changes explanation:

(1): Import Weave.js Websockets store dependency.
(2): Import persistence handlers previously defined.
(3): Define a regex to identify the Weave.js Websockets connection URI.
(4): Define an endpoint to fetch the initial content of a room.
(5): Extract the underlying HTTP server.
(6): Define the configuration for the WeaveWebsocketsServer class.
(7): Define the property performUpgrade, this indicates which URIs are valid for the Websocket upgrade protocol (upgrade from HTTP to WEBSOCKETS).
(8): Define the property extractRoomId, this indicates how to extract the room ID from the Websocket connection URI.
(9): Use the previous fetchRoom function for persistence management.
(10): Use the previous persistRoom function for persistence management.
(11): Instantiate the WeaveWebsocketsServer class.
(12): Attach the Weave.js Websockets server to the HTTP server.

Verify the backend setup

Finally, test that the project keeps starting with the changes made to the custom server.

From the project root folder, run the following command:

npm run dev

Navigate to http://localhost:3000 in a browser.

You should still see the Next.js default application running and the console should have no errors at all.

Next steps

Set up the frontend of the application.

Set up the backend

On this page