How does next-term convert API routes into CLI packages?

HEAD
5744832Merge pull request #5 from vercel-labs/init-bun
This post might have stale content, as HEAD is 24 commits ahead.

How does next-term convert a Next.js API route into a publishable NPM CLI package? What does the final built package look like? Does it just make fetch calls to the API and log responses, or is there more to it? Show me the build process and what the generated CLI code looks like.

Avatar of anthropic/claude-haiku-4.5
Claude Haiku 4.5
commented

Perfect! Now I have a complete picture. Let me create a comprehensive explanation with the source code.

##Overview

Next-term is a sophisticated tool that converts a Next.js-style file-based routing structure (with handlers) into either a self-hosted CLI package (bundle mode) or an npm-publishable package (registry mode). It's not just simple fetch calls - there's a complete build system that bundles everything, manages updates, and generates distribution artifacts.

##The Build Process

Here's the high-level flow:

Your Command Files (src/app/command.ts) 
    ↓
Find command files (find-command-files.ts)
    ↓
Generate import-map (generate-request-file.ts)
    ↓
Bundle with esbuild (compiler/index.ts)
    ↓
Generate manifest.json
    ↓
[Bundle Mode] → Install Script + manifest
[Registry Mode] → package.json + cli.js + manifest

##1. Finding & Mapping Commands

The compiler first discovers all command.ts and command.tsx files:

// packages/core/src/compiler/find-command-files.ts
// Searches recursively for command.ts/command.tsx files

// src/app/command.ts          → Route: "/"
// src/app/clock/command.tsx   → Route: "/clock"
// src/app/users/[id]/command.ts → Route: "/users/[id]"

These are converted to an import map that looks like:

// .next-term/import-map.ts (auto-generated)
(globalThis as any).importPaths = {
  "/": {
    "type": "ts",
    "import": () => import("../src/app/command.ts")
  },
  "/clock": {
    "type": "tsx",
    "import": () => import("../src/app/clock/command.tsx")
  }
} as const;

##2. The Runtime (The Heart of CLI Execution)

The runtime is embedded in the final CLI and handles routing:

// packages/core/src/runtime/index.ts
async function start() {
  const manifestPath = join(__dirname, 'manifest.json');
  const manifest: Manifest = JSON.parse(readFileSync(manifestPath, 'utf-8'));

  // Patch fetch to auto-add baseURL
  globalThis.fetch = (url, options = {}) => {
    if (typeof url === 'string' && url.startsWith('/')) {
      // Convert "/get-time" → "https://your-api.com/get-time"
      url = new URL(url, backendUrl).href;
      // Add deployment ID header for versioning
      options.headers = { 'x-deployment-id': deploymentId }
    }
    return originalFetch(url, options);
  };

  // Set up CLI routing
  const program = new Command();
  registerCommands(program, manifest.cliName);
  program.parse();
}

And the command registration logic:

// packages/core/src/runtime/create-commands.ts
export function registerCommands(program: Command, cliName: string) {
  program.name(cliName);

  // Register commands from importPaths
  for (const [routePath, routeConfig] of Object.entries(importPaths)) {
    const commandPath = routePathToCommandPath(routePath);

    if (commandPath.length === 0) {
      // Default command: my-cli
      program.action(async () => {
        const module = await routeConfig.import();
        const handler = module.default;
        await handler();
      });
    } else {
      // Nested commands: my-cli clock, my-cli users <id>
      let currentCmd = program;
      for (const part of commandPath) {
        currentCmd = currentCmd.command(part);
      }
      currentCmd.action(async () => {
        const module = await routeConfig.import();
        await module.default();
      });
    }
  }
}

##3. Demo: What a Command Looks Like

Here's an actual command from the demo:

// apps/demo/src/app/clock/command.tsx
import React from 'react';
import { render, Text } from 'ink';

export default async function handler() {
  // 1. Makes fetch request - automatically prepended with baseUrl
  const res = await fetch("/get-time");
  const data = await res.json();
  const time = data?.time || '';

  // 2. Renders UI using React + Ink (terminal rendering)
  const Clock = () => {
    return <Text color="blue">The time is {time}</Text>;
  };

  render(<Clock />);
}

When you run my-cli clock, the runtime:

  1. Dynamically imports the tsx file
  2. Calls the handler function
  3. The handler can fetch from your backend (fetch urls starting with / are auto-prefixed)
  4. Can render React components with Ink for beautiful terminal UI

##4. The Final Built Package Structure

###Bundle Mode Output (public/cli/)

public/
└── cli/
    ├── cli.js                 # Main executable (with #!/usr/bin/env node)
    ├── manifest.json          # Deployment metadata
    ├── install                # Bash install script
    └── [chunk files]          # Bundled dependencies

###Registry Mode Output (.next-term/dist/)

.next-term/
└── dist/
    ├── cli.js                 # NPM binary (chmod +x, with shebang)
    ├── package.json           # npm metadata (auto-generated with version)
    ├── manifest.json          # Deployment metadata
    ├── README.md              # Documentation
    └── [chunk files]          # Bundled dependencies

##5. The Generated package.json (Registry Mode)

// .next-term/dist/package.json (auto-generated)
{
  "name": "my-cli",
  "version": "2026.1.24-a1b2c3d4e5f6",  // YYYY.M.D-{hash}
  "description": "CLI built with Next Term",
  "main": "./cli.js",
  "type": "module",
  "bin": {
    "my-cli": "./cli.js"
  },
  "files": ["*.js", "manifest.json", "README.md"],
  "engines": { "node": ">=18.0.0" },
  "keywords": ["cli", "next-term"],
  "license": "MIT",
  "author": "...",
  "repository": { ... }
}

The version is generated deterministically from:

  • All files in the dist directory
  • The config file content
  • A hash of the CLI binary
  • The current date

This ensures unique versions even for multiple builds on the same day.

##6. The Build System (build.js)

// packages/core/build.js
const baseConfig = {
  bundle: true,
  sourcemap: true,
  platform: "node",
};

// 1. Build runtime first (esm format)
esbuild.build({
  entryPoints: ["src/runtime/index.ts"],
  outfile: "dist/runtime/index.js",
  format: "esm",
  target: "node18",
  plugins: [nodeExternalsPlugin()],
});

// 2. Build library exports (cjs + esm)
esbuild.build({
  entryPoints: ["src/index.ts"],
  outfile: "dist/index.js",
  format: "cjs",
});

esbuild.build({
  entryPoints: ["src/index.ts"],
  outfile: "dist/index.mjs",
  format: "esm",
});

// 3. Build CLI (cjs, with shebang)
esbuild.build({
  entryPoints: ["src/cli.ts"],
  outfile: "dist/cli.js",
  format: "cjs",
  banner: { js: "#!/usr/bin/env node" },
  define: { runtimeSource: JSON.stringify(runtimeSource) },
});

// 4. Generate TypeScript declarations
execSync("tsc --emitDeclarationOnly --declaration");

##7. The Install Script (Bundle Mode)

For self-hosted distribution, a bash script is generated:

#!/bin/bash
# Generated by next-term

CLI_NAME="my-cli"
BASE_URL="https://my-api.com"
CLI_PATH="cli"
INSTALL_DIR="$HOME/.my-cli"
BIN_DIR="$INSTALL_DIR/bin"

mkdir -p "$BIN_DIR"

# Download all files from manifest
curl -fsSL "$BASE_URL/$CLI_PATH/cli.js" -o "$INSTALL_DIR/cli.js"
curl -fsSL "$BASE_URL/$CLI_PATH/manifest.json" -o "$INSTALL_DIR/manifest.json"

# Create wrapper binary
cat > "$BIN_DIR/my-cli" << 'EOF'
#!/usr/bin/env node
require('../cli.js');
EOF

chmod +x "$BIN_DIR/my-cli"

# Add to PATH
echo "export PATH=\"$PATH:$BIN_DIR\"" >> ~/.zshrc

Users then install with: curl https://your-domain.com/cli/install | sh

##8. Auto-Updates (Bundle Mode Only)

The runtime has built-in update checking:

// packages/core/src/runtime/update.ts
checkAndApplyUpdates(
  __dirname,
  backendUrl,
  distFolder,
  deploymentId
);

This:

  1. Fetches remote manifest.json from your backend
  2. Compares file hashes with local files
  3. Downloads any changed files
  4. Applies updates atomically

This is only for bundle mode - registry mode relies on npm for updates.

##9. What Makes This Not Just Fetch Calls

The system does much more:

Dynamic imports - Commands are lazy-loaded only when needed
Nested routing - Support for subcommands with parameters
React/Ink UI - Full terminal UI framework support
Dependency bundling - All npm packages bundled into single executable
Auto-updates - Self-hosted CLIs can auto-update without npm
Manifest-based versioning - Tracks deployed versions with hashes
Two distribution modes - Self-hosted or npm registry
TypeScript support - Full TS/TSX with type safety
Smart fetch patching - Relative URLs auto-mapped to backend
Deterministic builds - Same source = same version hash

##Summary

The build process takes your command files, discovers them via file-system conventions, generates a routing manifest, bundles everything with esbuild (including dependencies), and produces either:

  • Bundle mode: Self-hosted CLI with install script and auto-updates
  • Registry mode: npm package with auto-generated versioning and documentation

The final CLI is a complete, standalone executable that can fetch from your API backend, render React UIs with Ink, and auto-update itself without requiring npm.


END OF POST

How does next-term convert API routes into CLI packages? — vercel-labs/next-term