npm optionalDependencies Pattern

npm optionalDependencies Pattern

How to Use This Skill

1. Review the patterns and examples below
2. Apply the relevant patterns to your implementation
3. Follow the best practices outlined in this skill

Expert skill for distributing native binaries via npm using the optionalDependencies pattern with automatic platform resolution and fallback mechanisms.

When to Use

Use this skill when:

• Distributing native CLI tools via npm (Rust, Go, C++ binaries)
• Need cross-platform support (macOS, Linux, Windows)
• Want single npm install -g command for all platforms
• Need fallback when optionalDependencies are disabled
• Building packages for npm, yarn, pnpm, and bun compatibility

Don't use this skill when:

• Pure JavaScript/TypeScript packages (no native binaries)
• Single-platform distribution only
• Using native installers exclusively (no npm)

Core Pattern Overview

@scope/package-name (wrapper)
├── bin/cli.js           → Entry point, locates + spawns binary
├── postinstall.js       → Fallback download if optionalDeps unavailable
├── package.json         → optionalDependencies for 6 platforms
│
└── optionalDependencies:
    ├── @scope/package-darwin-x64     → macOS Intel
    ├── @scope/package-darwin-arm64   → macOS Apple Silicon
    ├── @scope/package-linux-x64      → Linux x64 (glibc)
    ├── @scope/package-linux-arm64    → Linux ARM64
    ├── @scope/package-linux-x64-musl → Alpine Linux (musl)
    └── @scope/package-win32-x64      → Windows x64

Package.json Template (Wrapper Package)

{
  "name": "@scope/package-name",
  "version": "1.0.0",
  "description": "Cross-platform CLI tool description",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/org/repo"
  },
  "bin": {
    "command-name": "bin/cli.js"
  },
  "scripts": {
    "postinstall": "node postinstall.js"
  },
  "optionalDependencies": {
    "@scope/package-darwin-x64": "1.0.0",
    "@scope/package-darwin-arm64": "1.0.0",
    "@scope/package-linux-x64": "1.0.0",
    "@scope/package-linux-arm64": "1.0.0",
    "@scope/package-linux-x64-musl": "1.0.0",
    "@scope/package-win32-x64": "1.0.0"
  },
  "engines": {
    "node": ">=18"
  },
  "os": ["darwin", "linux", "win32"],
  "cpu": ["x64", "arm64"]
}

Platform Detection Algorithm

// lib/platform.js
const os = require('os');
const fs = require('fs');

/**
 * Detect current platform for binary selection
 * @returns {string} Platform key (e.g., 'darwin-arm64', 'linux-x64-musl')
 */
function detectPlatform() {
  const platform = os.platform();  // 'darwin', 'linux', 'win32'
  const arch = os.arch();          // 'x64', 'arm64'

  // Handle musl libc (Alpine Linux)
  if (platform === 'linux' && arch === 'x64') {
    if (isMusl()) {
      return 'linux-x64-musl';
    }
  }

  return `${platform}-${arch}`;
}

/**
 * Detect musl libc vs glibc
 * @returns {boolean} True if running on musl (Alpine Linux)
 */
function isMusl() {
  // Method 1: Check /etc/os-release for Alpine
  try {
    const osRelease = fs.readFileSync('/etc/os-release', 'utf8');
    if (osRelease.includes('Alpine')) {
      return true;
    }
  } catch {}

  // Method 2: Check ldd version output
  try {
    const { execSync } = require('child_process');
    const lddOutput = execSync('ldd --version 2>&1', { encoding: 'utf8' });
    return lddOutput.toLowerCase().includes('musl');
  } catch {}

  // Method 3: Check for musl dynamic linker
  try {
    return fs.existsSync('/lib/ld-musl-x86_64.so.1');
  } catch {}

  return false;
}

module.exports = { detectPlatform, isMusl };

Binary Location Algorithm

// lib/binary.js
const path = require('path');
const fs = require('fs');
const { detectPlatform } = require('./platform');

const BINARY_NAME = 'coditect';  // Customize per project

/**
 * Locate the native binary for current platform
 * Searches 5 locations in priority order
 * @returns {string|null} Path to binary or null if not found
 */
function getBinaryPath() {
  const platform = detectPlatform();
  const platformPkg = `@scope/package-${platform}`;
  const binaryName = process.platform === 'win32'
    ? `${BINARY_NAME}.exe`
    : BINARY_NAME;

  // Search paths in priority order
  const searchPaths = [
    // 1. Local development (bin/ in current package)
    path.join(__dirname, '..', 'bin', binaryName),

    // 2. Hoisted node_modules (npm default)
    path.join(__dirname, '..', '..', platformPkg, 'bin', binaryName),

    // 3. Nested node_modules
    path.join(__dirname, '..', 'node_modules', platformPkg, 'bin', binaryName),

    // 4. pnpm structure
    path.join(__dirname, '..', '..', '.pnpm', `${platformPkg}@*`, 'node_modules', platformPkg, 'bin', binaryName),

    // 5. Environment override
    process.env.CODITECT_BIN,
  ].filter(Boolean);

  for (const searchPath of searchPaths) {
    // Handle pnpm glob pattern
    if (searchPath.includes('*')) {
      const globResult = findPnpmBinary(searchPath, binaryName);
      if (globResult) return globResult;
      continue;
    }

    if (fs.existsSync(searchPath)) {
      return searchPath;
    }
  }

  return null;
}

function findPnpmBinary(pattern, binaryName) {
  const pnpmDir = pattern.split('*')[0];
  if (!fs.existsSync(pnpmDir)) return null;

  const entries = fs.readdirSync(path.dirname(pnpmDir));
  for (const entry of entries) {
    const binPath = path.join(path.dirname(pnpmDir), entry, 'node_modules',
      pattern.split('/node_modules/')[1].split('@*')[0], 'bin', binaryName);
    if (fs.existsSync(binPath)) return binPath;
  }
  return null;
}

module.exports = { getBinaryPath, BINARY_NAME };

CLI Entry Point (bin/cli.js)

#!/usr/bin/env node
// bin/cli.js - CLI entry point that spawns the native binary

const { spawn } = require('child_process');
const { getBinaryPath } = require('../lib/binary');

function main() {
  const binaryPath = getBinaryPath();

  if (!binaryPath) {
    console.error('Error: Could not find native binary for your platform.');
    console.error('');
    console.error('Try reinstalling:');
    console.error('  npm uninstall -g @scope/package-name');
    console.error('  npm install -g @scope/package-name');
    console.error('');
    console.error('If the problem persists, please report an issue:');
    console.error('  https://github.com/org/repo/issues');
    process.exit(1);
  }

  // Spawn binary with all arguments, inheriting stdio
  const child = spawn(binaryPath, process.argv.slice(2), {
    stdio: 'inherit',
    windowsHide: true,
  });

  child.on('error', (err) => {
    console.error(`Failed to start binary: ${err.message}`);
    process.exit(1);
  });

  child.on('exit', (code, signal) => {
    if (signal) {
      process.kill(process.pid, signal);
    } else {
      process.exit(code ?? 1);
    }
  });
}

main();

Postinstall Fallback (postinstall.js)

#!/usr/bin/env node
// postinstall.js - Download binary if optionalDependencies unavailable

const https = require('https');
const fs = require('fs');
const path = require('path');
const crypto = require('crypto');
const { detectPlatform } = require('./lib/platform');
const { getBinaryPath, BINARY_NAME } = require('./lib/binary');

const CDN_BASE = 'https://dist.example.com';
const MANIFEST_URL = `${CDN_BASE}/manifest.json`;

async function main() {
  // Check if binary already exists (optionalDeps worked)
  if (getBinaryPath()) {
    console.log('Binary found via optionalDependencies');
    return;
  }

  console.log('Binary not found, downloading from CDN...');

  try {
    // Fetch manifest
    const manifest = await fetchJSON(MANIFEST_URL);
    const platform = detectPlatform();
    const release = manifest.platforms[platform];

    if (!release) {
      throw new Error(`Unsupported platform: ${platform}`);
    }

    // Download binary
    const binDir = path.join(__dirname, 'bin');
    fs.mkdirSync(binDir, { recursive: true });

    const binaryName = process.platform === 'win32'
      ? `${BINARY_NAME}.exe`
      : BINARY_NAME;
    const binaryPath = path.join(binDir, binaryName);

    await downloadFile(release.url, binaryPath);

    // Verify checksum
    const actualHash = await hashFile(binaryPath);
    if (actualHash !== release.sha256) {
      fs.unlinkSync(binaryPath);
      throw new Error('Checksum verification failed');
    }

    // Make executable (Unix)
    if (process.platform !== 'win32') {
      fs.chmodSync(binaryPath, 0o755);
    }

    console.log('Binary installed successfully');
  } catch (err) {
    console.error(`Warning: Failed to download binary: ${err.message}`);
    console.error('You may need to install manually.');
    // Don't fail install - user can retry
  }
}

function fetchJSON(url) {
  return new Promise((resolve, reject) => {
    https.get(url, (res) => {
      let data = '';
      res.on('data', chunk => data += chunk);
      res.on('end', () => {
        try { resolve(JSON.parse(data)); }
        catch (e) { reject(e); }
      });
    }).on('error', reject);
  });
}

function downloadFile(url, dest) {
  return new Promise((resolve, reject) => {
    const file = fs.createWriteStream(dest);
    https.get(url, (res) => {
      res.pipe(file);
      file.on('finish', () => { file.close(); resolve(); });
    }).on('error', (err) => {
      fs.unlink(dest, () => {});
      reject(err);
    });
  });
}

function hashFile(filePath) {
  return new Promise((resolve, reject) => {
    const hash = crypto.createHash('sha256');
    const stream = fs.createReadStream(filePath);
    stream.on('data', data => hash.update(data));
    stream.on('end', () => resolve(hash.digest('hex')));
    stream.on('error', reject);
  });
}

main();

Platform Package Template

{
  "name": "@scope/package-darwin-arm64",
  "version": "1.0.0",
  "description": "macOS ARM64 binary for package-name",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/org/repo"
  },
  "os": ["darwin"],
  "cpu": ["arm64"],
  "files": ["bin/"]
}

Package Manager Compatibility

Manager	optionalDeps	Notes
npm	✅ Default	Standard resolution
yarn v1	✅ Default	Classic mode
yarn v2+	✅ With config	May need nodeLinker: node-modules
pnpm	✅ Default	Flat/non-flat supported
bun	✅ Default	npm compatibility mode

Best Practices

1. Keep wrapper package small (<100KB) - binaries in platform packages
2. Version lock optionalDeps - exact versions, not ranges
3. Implement postinstall fallback - for restricted environments
4. Test all 6 platforms - CI matrix with actual binaries
5. Use semantic versioning - all packages share version number
6. Include os/cpu fields - for early detection of unsupported platforms

Related Resources

• TEMPLATES.md - Copy-paste ready templates
• REFERENCE.md - Detailed API reference
• npm-packaging-specialist agent - Full package creation automation

---

Success Output

When successful, this skill MUST output:

✅ SKILL COMPLETE: npm-optionaldeps-pattern

Completed:
- [x] Wrapper package created with optionalDependencies
- [x] 6 platform packages configured (darwin-x64, darwin-arm64, linux-x64, linux-arm64, linux-x64-musl, win32-x64)
- [x] Platform detection implemented (musl detection working)
- [x] Binary location algorithm with 5 search paths
- [x] CLI entry point spawning native binary
- [x] Postinstall fallback with CDN download
- [x] Checksum verification implemented

Outputs:
- @scope/package-name/ (wrapper package)
  - package.json with optionalDependencies
  - bin/cli.js (entry point)
  - lib/platform.js (detection)
  - lib/binary.js (location)
  - postinstall.js (fallback)
- @scope/package-{platform}/ (6 platform packages)
  - package.json with os/cpu restrictions
  - bin/{binary} (native executable)

Tested on:
- npm, yarn, pnpm, bun (all working)
- macOS Intel, macOS ARM, Linux x64, Alpine Linux (verified)

Completion Checklist

Before marking this skill as complete, verify:

• Wrapper package.json has all 6 optionalDependencies
• Platform detection correctly identifies musl (Alpine Linux)
• Binary location algorithm searches 5 paths (local, hoisted, nested, pnpm, env)
• CLI entry point spawns binary with proper stdio inheritance
• Postinstall fallback downloads from CDN if optionalDeps unavailable
• SHA-256 checksum verification prevents corrupted downloads
• Platform packages have correct os/cpu restrictions
• Binary made executable on Unix (chmod 0o755)
• All package managers tested (npm, yarn, pnpm, bun)
• Version numbers synchronized across all packages

Failure Indicators

This skill has FAILED if:

• ❌ Platform detection misidentifies musl as glibc (Alpine fails)
• ❌ Binary not found on any package manager (search paths incomplete)
• ❌ CLI entry point doesn't inherit stdio (no output visible)
• ❌ Postinstall fallback doesn't verify checksums (security risk)
• ❌ Platform packages install on wrong OS/CPU
• ❌ Binary not executable on Unix (permission errors)
• ❌ pnpm installation fails (pnpm glob pattern broken)
• ❌ Version mismatch between wrapper and platform packages

When NOT to Use

Do NOT use this skill when:

• Pure JavaScript/TypeScript packages (no native binaries to distribute)
• Single-platform distribution only (simpler to use native installers)
• Using container-only distribution (Docker handles platform detection)
• Electron apps (use electron-builder with native modules)
• Browser-only packages (no native binaries possible)
• Development-only tools (not for end-user installation)
• Binaries >50 MB per platform (npm registry size limits)

Use alternative approaches:

• electron-builder - For Electron apps with native modules
• pkg - For single-executable Node.js apps
• Docker images - For containerized distribution
• Native installers - For OS-specific installation (DMG, MSI, DEB)

Anti-Patterns (Avoid)

Anti-Pattern	Problem	Solution
Missing musl detection	Alpine Linux users get glibc binary (fails)	Implement all 3 musl detection methods
Hardcoded binary path	Breaks on different package managers	Use 5-path search algorithm
No postinstall fallback	Fails in restricted npm environments	Always include CDN fallback
No checksum verification	Security risk from MITM attacks	Verify SHA-256 before execution
Missing os/cpu restrictions	Wrong binary installs (silent failures)	Always specify os/cpu in platform packages
Version range optionalDeps	Version mismatch between wrapper and platforms	Use exact versions (no ^ or ~)
Not inheriting stdio	No output visible to user	spawn() with stdio: 'inherit'
Forgetting chmod +x	Unix binaries not executable	fs.chmodSync(binaryPath, 0o755)

Principles

This skill embodies:

• #5 Eliminate Ambiguity - Clear platform detection, explicit error messages
• #8 No Assumptions - Detect platform, don't assume glibc
• #11 Reliability - Multiple fallbacks (optionalDeps → CDN → manual)
• #12 Observability - Clear errors when binary not found
• Security - Checksum verification, no arbitrary code execution
• Compatibility - Support all major package managers

Full Standard: CODITECT-STANDARD-AUTOMATION.md

---

Version: 1.1.0 | Updated: 2026-01-04 | Author: CODITECT Team