This directory contains utility scripts for managing and validating the AI Coding Stack project. Scripts are organized into four categories:

• generate/ - Generation scripts that create derived files
• refactor/ - Refactoring scripts that reorganize or reformat data
• fetch/ - Data fetching scripts that retrieve external data
• validate/ - URL validation scripts that check website accessibility

scripts/
├── generate/
│   ├── index.mjs              # Entry point for all generation scripts
│   ├── generate-manifest-indexes.mjs
│   └── generate-metadata.mjs
├── refactor/
│   ├── index.mjs              # Entry point for all refactoring scripts
│   ├── sort-manifest-fields.mjs
│   ├── sort-locales-fields.mjs
│   └── export-vendors.mjs
├── fetch/
│   ├── index.mjs              # Entry point for all fetch scripts
│   ├── fetch-github-stars.mjs
│   └── compare-models.mjs
├── validate/
│   ├── visit-all-urls.mjs
│   ├── visit-urls-en-static-all-slugs.mjs
│   ├── visit-urls-en-static-one-slug.mjs
│   ├── visit-urls-all-locales-static-all-slugs.mjs
│   ├── visit-urls-all-locales-static-one-slug.mjs
│   └── lib/                   # Shared validation utilities
├── _shared/
│   └── runner.mjs             # Shared script runner for entry points
└── temp/                      # Temporary experimental scripts

Each category has an entry point script (index.mjs) that can run all scripts in that category:

# Run all validation tests
npm run test:validate

# Run all generation scripts
npm run generate

# Run all refactoring scripts
npm run refactor

# Run all fetch scripts
npm run fetch

# Run all validation scripts (manual execution)
node scripts/validate/visit-all-urls.mjs

You can also run individual scripts by passing the script name to the entry point:

# Generation scripts
npm run generate:manifests
npm run generate:metadata

# Refactoring scripts
npm run refactor:sort-fields

# Fetch scripts
npm run fetch:github-stars

# Validation scripts (run directly with node)
node scripts/validate/visit-all-urls.mjs
node scripts/validate/visit-urls-en-static-all-slugs.mjs
node scripts/validate/visit-urls-all-locales-static-all-slugs.mjs

Or directly using Node:

# Run generation/fetch scripts
node scripts/generate/index.mjs metadata
node scripts/fetch/index.mjs github-stars
node scripts/fetch/index.mjs compare-models

# Run specific refactor scripts
node scripts/refactor/index.mjs sort-manifest-fields
node scripts/refactor/index.mjs sort-locales-fields
node scripts/refactor/index.mjs export-vendors

# Run validation scripts
node scripts/validate/visit-all-urls.mjs
node scripts/validate/visit-urls-en-static-all-slugs.mjs
node scripts/validate/visit-urls-all-locales-static-all-slugs.mjs

Validation is implemented as Vitest-based automated tests under tests/validate/.

npm run test:validate

What it checks:

• JSON syntax validity
• Schema compliance for each manifest type
• Required fields presence
• Field format validation (URLs, enums, etc.)
• Filename matches the id field in the manifest

Manifest types validated:

• manifests/clis/*.json - CLI tools
• manifests/ides/*.json - IDEs
• manifests/extensions/*.json - Editor extensions
• manifests/providers/*.json - API providers
• manifests/models/*.json - LLM models
• manifests/vendors/*.json - Vendor information
• manifests/collections.json - Collections data

npm run test:validate

What it checks:

• All entries in github-stars.json have corresponding manifest files
• All manifest files are present in github-stars.json
• No orphaned entries in either direction

Categories validated:

• extensions
• clis
• ides

Common issues:

• Orphaned entries: Entries in github-stars.json without manifest files
• Missing entries: Manifest files without corresponding github-stars.json entries

How to fix:

1. Remove orphaned entries from data/github-stars.json
2. Add missing entries to data/github-stars.json (set value to null if unknown)
3. Or remove unused manifest files if they are not needed

npm run test:urls

What it checks:

• URL accessibility (HTTP status codes)
• Network connectivity
• URL format validity

Note: This check makes HTTP requests and can be flaky; it is typically run in CI and configured as non-blocking.

Generates TypeScript index files from individual manifest files.

npm run generate:manifests

What it generates:

• src/lib/generated/ides.ts - IDE manifest index
• src/lib/generated/clis.ts - CLI manifest index
• src/lib/generated/models.ts - Model manifest index
• src/lib/generated/providers.ts - Provider manifest index
• src/lib/generated/extensions.ts - Extension manifest index
• src/lib/generated/vendors.ts - Vendor manifest index
• src/lib/generated/index.ts - Main manifest index
• src/lib/generated/github-stars.ts - GitHub stars data

Generates TypeScript metadata files from MDX content and manifest data.

npm run generate:metadata

What it generates:

• src/lib/generated/metadata.ts - Articles, docs, FAQ, and collections metadata
• src/lib/generated/articles.ts - Article components and metadata
• src/lib/generated/docs.ts - Doc components and metadata
• src/lib/generated/manifesto.ts - Manifesto component loader

Sorts fields in manifest JSON files according to their schema definitions.

npm run refactor:sort-fields
# or
node scripts/refactor/index.mjs sort-manifest-fields

What it does:

• Reorders fields in manifest files to match schema property order
• Ensures consistent field ordering across all manifests
• Handles nested objects and arrays

Sorts fields in locale translation JSON files alphabetically for consistency.

node scripts/refactor/index.mjs sort-locales-fields

What it does:

• Sorts keys in translation files (translations/*/pages/*.json, translations/*/components.json, etc.)
• Ensures consistent ordering across all locale files
• Helps with maintainability and git diff readability

Exports vendor information from manifest files to vendor manifests.

node scripts/refactor/index.mjs export-vendors

What it does:

• Extracts vendor data from ide, cli, extension, model, and provider manifests
• Creates vendor files in manifests/vendors/ if they don't already exist
• Merges vendor information from multiple manifest sources
• Skips existing vendor files (does not overwrite)

Note: This script only creates new vendor files. It will skip vendors that already have a manifest file.

Fetches GitHub star counts for projects listed in manifests.

npm run fetch:github-stars
# or
node scripts/fetch/index.mjs github-stars

What it does:

• Read githubUrl from manifest files
• Fetches star counts from GitHub API
• Updates data/github-stars.json with latest counts

Environment variables:

• GITHUB_TOKEN - Optional GitHub token to avoid rate limits (recommended)

Note: Without a GitHub token, you may hit rate limits (60 requests/hour).

Compares model manifest data with API reference data to identify mismatches.

node scripts/fetch/index.mjs compare-models
# or
node scripts/fetch/compare-models.mjs

What it does:

• Reads model manifests from manifests/models/
• Compares with API reference data from tmp/models-dev-api.json
• Uses mapping from manifests/mapping.json to match vendors and models
• Reports matched models, mismatched fields, and unmatched models

Output:

• Perfectly matched models (all fields match)
• Matched models with field mismatches
• Matched models with skipped fields (null in manifest)
• Unmatched models (not found in API)

Note: This script requires tmp/models-dev-api.json to be present. It's used for data validation and synchronization.

Fetching benchmark performance data is handled by a separate skill:

node .claude/skills/benchmark-fetcher/scripts/fetch-benchmarks.mjs

What it does:

• Fetches benchmark scores from 6 leaderboard websites using Playwright MCP
• Supports SWE-bench, TerminalBench, SciCode, LiveCodeBench, MMMU, MMMU Pro, and WebDevArena
• Updates model manifests with latest benchmark scores

For full documentation, see .claude/skills/benchmark-fetcher/SKILL.md

The validate/ directory contains scripts for checking website URL accessibility. These scripts visit URLs and verify they return successful HTTP responses.

Visits all URLs on the website (all locales, all static pages, all slugs).

node scripts/validate/visit-all-urls.mjs

What it does:

• Builds a comprehensive list of all website URLs
• Visits each URL with HTTP HEAD requests
• Reports success/failure status for each URL
• Supports retries and timeout handling

Environment variables:

• BASE_URL - Base URL to test (default: http://localhost:3000)

Visits URLs with specific configuration: English locale only, all static pages, all slugs per route type.

node scripts/validate/visit-urls-en-static-all-slugs.mjs

Configuration:

• Locales: English only
• Static pages: All
• Dynamic routes: All slugs for each route type

Visits URLs with specific configuration: English locale only, all static pages, one slug per route type.

node scripts/validate/visit-urls-en-static-one-slug.mjs

Configuration:

• Locales: English only
• Static pages: All
• Dynamic routes: One slug per route type (for faster testing)

Visits URLs with specific configuration: All locales, all static pages, all slugs per route type.

node scripts/validate/visit-urls-all-locales-static-all-slugs.mjs

Configuration:

• Locales: All configured locales
• Static pages: All
• Dynamic routes: All slugs for each route type

Visits URLs with specific configuration: All locales, all static pages, one slug per route type.

node scripts/validate/visit-urls-all-locales-static-one-slug.mjs

Configuration:

• Locales: All configured locales
• Static pages: All
• Dynamic routes: One slug per route type (for faster testing)

Common features:

• Concurrent requests (default: 5 concurrent)
• Request timeout (default: 10 seconds)
• Automatic retries (default: 2 retries)
• Summary statistics

Note: These scripts make HTTP requests and require the website to be running. They are useful for pre-deployment validation and CI/CD pipelines.

The build process runs validation tests and generation scripts automatically:

npm run build:next

This runs in order:

1. test:validate - Validate repository data integrity (schemas, translations, alignment, etc.)
2. generate:manifests - Generate manifest indexes
3. generate:metadata - Generate TypeScript metadata
4. Next.js build

During development, use:

npm run dev

This will:

1. Generate manifest indexes
2. Generate metadata
3. Start Next.js development server

For CI/CD pipelines, you can run the validation test suite:

# Run validations (recommended for CI)
npm run test:validate

# Run all generation scripts
npm run generate

# Run all refactoring scripts
npm run refactor

# Run all fetch scripts (if needed)
npm run fetch

# Run URL validation scripts (if needed)
node scripts/validate/visit-all-urls.mjs

Or run individual checks as needed:

npm run test:validate
npm run generate:manifests
npm run generate:metadata
npm run refactor:sort-fields
npm run refactor:sort-locales-fields
npm run fetch:github-stars
node scripts/fetch/index.mjs compare-models
node scripts/validate/visit-all-urls.mjs

To run tests manually without npm, you can use Vitest directly:

vitest run tests/validate --reporter=verbose