Skip to content

doc engine#1179

Open
micahscopes wants to merge 225 commits intoargotorg:masterfrom
micahscopes:doc-engine
Open

doc engine#1179
micahscopes wants to merge 225 commits intoargotorg:masterfrom
micahscopes:doc-engine

Conversation

@micahscopes
Copy link
Collaborator

@micahscopes micahscopes commented Dec 10, 2025

Builds on #1175.

Summary

Interactive documentation engine for Fe, with SCIP-powered code intelligence that works both as static sites, embeddable web components, and live-updating pages during LSP development.

What this adds

fe doc CLI — Subcommand-based interface for documentation generation:

fe doc                          # summary (default)
fe doc static -o ./dist         # self-contained index.html with SCIP
fe doc json -o ./dist           # docs.json for web component consumption
fe doc bundle -o ./dist         # fe-web.js component bundle (~900KB, ~220KB gzipped)
fe doc pages --base-url /api/   # Starlight-compatible markdown pages
fe doc serve --port 8080        # live dev server

Shared flags: path (positional), --output, --builtins (include core/std).

fe-web crate — Documentation model, rendering, and web components

  • DocIndex model types with module tree, trait impls, rich signatures
  • Static site generator (single-page HTML with hash routing)
  • Standalone fe-web.js bundle for external site integration
  • Merged docs.json format (DocIndex + SCIP in one file)

Web components — Flat, provider-free elements for embedding Fe docs:

  • <fe-code-block> — Syntax highlighting + SCIP annotations. Attributes: symbol (fetch source from DocIndex), region (extract // #region name blocks), data-file, line-numbers, collapsed
  • <fe-doc-item symbol="..."> — Full doc item rendering, delegates to the same renderDocItem() as the static site. Attributes: compact, show-source
  • <fe-symbol-link symbol="..."> — Inline type link with SCIP hover + tooltip
  • <fe-signature data="..."> — Type-linked function signature
  • <fe-search> — Fuzzy search over the doc index

Integration API — One script tag, one data file:

<script src="fe-web.js" data-src="docs.json" data-docs="/api/"></script>
<fe-code-block symbol="mylib::Game/struct"></fe-code-block>
<fe-symbol-link symbol="mylib::Game/struct">Game</fe-symbol-link>
  • data-src points to docs.json (fetched at load time)
  • data-docs sets the base URL for type links to the full API docs
  • CSS custom properties for theming: --fe-hl-keyword, --fe-hl-type, --fe-code-bg, --fe-code-font, etc.

SCIP code intelligence — Click-to-navigate, hover highlighting, symbol search

  • fe scip CLI for standalone SCIP index generation
  • Full positional symbol resolution in source code and signature blocks
  • Cross-ingot reference tracking (e.g. Option from core is clickable in user code)
  • Virtual SCIP documents for signature code blocks (__sig__/ paths)
  • inject_doc_urls links SCIP symbols to documentation pages
  • enrich_signatures overlays compiler-resolved type links onto signatures
  • Shared helpers (feEnrichLink, feWhenReady, feFindItem, feEscapeHtml) eliminate duplication across components

Live doc reload via LSP — Docs update as you edit code

  • Combined HTTP+WebSocket server serves docs alongside LSP
  • fe/docReload notification pushes updated DocIndex + SCIP to browser
  • Debounced regeneration using read-only Salsa snapshots (500ms after file changes)
  • Shadow DOM in <fe-code-block> preserves raw source while re-rendering highlighted content
  • Cross-component highlight synchronization across shadow roots

HIR additionsSymbolView and ReferenceIndex in hir::core::semantic

  • SymbolView: unified API for item and sub-item metadata (name, kind, docs, signature, children)
  • ReferenceIndex: inverted index of scope references, built once per ingot
  • Used by SCIP, LSIF, and doc extraction to avoid redundant HIR traversals

Files changed

Area New Modified
crates/fe-web/ 24 files (~9k LOC) -
crates/fe/src/ (doc infra) 4 files (~2k LOC) main.rs, scip_index.rs, lsif.rs, Cargo.toml
crates/hir/ symbol.rs (~600 LOC) scope_graph.rs, semantic/mod.rs (small)
crates/language-server/ combined_server.rs, ws_lsp.rs server.rs, handlers.rs, lib.rs, + 8 others
Test fixtures 10 snapshot files -

Test plan

  • cargo clippy --workspace --all-targets --all-features clean
  • cargo fmt --all -- --check clean
  • cargo check -p fe-hir --target wasm32-unknown-unknown passes
  • cargo test -p fe-web — 46 tests pass
  • cargo test -p fe-language-server — 83 tests pass
  • cargo test -p fe — 260 tests pass (36 unit + 15 CLI + 208 fixture + 1 integration)
  • fe doc static generates working site with SCIP navigation
  • fe doc json produces merged docs.json
  • fe doc bundle writes standalone fe-web.js
  • Cross-ingot types (Option, etc.) resolve and link correctly
  • fe lsp with doc server: live reload updates docs on file edit
  • fe/navigate (hover in editor → browser follows) works

Add a generic items_in_scope API to HIR that collects all visible items
in a given scope across the specified name domains. This follows the same
pattern as available_traits_in_scope and properly handles:
- Named imports
- Glob imports
- Unnamed/prelude imports
- Direct child items
- Parent scope items (recursively)

Add ScopeId::items_in_scope() as a convenient traversal-like method.

Rewrite completion handler to:
- Find the most specific scope at cursor position
- Use items_in_scope to get properly scoped completions
- Convert NameRes items to appropriate CompletionItem kinds

This fixes completion showing random unrelated symbols and provides
context-aware suggestions that respect lexical scope.
- Set insert_text, insert_text_format, and insert_text_mode to prevent
  unwanted text replacement and indentation issues
- Detect when completion is triggered by '.' for future member access
  implementation
- Add TODO for member completion (needs proper public API for type members)

This fixes the issue where completion would replace existing text and
add extra indentation.
Use top_mod.target_at() instead of reference_at() + target_at() so that
rename works when cursor is on a definition name, not just references.
- Add textDocument/implementation handler for traits and trait methods
  - On traits: navigates to all impl Trait blocks
  - On trait methods: navigates to all implementations of that method

- Fix trait method rename to also rename implementations
  - When renaming a trait method, all corresponding method definitions
    in impl blocks are also renamed
- Remove the heuristic text scanner that built name→symbol maps from
  existing occurrences and searched signature text for matches. This
  was compensating for missing HIR visitor coverage (now fixed).
- Keep targeted extra_names injection for parent type params in child
  signatures (known parent-scoped symbols, not heuristic scanning).
- Index generic params of child items (trait methods) in the child-
  indexing path since items_dfs no longer visits them.
- Add regression tests for correct trait method SCIP symbols (Foo#bar.
  not Foo/bar.) and method-level type param indexing.
- Add generic_params() method to SymbolView in HIR semantic API so
  both LSIF and SCIP generators use the same traversal
- Extract index_generic_params_for() helper in SCIP to eliminate
  duplicated generic param indexing code
- Refactor index_unnamed_item_generic_params to use the shared helper
- Update LSIF to use SymbolView::generic_params() instead of manual
  scope_graph.children() filtering
When ingot config metadata doesn't have a name field, fall back to
kind-based names ("core", "std") instead of "unknown". This fixes
symbol identity mismatches where definitions use "unknown 0.0.0" but
cross-ingot references use the proper "core 0.1.0" package name.
Impl/ImplTrait blocks have no resolvable pretty_path, so item_symbol()
returns None and the child-indexing block is never reached. Meanwhile,
methods inside these blocks are skipped by items_dfs because their
parent is a container. This left method-level type params (e.g. T, U, F
in `impl Applicative for Result<E> { fn ap<T, U, F>(...) }`) unindexed.

Fix: after indexing an unnamed item's own generic params, also iterate
its children and index their generic params with synthetic parent
symbols. Add regression tests for impl block method type params and
builtin core ingot symbol identity.
collect_ingot_urls was treating workspace roots as single ingots because
containing_ingot found the workspace's fe.toml and created an Ingot with
no ingot metadata (workspace configs have no name/version). This produced
"unknown 0.0.0" symbols in all SCIP output for workspace doc generation.

Root cause: the function checked containing_ingot before checking whether
the fe.toml was a workspace config. Now it checks for [workspace] first
and expands to member URLs, only falling through to single-ingot mode
when no workspace config is found.

Also introduces generate_scip_with_root to accept workspace root for
relative path computation, ensuring SCIP document paths match the
display_file paths used by the doc extractor's source views. Unifies
enrichment so both file:// and builtin:// ingots go through
enrich_signatures_with_base.
The § anchor link was only visible on hover for impl blocks, method
items, and h2 headers. Now also visible for member-item (variants,
fields, etc.) and implementor-item elements.

Also fix double border on signature blocks containing fe-code-block
by suppressing the inner wrapper's border/margin/background.
The source toggle was inside the flex item-header, causing code blocks
to be constrained to the right side of the header row. Now the file:line
stays in the header as a static label, and the expandable source view
sits below as a full-width collapsible block capped at 70vh.
Signature blocks were wrapped in <pre class="signature"> around
<fe-code-block>, causing double borders since the shadow DOM wrapper
has its own border/background. Now fe-code-block gets the class
directly (e.g. class="signature") with no outer wrapper.

For inline contexts (member-header, code-header, implementor-sig),
use CSS custom properties (--fe-code-bg, --fe-code-border) which
inherit through shadow DOM boundaries, replacing the unreachable
descendant selectors that couldn't pierce the shadow root.
The file:line label now serves as the toggle summary for the source
view, instead of being in the header with a separate "Source" button
below. Clicking the file path expands the source code.
Line numbers were 0.8rem while code text was 0.875rem, causing them
to drift out of alignment over many lines. Both now use 0.875rem.
Both line numbers and code text now use 0.8rem (down from 0.875rem),
keeping them aligned while making source views visually smaller than
the surrounding documentation text. Tightened padding to match.
The item name (h1) and signature wrapper now show a subtle § link on
hover for easy deep-linking. Also tighten member-header gap to reduce
space between the § anchor and the member signature.
Reduce gap to 0.25rem and zero out anchor margin-right for
member-header, method-header, and implementor-item elements.
Move signature-wrapper anchor to left margin, remove gap properties
from member-header/method-header/implementor-item, and zero out
anchor margin-right for inline contexts so padding alone controls
spacing.
…e margin

- Move all § anchors (member-header, method-header, implementor-item,
  impl-header, h2, item-header h1, signature-wrapper) to absolute
  positioning at left: -1.25rem in the page padding
- Remove gap properties from inline flex containers; padding alone
  controls spacing
- Remove duplicate shadow-DOM-unreachable styles from styles.css;
  fe-highlight.css is now the single source of truth for code block
  internals
- Add --fe-code-size and --fe-code-line-height CSS custom properties
  for consistent theming
- Add explicit line-height on line-number spans to prevent drift
- Remove all absolute-positioning hacks from anchors; use simple
  inline flex-item approach with fixed 1rem width across all contexts
- Fix stray CSS brace that broke stylesheet parsing
- Remove duplicate position:relative from containers
- Fix line number / code drift: set font properties on <pre> directly
  and use inherit on <code> to prevent browser defaults from diverging
- Remove dead shadow-DOM-unreachable wrapper/gutter/code rules from
  styles.css
- Unify border-radius to 4px across impl-block, implementor-item,
  member-item, method-item
- Match padding across impl-block summary and inner items (0.375rem
  0.625rem)
- Add background/border-radius to method-item to match member-item
- Remove redundant summary/no-toggle padding overrides
- Remove signature-wrapper: main item signature is just a
  fe-code-block now (h1 anchor is sufficient)
- Add align-self: center and line-height: 1 to .anchor for vertical
  centering
- Move all § anchors to appear after content (not before) in member
  headers, impl headers, method headers, and implementor items
- Make .anchor a simple inline element with margin-left, no flex/width
  tricks
- Restore signature-wrapper for main item signature with border
  suppression on inner fe-code-block to prevent double border
- All anchors now use the same simple pattern: inline, after content,
  appear on hover
Anchors are now the last child in flex rows with margin-left: auto,
pushing them flush to the right edge.
Replace div with <a> so clicking navigates to the impl block.
Use baseline alignment for signature flex rows. Anchor sits
far-right via margin-left: auto.
- Collapse nested if blocks in extract.rs and scip_index.rs
- Use !is_empty() instead of len() >= 1
- Avoid owned string comparison in test
- Switch implementor arrow to ↪, remove bold weight
@micahscopes
Copy link
Collaborator Author

this should be ready as ever

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants