[Feature]: Carry source version (git commit / Feishu revision) into the knowledge graph — the connectors already fetch it #3192

Affected Component

Backend Service & API

Problem Description

#3191 argues the knowledge graph needs a time axis. This issue argues something narrower and cheaper: for Git- and Feishu-backed corpora, that time does not have to be inferred by an LLM at all — the source already carries authoritative version history, WeKnora already fetches part of it, and the pipeline drops it one step before the graph.

The distinction matters because inferred time and source time are not the same quality of data:

That last row is the important one, and I'll come back to it.

What the code already does

types.FetchedItem (internal/types/datasource.go:313) carries source time:

// When last modified in external system
UpdatedAt time.Time `json:"updated_at"`
// When created in external system. Zero when the source does not expose it.
CreatedAt time.Time `json:"created_at"`

The Feishu connector populates this carefully — it distinguishes content changes from attribute changes, which is exactly the right call:

ObjEditTime    string  // document last edit time — tracks content changes
NodeEditTime   string  // node edit time — only tracks node attribute changes

contentEditTime() prefers ObjEditTime, with a comment noting that NodeEditTime alone would move on cosmetic edits. Ingestion then persists it (datasource_service.go:1264):

if !item.UpdatedAt.IsZero() {
    metadata["source_updated_at"] = item.UpdatedAt.UTC().Format(time.RFC3339)
}

So source time already reaches the knowledge layer.

Where it stops

extract.go:371 — the graph write:

if err = s.graphEngine.AddGraph(ctx,
    types.NameSpace{KnowledgeBase: chunk.KnowledgeBaseID, Knowledge: chunk.KnowledgeID},
    []*types.GraphData{graph},
)

NameSpace has exactly two fields. No source_updated_at, no revision, no commit. The graph write is the one place in the pipeline where provenance-in-time is available upstream and simply not passed down. Combined with graph.go having zero time.Time fields, there is nowhere to put it even if it were passed.

Two sources that could carry much more

GitLab — the connector already resolves commit SHAs and then discards them.

client.go:159 has commitSHA(ctx, id, ref), and connector.go:163 calls it to drive incremental sync. But the per-file item is built without it (connector.go:407):

// UpdatedAt is intentionally left unset: the file's last commit time is not
// fetched here, and a fetch timestamp would be a fabricated source time.

I want to be clear that this comment is correct and principled — refusing to fabricate a source time is the right instinct, and I'd rather have this than a silently wrong timestamp. But the consequence is that the single richest version signal available to any connector (git history: who, when, which lines, which commit) is currently unused. The SHA is already in hand; only the per-file commit metadata is missing.

Feishu — the API accepts a revision parameter, and the connector pins it to latest.

core/blocks.go:130:

path := fmt.Sprintf("/open-apis/docx/v1/documents/%s/blocks?page_size=500&document_revision_id=-1", ...)

document_revision_id=-1 means "current". The parameter exists precisely because Feishu can serve earlier revisions. Today WeKnora only ever asks for the newest one.

Proposed Solution

Three stages, each independently useful and none requiring the graph-semantics change discussed in #3191.

Stage A — stop discarding what is already fetched (connector-local, no schema change).

Put source version into the existing Metadata map, which connectors already populate freely:

Metadata: map[string]string{
    "channel":           types.ConnectorTypeGitLab,
    "gitlab_ref":        ref,
    "gitlab_commit":     headSHA,        // already resolved in Fetch
    "gitlab_path":       file,
}

and for Feishu, carry obj_edit_time / the docx revision alongside source_updated_at. For GitLab, populating UpdatedAt honestly requires one extra API call per file (/repository/commits?path=<file>&ref_name=<ref>&per_page=1) — worth gating behind a config flag, since it is a real cost on large repos.

Stage B — carry source version to the graph write.

Extend NameSpace (or add a sibling parameter) so AddGraph receives the source revision, and write it onto nodes/edges as ordinary properties:

CALL apoc.merge.relationship(source, row.type,
  {}, {source_version: row.source_version, source_time: row.source_time}, target)

This alone is additive and changes no retrieval behaviour — but it makes every edge answerable for "which document revision asserted this".

Stage C — skip re-extraction for unchanged content.

This is where the cost win is. Today a re-parse deletes the whole graph namespace (knowledge_process.go:355 → DelGraph → apoc.periodic.iterate / DELETE) and re-extracts every chunk, at one LLM call each — the repo's own comment calls graph extraction the most expensive fan-out in the pipeline, and #1765 exists because people are scaling model endpoints to survive it.

With a source version on hand, the sync layer can ask git (or Feishu's revision) which files actually changed and re-extract only those. Not a content hash computed after downloading everything — the version system already knows. Cost stops scaling with corpus size and starts scaling with actual change.

This also directly addresses a problem raised in #3120:

文档重新解析会更新 updated_at，但不代表原始内容发生变化

Source version distinguishes "genuinely edited" from "re-ingested" for free. A fetch timestamp cannot.

Alternatives

• Infer validity with the LLM ([Feature]: Give the knowledge graph a time axis — supersede instead of delete on re-parse (field report from 125 days of bi-temporal operation) #3191's approach). Still needed for sources with no version history (uploads, RSS, pasted text), and for world time stated inside the text ("effective 2027-01-01") which no VCS can know. The two are complementary: source version gives knowledge time cheaply and exactly; LLM extraction gives world time. A document-derived graph ideally wants both.
• Content hashing per chunk. Works everywhere but pays the download and diff cost, and gives no author/commit provenance.
• Rely on Wiki revision history. WikiPageRevision versions generated prose, not source documents, so it cannot tell you which upstream commit caused a change.

Use Case

This is the normal shape of enterprise knowledge:

• Git-backed docs-as-code — specs, runbooks, ADRs, API contracts. "Which commit changed this rule, and what did it say before?" is answerable from git today, and unanswerable from WeKnora's graph.
• Feishu wiki — policies and standards edited continuously by many people; obj_edit_time and docx revisions are the audit trail.
• Large mostly-static corpora — where re-parsing one edited file should not cost a full-corpus re-extraction.

Additional Information

A note on where this idea came from, since it may be useful design context. Earlier this year I built a small system for an agent of mine (internally "Soul Guard") whose purpose was to keep a long-running assistant from drifting: it kept its working knowledge as structured documents under git, and treated the commit history as the record of how its understanding evolved — so a later state could be compared against, and re-derived from, an earlier one. The components were deviation signals, guard actions, and cross-review; it was eventually folded into a single identity document rather than kept standalone.

What survived from that experiment is the part relevant here: when knowledge lives in a versioned store, the version history is itself a first-class knowledge artifact. Structure organises what is true; version history records how it became true. Systems that keep only the current projection can answer the first question and permanently lose the second. That is the same gap that shows up in graph.go today.

Related: #3191 (temporal graph semantics — this issue is the cheap, source-authoritative half), #3120 (retrieval-time time-awareness), #1765 (extraction throughput — Stage C addresses the root cause rather than adding endpoints), #1257 (Wiki → GraphRAG reuse).

Happy to help with Stage A for the GitLab connector if maintainers think the direction is right — it is contained and does not touch graph semantics.

Confirmation

• I have searched existing issues and confirmed this is a new request
• I understand this request may need discussion and evaluation