Executive Summary
A systematic literature review across arXiv, PubMed, and Semantic Scholar confirms a significant and actionable research gap: no existing publication proposes, defines, or demonstrates a standard for documenting the intellectual provenance of AI-augmented scientific ideation. Computational provenance for data and code is a mature field; provenance for ideas is not.
This gap exists alongside urgent policy developments — NIH NOT-OD-25-132 (refusing AI-developed ideas as original) and the Genesis Mission Executive Order — that demand exactly this capability. Karpathy's LLM Wiki pattern, which exploded into multiple open-source implementations in April 2026, provides a viable implementation pathway for a "Science Flight Recorder." A full research paper demonstrating process provenance is strongly warranted, timely, and would occupy a novel position in the literature.
Key Takeaways
- The research gap is real and large. Computational provenance is well-studied; ideation provenance is essentially unaddressed.
- The policy window is open. NIH NOT-OD-25-132 and the Genesis Mission EO create demand for ideation provenance that no existing standard fills.
- Karpathy's LLM Wiki provides a viable implementation pathway. The April 2026 explosion of open-source implementations demonstrates a working model for persistent, compoundable knowledge documentation.
- Adjacent fields validate the approach. Work on AI attribution, reproducibility, and AI-augmented ideation all converge on the need for better documentation of the creative process.
- A full research paper is strongly recommended. A demonstration paper implementing process provenance with an LLM Wiki would fill a clear gap and address an urgent policy need.
Key Findings
1 Finding 1: Computational Provenance Exists — Ideation Provenance Does Not
The literature on computational provenance for scientific workflows is mature. Stage et al. (2025) provide a comprehensive literature review establishing that provenance tracking — logging data lineage, transformation steps, and software environments — is well-understood and widely implemented. Koop (2023) established that "the provenance of a scientific result is important, sometimes more important than the actual result." Beber (2025) provides a broad-spectrum reproducibility manifesto that acknowledges current standards address data and code but not ideation.
However, none of these works extend provenance to the creative ideation phase — no paper proposes tracking prompt strategies, AI model versions, iterative concept refinement, or the human-AI dialogue that produces research ideas. This is the gap our paper would fill.
2 Finding 2: Policy Demands Are Creating Urgency
Two major policy developments in late 2025 have created immediate demand for process provenance:
NIH NOT-OD-25-132 (2025): The NIH clarified that it "will not consider applications that are either substantially developed by AI... to be original ideas of applicants." This creates a structural need: researchers must be able to demonstrate that their ideas are human-originated. Without a documentation standard, compliance is ad hoc and unverifiable.
Genesis Mission Executive Order (November 2025): The White House directed federal platforms for automated hypothesis testing and research workflows, emphasizing dataset provenance — but not ideation provenance. This partial coverage creates a policy gap: if workflows are automated but idea generation is undocumented, the record is incomplete.
A 2026 PubMed paper titled "A Call for Clarity: A Unified Checklist for Reporting Use of Large Language Models in Academic Writing" independently confirms that disclosure norms haven't kept pace with adoption. This is the most directly aligned existing work, but it focuses on writing disclosure rather than ideation provenance.
3 Finding 3: The LLM Wiki Ecosystem Provides a Viable Implementation Pathway
The April 2026 emergence of Karpathy's LLM Wiki pattern represents a critical development. Miteski (April 2026) identifies "a visible cluster of personal wiki-style memory architectures" built on the LLM Wiki pattern. Wen and Ku (April 2026) introduce "knowledge compounding" as a measurable concept, validating that self-evolving knowledge wikis produce compounding returns.
The open-source ecosystem has exploded: claude-obsidian (5,448★), llm-wiki-skill (1,647★), obsidian-wiki (1,482★), llm-wiki-compiler (1,280★), and llmwiki (962★) all demonstrate working implementations of persistent, interlinked, AI-augmented knowledge documentation [Source 15]. These systems already capture AI-human dialogue history, interlinked knowledge evolution, timestamped contributions, and attribution of sources. Adding provenance-specific metadata is an incremental step, not a revolutionary one.
4 Finding 4: Adjacent Research Validates the Direction
Multiple lines of research independently converge on the need for process provenance:
- AI-Augmented Ideation: Liu et al. (2025) show LLMs generate novel ideas but struggle with feasibility — underscoring the need for documentation of evaluation. ResearchCube (April 2026) identifies that most AI-assisted ideation tools lack multi-dimensional evaluation.
- AI Attribution: Ghaleb (January 2026) demonstrates that AI-generated code can be detected via fingerprinting — technical precedent for automated provenance logging. Mukherjee and Chang (January 2026) establish that AI exhibits "fluid agency" requiring human sponsorship.
- Citation Failures: Shukla et al. (2025) demonstrate existing citation frameworks are inadequate for AI collaboration.
- Auditable AI: Schemmer et al. (2026) show schema-constrained approaches produce verifiable biomedical outputs.
5 Finding 5: The Biomedical Context Adds Weight
The biomedical research context — where stakes are highest for research integrity — shows particular urgency. Schemmer et al. (2026) demonstrate auditable AI in biomedical evidence extraction. The Helix 1.0 framework (2026) exemplifies the maturation of reproducible ML infrastructure in biomedical contexts. SciSciGPT (2025) demonstrates the meta-pattern of human-AI co-creation that process provenance would document.
The JCTS venue (Journal of Clinical and Translational Science) is precisely where this argument would land with maximum impact: it serves the translational research community that bridges discovery to practice, where attribution and integrity matter most. The 2026 unified checklist paper from PubMed further confirms that leading organizations are calling for standardized AI disclosure — the adjacent demand signal is clear.
Risks, Gaps & Uncertainty
- LLM Wiki volatility: The ecosystem is only weeks old (April 2026). Implementation choices may not age well. Mitigation: document the pattern rather than tie to a specific tool.
- Adoption unknown: Process provenance requires behavior change from researchers. A technical demonstration proves feasibility but not adoption. Mitigation: frame as proof-of-concept with recommendations for incentive design.
- Attribution vs. surveillance: Process provenance must not become a surveillance mechanism. Clear boundaries and consent frameworks are essential. Mitigation: explicitly address ethical design principles.
- Scalability: An LLM Wiki for a single research project is straightforward; federated approaches across institutions remain unexplored. Mitigation: acknowledge as limitation; suggest federated approaches.
- JCTS reviewer expectations: Reviews requested "demonstration of process provenance." A purely theoretical paper likely won't satisfy. Mitigation: prioritize the concrete implementation demonstration.
Recommended Next Actions
Select implementation substrate. Evaluate llm-wiki-compiler vs. claude-obsidian for the demonstration component. Preference for the compiler pattern (source-in, wiki-out) as more provenance-native.
Draft the Introduction and Background sections. These are largely literature-driven and can be written directly from this gap assessment.
Implement the provenance extensions. Add structured logging, human attestation, and immutable audit trail to the chosen LLM Wiki fork. This is the technical core of the paper.
Select a demonstration case study. Ideally the JCTS manuscript development process itself (meta-demonstration), or a representative Duke translational research workflow.
Conduct a second-round literature search prior to submission to capture any late-breaking 2026 publications in this rapidly evolving space.
Target submission: JCTS, 8–12 week timeline from initiation.
Annotated References
[1] Ludwig Stage, Julia Dahlberg, Dimka Karastoyanova. (2025). Provenance of Adaptation in Scientific and Business Workflows — Literature Review. arXiv preprint. Link
Comprehensive literature review establishing that computational provenance is well-studied but intellectual/creative provenance in scientific ideation remains largely unexplored.
[2] David Koop. (2023). When Provenance Aids and Complicates Reproducibility Judgments. arXiv preprint. Link
Seminal work demonstrating that provenance of a scientific result is sometimes more important than the result itself. Establishes precedent for provenance as a first-class scientific concern.
[3] Stefan Miteski. (2026). Memory as Metabolism: A Design for Companion Knowledge Systems. arXiv preprint (April 2026). Link
Describes the April 2026 emergence of personal wiki-style memory architectures based on Karpathy's LLM Wiki pattern. Identifies a 'visible cluster' of designs for persistent knowledge systems.
[4] Shuide Wen, Beier Ku. (2026). Knowledge Compounding: An Empirical Economic Analysis of Self-Evolving Knowledge Wikis under the Agentic ROI Framework. arXiv preprint (April 2026). Link
Introduces knowledge compounding as a measurable concept. Validates that self-evolving knowledge wikis produce compounding returns, providing economic justification for process provenance infrastructure.
[5] Multiple authors. (2026). A Call for Clarity: A Unified Checklist for Reporting Use of Large Language Models in Academic Writing. PubMed (PMID: 42106879).
Most directly relevant paper: proposes standardized disclosure framework for LLM use in academic work. Demonstrates growing recognition that documentation norms for AI-augmented scholarship are urgently needed.
[6] Xiao Liu, Xinyi Dong, Xinyang Gao. (2025). Augmenting Research Ideation with Data: An Empirical Investigation in Social Science. arXiv preprint. Link
Shows LLMs can generate novel research ideas but struggle with feasibility. Underscores the need for documentation systems that track how AI-generated ideas are evaluated and refined.
[7] Zijian Ding, Fenghai Li, Ziyi Wang. (2026). ResearchCube: Multi-Dimensional Trade-off Exploration for Research Ideation. arXiv preprint (April 2026). Link
Identifies that most AI-assisted ideation tools lack multi-dimensional evaluation. Process provenance could fill this gap by documenting the evaluative reasoning alongside generated ideas.
[8] Taher A. Ghaleb. (2026). Fingerprinting AI Coding Agents on GitHub. arXiv preprint. Link
Technical demonstration that AI-generated contributions can be detected and attributed. Supports feasibility of automated process provenance logging.
[9] Anirban Mukherjee, Hannah Hanwen Chang. (2026). Fluid Agency in AI Systems: A Case for Functional Equivalence in Copyright, Patent, and Tort. arXiv preprint. Link
Legal analysis of AI agency demonstrating that AI exhibits 'fluid agency' — stochastic, dynamic behavior without consciousness. Supports need for human sponsorship framework.
[10] Prakash Shukla, Suchismita Naik, Ike Obi. (2025). Rethinking Citation of AI Sources in Student-AI Collaboration within HCI Design Education. arXiv preprint. Link
Demonstrates that existing citation frameworks are inadequate for AI collaboration. Parallel finding to the process provenance gap in professional research.
[11] Multiple authors. (2026). From Chaos to Clarity: Schema-Constrained AI for Auditable Biomedical Evidence Extraction. PubMed (PMID: 41975285).
Demonstrates auditable AI in biomedical context. Shows that schema-constrained approaches produce verifiable outputs — a principle applicable to process provenance documentation.
[12] Moritz E. Beber. (2025). In Pursuit of Total Reproducibility. arXiv preprint. Link
Broad-spectrum reproducibility manifesto from computational systems biology. Acknowledges that current reproducibility standards address data and code but not ideation.
[13] National Institutes of Health. (2025). Supporting Fairness and Originality in NIH Research Applications (NOT-OD-25-132). NIH Guide for Grants and Contracts. Link
Foundational policy document: NIH will not consider AI-substantially-developed applications as original ideas. Creates urgent need for process provenance to distinguish human from AI contributions.
[14] The White House. (2025). Executive Order on Launching the Genesis Mission. Federal Register (November 24, 2025). Link
Directs federal platforms for automated hypothesis testing and research workflows. Emphasizes provenance tracking for datasets but does not extend provenance to ideation — creating a policy gap our paper addresses.
[15] Andrej Karpathy / Open-source community. (2026). Karpathy LLM Wiki — GitHub Ecosystem. Multiple GitHub repositories. Link
Foundational pattern for knowledge documentation. Five+ open-source implementations (claude-obsidian: 5,448★, llm-wiki-skill: 1,647★, obsidian-wiki: 1,482★, llm-wiki-compiler: 1,280★) demonstrating a viable infrastructure for process provenance.
[16] Multiple authors. (2026). Helix 1.0: An Open-Source Framework for Reproducible and Interpretable Machine Learning Workflows. PubMed (PMID: 42130947).
Example of the maturation of computational reproducibility infrastructure. Shows the technical ecosystem is ready for the addition of ideation provenance layers.
[17] Erzhuo Shao, Yifang Wang, Yifan Qian. (2025). SciSciGPT: Advancing Human-AI Collaboration in the Science of Science. Semantic Scholar. Link
Open-source AI collaborator for science of science. Demonstrates the pattern of human-AI co-creation that process provenance would need to document.