Skip to main content

Scalable KB Pattern for Large Codebases

Task: E1:S03:T06 – Document scalable KB pattern for large codebases
Date: 2025-12-02
Status: IN PROGRESS
Version: Draft

Executive Summary​

This document defines the canonical, scalable KB structure pattern that supports projects ranging from small framework repositories (like ai-dev-kit) to large codebases with 100K+ lines of code. The pattern maintains a 3-level default depth while providing a comprehensive "menu" of sections that projects can adopt based on their needs.

Key Principles:

  • Core sections: Always present (Architecture, Project Management, Changelog, Guides)
  • Optional sections: Scale-dependent (Engineering, Operations, Testing, Enablement, etc.)
  • 3-level default: docs/Section/Category/Item (maximum depth)
  • Self-documenting: Directory names clearly indicate purpose
  • Scalable: Pattern works for small projects (minimal subset) and large projects (full menu)

1. Canonical KB Pattern (Full Menu)​

The canonical KB pattern defines all possible sections that a project might need. Individual projects adopt a subset based on their scope and requirements.

1.1 Core Sections (Always Present)​

These sections are fundamental to any Kanban-driven project using the dev-kit frameworks:

docs/architecture/​

Purpose: Technical standards, ADRs, system design, integration documentation
Audience: Developers, architects, technical leads
Categories:

  • standards-and-adrs/ – Architecture Decision Records, technical standards
  • System_Design/ – System architecture, component design (optional, for large projects)
  • APIs_and_Contracts/ – API documentation, service contracts (optional, for large projects)
  • Security_and_Compliance/ – Security policies, compliance docs (optional, for large projects)
  • Data_and_Storage/ – Data models, storage patterns (optional, for large projects)

Depth: 3 levels (docs/architecture/Category/Item.md)

docs/project-management/​

Purpose: Project management, Kanban, governance, rituals
Audience: Product owners, project managers, team leads
Categories:

  • kanban/ – Kanban board, epics, stories (required)
  • rituals/ – Governance policies, cadences, Release Workflow (required)
  • risk_and_dependencies/ – Risk tracking, dependency management (optional)

Depth: 3-4 levels (docs/project-management/kanban/epics/epic-\{N\}.md)

docs/changelog-and-release-notes/​

Purpose: Release documentation, changelogs, version history
Audience: All team members, users
Categories:

  • changelog-archive/ – Detailed per-version changelogs (required)

Depth: 3 levels (docs/changelog-and-release-notes/changelog-archive/CHANGELOG_v{VERSION}.md)

docs/guides/​

Purpose: User-facing documentation, getting started, how-to guides
Audience: End users, adopters, new contributors
Categories:

  • getting-started/ – Quick start guides, onboarding (recommended)
  • framework-consumption/ – How to adopt dev-kit frameworks (for dev-kit)
  • How_To/ – Step-by-step guides (optional)

Depth: 3 levels (docs/guides/Category/guide-name.md)

1.2 Optional Sections (Scale-Dependent)​

These sections are added as needed for larger, more complex projects:

docs/engineering-and-platform/​

Purpose: Code organization, services, components, tooling
Audience: Engineers, platform teams
When to Add: Projects with multiple services, microservices, or complex platform needs
Categories:

  • Services_and_Components/ – Service documentation, component catalogs
  • Code_Conventions/ – Coding standards, style guides
  • Tooling_and_Integrations/ – Development tools, CI/CD, integrations
  • Infrastructure_as_Code/ – IaC documentation, deployment configs

Depth: 3 levels (docs/engineering-and-platform/Category/service-name.md)

docs/Operations_and_SRE/​

Purpose: Runbooks, monitoring, incident management, reliability
Audience: SREs, operations teams, on-call engineers
When to Add: Projects with production systems requiring operational documentation
Categories:

  • Runbooks/ – Operational procedures, troubleshooting guides
  • Monitoring_and_Observability/ – Monitoring setup, dashboards, alerts
  • Incident_Management/ – Incident response procedures, postmortems
  • Reliability/ – SLOs, error budgets, reliability patterns

Depth: 3 levels (docs/Operations_and_SRE/Category/runbook-name.md)

docs/Testing/​

Purpose: Test strategy, test suites, coverage, testing patterns
Audience: QA engineers, developers
When to Add: Projects with comprehensive testing requirements
Categories:

  • Test_Strategy/ – Testing approach, test pyramid, coverage goals
  • Test_Suites_and_Coverage/ – Test suite documentation, coverage reports
  • Testing_Patterns/ – Testing best practices, patterns

Depth: 3 levels (docs/Testing/Category/test-strategy.md)

docs/Product_and_Experience/​

Purpose: Product vision, roadmap, user stories, UX design
Audience: Product managers, designers, stakeholders
When to Add: Projects with dedicated product management and design teams
Categories:

  • Product_Vision_and_Roadmap/ – Product strategy, roadmaps
  • User_Stories_and_Requirements/ – User stories, requirements docs
  • UX_and_Design/ – Design system, UX patterns, mockups

Depth: 3 levels (docs/Product_and_Experience/Category/product-vision.md)

docs/Enablement/​

Purpose: Onboarding, training, enablement materials
Audience: New team members, customers, partners
When to Add: Projects requiring comprehensive onboarding or customer enablement
Categories:

  • Onboarding_and_Enablement/ – New team member onboarding
  • Internal_Enablement_and_People/ – Internal training, team enablement
  • Customer_Enablement/ – Customer-facing enablement materials

Depth: 3 levels (docs/Enablement/Category/onboarding-guide.md)

docs/Data_and_Insights/​

Purpose: Data models, analytics, insights, reporting
Audience: Data engineers, analysts, product managers
When to Add: Projects with significant data requirements or analytics needs
Categories:

  • Data_Models/ – Data schemas, models
  • Analytics_and_Reporting/ – Analytics setup, reporting dashboards
  • Data_Pipelines/ – ETL pipelines, data processing

Depth: 3 levels (docs/Data_and_Insights/Category/data-model.md)

2. Dev-Kit Instantiation (Minimal Subset)​

The ai-dev-kit repository uses a minimal subset of the canonical pattern:

2.1 Current Structure​

docs/
├── Architecture/                    # Core section
│   └── standards-and-adrs/
├── changelog-and-release-notes/   # Core section
│   └── changelog-archive/
├── project-management/               # Core section
│   ├── kanban/
│   └── rituals/
└── Guides/                         # Core section (planned)
    └── framework-consumption/

Rationale:

  • No runtime code: No need for Engineering, Operations, Testing sections
  • Framework repository: Focus on Architecture (standards), PM (Kanban), and Guides (consumption)
  • Minimal scope: Only core sections needed for framework documentation

2.2 Why This Works​

  • Pattern compliance: Follows canonical 3-level structure
  • Scalable: Can add optional sections if dev-kit grows
  • Self-documenting: Clear purpose for each section
  • Example for others: Shows how to adopt minimal subset

3. Example Project Mapping​

3.1 Large Codebase Example (100K+ LOC)​

Example: A project with the following KB structure:

docs/
├── _index.md
├── Architecture
├── changelog-and-release-notes
├── Data_and_Insights
├── Documentation
├── engineering-and-platform
├── Governance_and_Process
├── Monitoring_and_Observability
├── Onboarding_and_Enablement
├── Operations_and_Runbooks
├── Operations_and_SRE
├── project-management
├── Product_and_Experience
├── Security_and_Compliance
├── Testing
└── Tooling_and_Integrations

3.2 Mapping to Canonical Pattern​

Current SectionCanonical SectionCategoryNotes
Architecturedocs/architecture/standards-and-adrs/Direct mapping
changelog-and-release-notesdocs/changelog-and-release-notes/changelog-archive/Direct mapping
project-managementdocs/project-management/kanban/, rituals/Direct mapping
Documentationdocs/guides/getting-started/, How_To/Consolidate into Guides
Onboarding_and_Enablementdocs/Enablement/Onboarding_and_Enablement/Direct mapping
engineering-and-platformdocs/engineering-and-platform/Services_and_Components/, Code_Conventions/Direct mapping
Operations_and_Runbooksdocs/Operations_and_SRE/Runbooks/Consolidate with Operations_and_SRE
Operations_and_SREdocs/Operations_and_SRE/Monitoring_and_Observability/, Incident_Management/Direct mapping
Monitoring_and_Observabilitydocs/Operations_and_SRE/Monitoring_and_Observability/Consolidate into Operations_and_SRE
Testingdocs/Testing/Test_Strategy/, Test_Suites_and_Coverage/Direct mapping
Security_and_Compliancedocs/architecture/Security_and_Compliance/Move to Architecture
Data_and_Insightsdocs/Data_and_Insights/Data_Models/, Analytics_and_Reporting/Direct mapping
Tooling_and_Integrationsdocs/engineering-and-platform/Tooling_and_Integrations/Move to engineering-and-platform
Governance_and_Processdocs/project-management/rituals/Consolidate into project-management
Product_and_Experiencedocs/Product_and_Experience/Product_Vision_and_Roadmap/, UX_and_Design/Direct mapping
docs/
├── README.md
├── Architecture/
│   ├── standards-and-adrs/
│   └── Security_and_Compliance/
├── changelog-and-release-notes/
│   └── changelog-archive/
├── project-management/
│   ├── kanban/
│   └── rituals/
├── Guides/
│   ├── getting-started/
│   └── How_To/
├── engineering-and-platform/
│   ├── Services_and_Components/
│   ├── Code_Conventions/
│   └── Tooling_and_Integrations/
├── Operations_and_SRE/
│   ├── Runbooks/
│   ├── Monitoring_and_Observability/
│   └── Incident_Management/
├── Testing/
│   ├── Test_Strategy/
│   └── Test_Suites_and_Coverage/
├── Product_and_Experience/
│   ├── Product_Vision_and_Roadmap/
│   └── UX_and_Design/
├── Enablement/
│   └── Onboarding_and_Enablement/
└── Data_and_Insights/
    ├── Data_Models/
    └── Analytics_and_Reporting/

Benefits:

  • ✅ All sections mapped to canonical pattern
  • ✅ 3-level default depth maintained
  • ✅ Self-documenting structure
  • ✅ Clear separation of concerns
  • ✅ Scalable and maintainable

4. Adoption Guidance​

4.1 For New Projects​

Step 1: Start with Core Sections

  • Create docs/architecture/, docs/project-management/, docs/changelog-and-release-notes/, docs/guides/
  • Use minimal categories (e.g., Architecture/standards-and-adrs/)

Step 2: Add Optional Sections as Needed

  • Add docs/engineering-and-platform/ when you have multiple services
  • Add docs/Operations_and_SRE/ when you have production systems
  • Add docs/Testing/ when you need comprehensive test documentation
  • Add other sections based on project needs

Step 3: Maintain 3-Level Default

  • Keep structure shallow: docs/Section/Category/Item.md
  • Use 4th level only when clearly justified (e.g., epic-specific assets)

4.2 For Existing Projects​

Step 1: Map Current Structure

  • List all current KB top-level directories
  • Map each to canonical section (see Section 3.2)

Step 2: Identify Consolidations

  • Merge duplicate sections (e.g., Operations_and_Runbooks + Operations_and_SRE → Operations_and_SRE)
  • Move misplaced sections (e.g., Security_and_Compliance → Architecture/Security_and_Compliance)

Step 3: Plan Migration

  • Create migration plan (see T003 migration guide)
  • Move files to target locations
  • Update cross-references
  • Validate structure

Step 4: Document Decisions

  • Document which canonical sections you're using
  • Document any deviations from canonical pattern (with rationale)
  • Update docs/README.md with structure overview

5. Depth Management Rules​

5.1 Default Rule: 3 Levels Maximum​

Pattern: docs/Section/Category/Item.md

Examples:

  • ✅ docs/architecture/standards-and-adrs/dev-kit-versioning-policy.md (3 levels)
  • ✅ docs/project-management/kanban/epics/epic-1.md (4 levels, acceptable for Kanban)
  • ❌ docs/project-management/kanban/epics/epic-1/stories/story-1/tasks/task-1.md (7 levels, too deep)

5.2 Exception Rule: 4th Level Only When Justified​

When 4th Level is Acceptable:

  • Epic-specific assets: docs/project-management/kanban/epics/epic-1/assets/
  • Service-specific runbooks: docs/Operations_and_SRE/Runbooks/service-x/
  • Category with many items requiring sub-organization

When 4th Level is NOT Acceptable:

  • General content organization (use better naming instead)
  • Avoiding consolidation of related content
  • Creating unnecessary nesting

5.3 Anti-Pattern: Deep Nesting​

Avoid:

  • 5+ level depths
  • Meandering directory structures
  • Unclear purpose at each level

Instead:

  • Use flat collections with self-documenting names
  • Consolidate related content
  • Use better naming conventions (e.g., epic-1-story-1.md instead of epic-1/stories/story-1.md)

6. Implementation Plan​

6.1 Phase 1: Document Canonical Pattern (T006)​

Deliverables:

  • ✅ This document (scalable KB pattern)
  • ✅ Example project mapping
  • ✅ Adoption guidance

Status: IN PROGRESS

6.2 Phase 2: Update KB Structure Principles (T002 Update)​

Actions:

  • Update T002-kb-structure-principles.md to reference canonical pattern
  • Add section on "Core vs Optional Sections"
  • Document depth management rules

Dependencies: T006

6.3 Phase 3: Create Adoption Guide​

Actions:

  • Create docs/architecture/standards-and-adrs/canonical-kb-pattern.md (or similar)
  • Create quick reference guide for projects adopting the pattern
  • Document dev-kit's minimal subset as example

Dependencies: T006, T002 update

6.4 Phase 4: Update Framework Documentation​

Actions:

  • Update Kanban framework README to reference canonical KB pattern
  • Update other framework READMEs as needed
  • Ensure consumption guides reference KB structure

Dependencies: Phase 3

7. References​

  • T001-kb-structure-analysis.md – Current structure analysis
  • T002-kb-structure-principles.md – KB structure principles
  • T003-canonical-kb-structure-research.md – Research on canonical patterns
  • docs/architecture/standards-and-adrs/dev-kit-versioning-policy.md – Versioning policy

8. Conclusion​

The canonical KB pattern provides a scalable, maintainable structure that works for:

  • Small projects: Minimal subset (Architecture, PM, Changelog, Guides)
  • Large projects: Full menu (all sections as needed)
  • Any project: 3-level default depth, self-documenting names

Next Steps:

  1. Complete this document (T006)
  2. Update T002 with canonical pattern references
  3. Create adoption guide
  4. Update framework documentation

End of Scalable KB Pattern Document