Skip to main content

Changelog Ordering Investigation Report

Status: Complete
Owner: AI Dev Kit / Book Project Lead
Last Updated: 2025-12-04
Related Work: Epic 3, Story 2, Task 6 (E3:S02:T06)

Executive Summary​

Issue: Changelog entries appeared in incorrect order (v0.3.2.4+1 before v0.2.4.9+3), violating the canonical ordering principle which states versions must be ordered by version number (RC.EPIC.STORY.TASK+BUILD).

Root Cause: Release Workflow Step 4 (Update Main Changelog) uses chronological insertion (newest at top) instead of canonical ordering (by version number).

Impact: Critical - breaks versioning schema's core invariant and violates documented policy.

Resolution: Updated RW Step 4 to implement canonical ordering logic and added validation to RW Step 8.

1. Root Cause Analysis​

1.1 Issue Discovery​

Date: 2025-12-04
Version Affected: v0.3.2.4+1 (Epic 3) and v0.2.4.9+3 (Epic 2)
Symptom: Changelog showed 0.3.2.4+1 before 0.2.4.9+3, but canonical ordering requires 0.2.4.9+3 before 0.3.2.4+1 (Epic 2 < Epic 3).

Canonical Ordering Rule:

  • Versions must be ordered by version number: RC.EPIC.STORY.TASK+BUILD
  • Comparison order: RC → EPIC → STORY → TASK → BUILD
  • 0.2.4.9+3 (Epic 2) < 0.3.2.4+1 (Epic 3) because EPIC component: 2 < 3

1.2 Root Cause: RW Step 4 Implementation​

File: packages/frameworks/workflow mgt/docs/documentation/Developer_Docs/vwmp/release-workflow-agent-execution.md
Section: Step 4: Update Main Changelog
Lines: 599-606

Problematic Instructions:

2. **DETERMINE:**
   - Find insertion point: After "## Recent Releases" header, before first existing entry
   
3. **EXECUTE:**
   - Insert new entry at top of Recent Releases section

Root Cause:

  1. Chronological Assumption: Step 4 assumes newest entries go at the top (chronological order)
  2. No Version Comparison: Step 4 does not read existing changelog entries to extract version numbers
  3. No Ordering Logic: Step 4 does not compare new version with existing versions to find correct insertion point
  4. Missing Validation: Step 4 validation (line 622) only checks "entry is at top of Recent Releases" - doesn't validate canonical ordering

What Should Happen:

  1. Read all existing changelog entries
  2. Extract version numbers from each entry (parse ## [version] headers)
  3. Compare new version with existing versions using canonical ordering
  4. Find correct insertion point based on version number comparison
  5. Insert new entry at correct position (not necessarily at top)

1.3 Process Gap Analysis​

Missing Validation Steps:

  1. RW Step 4 (Update Main Changelog):

    • ❌ No version number extraction from existing entries
    • ❌ No version number comparison logic
    • ❌ No canonical ordering validation
    • ❌ Validation only checks "at top" (line 622), not "correct order"

  2. RW Step 8 (Run Validators):

    • ❌ validate_changelog_format.py does not check ordering
    • ❌ No canonical ordering validator exists
    • ❌ No validation that changelog entries are ordered by version number

Gap Summary:

  • Insertion Logic: Uses chronological insertion instead of canonical ordering
  • Validation: No validation of changelog ordering exists
  • Documentation: Step 4 instructions don't mention canonical ordering requirement

2. Corrective Actions​

2.1 Updated RW Step 4 Instructions​

File: packages/frameworks/workflow mgt/docs/documentation/Developer_Docs/vwmp/release-workflow-agent-execution.md

Changes Made:

  1. ANALYZE Phase (Updated):

    • Added requirement to read all existing changelog entries
    • Added requirement to extract version numbers from existing entries
    • Added requirement to understand canonical ordering rules

  2. DETERMINE Phase (Updated):

    • Changed from "Find insertion point: After header, before first entry"
    • To: "Find insertion point: Compare new version with existing versions, find correct position based on canonical ordering"
    • Added version comparison logic

  3. EXECUTE Phase (Updated):

    • Changed from "Insert new entry at top of Recent Releases section"
    • To: "Insert new entry at correct position based on version number comparison"
    • Added version parsing and comparison implementation

  4. VALIDATE Phase (Updated):

    • Changed from "Ensure entry is at top of Recent Releases"
    • To: "Verify entry is in correct canonical order (by version number)"
    • Added ordering validation check

New Step 4 Logic:

# Pseudo-code for canonical ordering insertion
def find_insertion_point(new_version, existing_entries):
    """
    Find correct insertion point for new_version in existing_entries
    using canonical ordering (RC.EPIC.STORY.TASK+BUILD).
    """
    new_components = parse_version(new_version)  # (RC, EPIC, STORY, TASK, BUILD)
    
    for i, entry in enumerate(existing_entries):
        existing_version = extract_version(entry)
        existing_components = parse_version(existing_version)
        
        # Compare components in order: RC → EPIC → STORY → TASK → BUILD
        if compare_versions(new_components, existing_components) < 0:
            return i  # Insert before this entry
    
    return len(existing_entries)  # Insert at end

2.2 Added Changelog Ordering Validation​

File: packages/frameworks/workflow mgt/scripts/validation/validate_changelog_format.py

Changes Made:

  • Added function to extract version numbers from changelog entries
  • Added function to parse version components (RC.EPIC.STORY.TASK+BUILD)
  • Added function to compare versions using canonical ordering
  • Added validation check that all entries are in canonical order
  • Added error reporting for ordering violations

Validation Logic:

def validate_changelog_ordering(changelog_path):
    """
    Validate that changelog entries are in canonical order
    (by version number, not timestamp).
    """
    entries = extract_changelog_entries(changelog_path)
    versions = [extract_version(entry) for entry in entries]
    
    for i in range(len(versions) - 1):
        current = parse_version(versions[i])
        next_version = parse_version(versions[i + 1])
        
        if compare_versions(current, next_version) >= 0:
            raise ValidationError(
                f"Changelog ordering violation: {versions[i]} appears before "
                f"{versions[i + 1]}, but canonical ordering requires "
                f"{versions[i + 1]} before {versions[i]}"
            )
    
    return True

2.3 Updated RW Step 8 Instructions​

File: packages/frameworks/workflow mgt/docs/documentation/Developer_Docs/vwmp/release-workflow-agent-execution.md

Changes Made:

  • Added changelog ordering validation to Step 8 (Run Validators)
  • Updated validator list to include ordering check
  • Added error handling for ordering violations

3. Prevention Strategy​

3.1 Process Improvements​

RW Step 4 (Update Main Changelog):

  • ✅ MANDATORY: Read all existing changelog entries before insertion
  • ✅ MANDATORY: Extract version numbers from existing entries
  • ✅ MANDATORY: Compare new version with existing versions using canonical ordering
  • ✅ MANDATORY: Find correct insertion point based on version comparison
  • ✅ MANDATORY: Validate entry is in correct canonical order after insertion
  • ❌ NEVER: Assume newest entry goes at top
  • ❌ NEVER: Use chronological insertion

RW Step 8 (Run Validators):

  • ✅ MANDATORY: Run changelog ordering validation
  • ✅ MANDATORY: Fail workflow if ordering violation detected
  • ✅ MANDATORY: Report exact ordering violation with version numbers

3.2 Documentation Updates​

Updated Files:

  1. release-workflow-agent-execution.md - Step 4 instructions updated
  2. release-workflow-agent-execution.md - Step 8 instructions updated
  3. validate_changelog_format.py - Added ordering validation
  4. dev-kit-versioning-cookbook.md - Added edge case entry (if not already present)

New Documentation:

  • This investigation report
  • Updated Step 4 canonical ordering procedure
  • Updated Step 8 validation checklist

3.3 Automation Recommendations​

Future Improvements:

  1. Automated Ordering Check: Add automated test that validates changelog ordering on every commit
  2. Pre-commit Hook: Add Git pre-commit hook to validate changelog ordering
  3. CI/CD Validation: Add changelog ordering validation to CI/CD pipeline
  4. Version Comparison Library: Create reusable version comparison function for all tools

4. Verification​

4.1 Fix Verification​

Test Case: Insert 0.2.4.9+3 when 0.3.2.4+1 already exists

Expected Behavior:

  • Step 4 reads existing entry 0.3.2.4+1
  • Step 4 compares 0.2.4.9+3 with 0.3.2.4+1
  • Step 4 determines: 0.2.4.9+3 < 0.3.2.4+1 (Epic 2 < Epic 3)
  • Step 4 inserts 0.2.4.9+3 before 0.3.2.4+1
  • Step 4 validates: Entry is in correct canonical order
  • Step 8 validates: Changelog ordering is correct

Result: ✅ PASS - Changelog entries now in correct canonical order

4.2 Validation Test​

Test Case: Run validate_changelog_format.py on changelog with ordering violation

Expected Behavior:

  • Validator detects ordering violation
  • Validator reports exact violation: "0.3.2.4+1 appears before 0.2.4.9+3"
  • Validator fails with non-zero exit code
  • RW Step 8 stops workflow

Result: ✅ PASS - Validator correctly detects and reports ordering violations

5. Lessons Learned​

5.1 Key Insights​

  1. Chronological vs. Canonical: Don't assume chronological order matches canonical order
  2. Version Comparison: Always compare versions using canonical ordering rules
  3. Validation Gaps: Missing validation can allow violations to persist
  4. Documentation Clarity: Instructions must explicitly state canonical ordering requirement

5.2 Process Improvements​

  1. Explicit Ordering Logic: Step 4 now explicitly implements canonical ordering
  2. Validation Layer: Step 8 now validates ordering as a safety check
  3. Clear Instructions: Documentation now clearly states ordering requirements
  4. Error Prevention: Validation catches violations before they reach main branch

5.3 Future Considerations​

  1. Automated Testing: Consider adding automated tests for changelog ordering
  2. Pre-commit Hooks: Consider adding Git hooks to prevent ordering violations
  3. Version Library: Consider creating reusable version comparison utilities
  4. CI/CD Integration: Consider adding ordering validation to CI/CD pipeline

6. References​

Policy Documents:

  • docs/architecture/standards-and-adrs/dev-kit-versioning-policy.md - Section 8: Canonical Ordering Principle
  • packages/frameworks/numbering & versioning/versioning-strategy.md - Core Principle: Version Numbers Are Canonical

Workflow Documentation:

  • packages/frameworks/workflow mgt/docs/documentation/Developer_Docs/vwmp/release-workflow-agent-execution.md - Step 4 and Step 8

Validation Scripts:

  • packages/frameworks/workflow mgt/scripts/validation/validate_changelog_format.py

Related Work:

  • Epic 3, Story 2, Task 6 (E3:S02:T06) - Investigate and harden changelog ordering process

Last Updated: 2025-12-04
Document Version: 1.0.0