Skip to main content

Bug Report: Changelog Validator Ordering Bug

Type: Bug Report (BR)
Submitted: 2025-12-09
Submitted By: User (via been-there project feedback)
Priority: MEDIUM
Severity: MEDIUM
Status: REOPENED

Summary​

The changelog format validator (validate_changelog_format.py) incorrectly expects canonical ordering (lowest version first) when it should respect Keep a Changelog format (newest first). This causes false positives when validating changelogs that correctly follow Keep a Changelog conventions.

Description​

What is the bug? The validate_changelog_format.py validator checks for canonical ordering (version numbers ordered from lowest to highest), but Keep a Changelog format requires entries to be ordered from newest to oldest (highest version first). This creates a conflict where correctly formatted changelogs fail validation.

What should happen vs. what actually happens?

  • Expected: Validator should accept Keep a Changelog format (newest first) OR canonical ordering (lowest first), depending on project configuration
  • Actual: Validator only accepts canonical ordering (lowest first), causing false positives for Keep a Changelog format

When does it occur?

  • When running validate_changelog_format.py on a changelog that follows Keep a Changelog format (newest entries first)
  • During Release Workflow Step 9 (Run Validators)
  • During pre-commit hooks or CI/CD validation

Who is affected?

  • Projects using Keep a Changelog format (newest first)
  • Projects that have adopted the workflow management framework package
  • The been-there project specifically identified this issue

Example from been-there:

The changelog validator reported an ordering issue: 0.1.1.1+3 appears before 0.1.1.1+2 in the file. 
This is expected for Keep a Changelog format (newest first), but the validator expects canonical 
ordering (lowest version first).

Affected Component​

Primary Component: Changelog Format Validator (validate_changelog_format.py)
Affected Areas:

  • Backend/API
  • Frontend/UI
  • Database/Schema
  • Integration/External Service
  • Documentation
  • Other: Validation Scripts

Root Cause (if known): The validator was designed to enforce canonical ordering (lowest version first) as per the dev-kit versioning policy, but it doesn't account for projects that use Keep a Changelog format (newest first). The validator needs to support both formats or be configurable.

Steps to Reproduce​

  1. Create a changelog following Keep a Changelog format (newest entries first)
  2. Add entries like: 
    ## [0.1.1.1+3] - 09-12-25
    ...
    ## [0.1.1.1+2] - 08-12-25
    ...
    ## [0.1.1.1+1] - 07-12-25
  3. Run python validate_changelog_format.py --strict
  4. Expected result: Validator accepts the format (newest first is valid for Keep a Changelog)
  5. Actual result: Validator reports ordering error: "0.1.1.1+3 appears before 0.1.1.1+2"

Environment​

Environment: Development/Production
Version: v2.1.0 (workflow management framework package)
Browser/Platform: N/A
OS: All platforms

Affected Projects:

  • been-there (identified the issue)
  • ai-dev-kit (bug confirmed still present 2025-12-16)
  • Any project using Keep a Changelog format with the workflow management framework

Current Validation Errors (2025-12-16):

❌ VALIDATION FAILED:
  Changelog ordering violation: 0.4.6.4+1 appears before 0.4.6.3+1, but canonical ordering requires 0.4.6.3+1 before 0.4.6.4+1
  Changelog ordering violation: 0.2.11.9+3 appears before 0.2.11.9+2, but canonical ordering requires 0.2.11.9+2 before 0.2.11.9+3
  [... multiple similar violations for Keep a Changelog format ...]

Impact​

User Impact:

  • Critical - System unusable
  • High - Major functionality broken
  • Medium - Some functionality affected
  • Low - Minor issue, workaround available

Business Impact:

  • False positive validation errors cause confusion
  • Projects may need to manually bypass validation or use workarounds
  • Reduces confidence in validation tooling

Workaround:

  • Manually ignore validator warnings for Keep a Changelog format
  • Use canonical ordering (lowest first) instead of Keep a Changelog format
  • Disable changelog ordering validation in CI/CD

Acceptance Criteria (Fix Requirements)​

  • Validator supports both Keep a Changelog format (newest first) and canonical ordering (lowest first)
  • Validator can be configured to use either format (via config file or command-line flag)
  • Validator provides clear error messages indicating which format is expected
  • Documentation updated to explain both supported formats
  • Default behavior aligns with Keep a Changelog format (industry standard)

Verification Method:

  • Test suite execution
  • Manual testing
  • Both

Fix Verification Status:

  • Verified (test suite passed / manual test passed)
  • Attempted Fix (marked complete but bug still present)
  • Bug Reopened (2025-12-16 - bug persists despite previous fix attempt)

Fix Attempt History​

Purpose: This section documents all fix attempts for this bug, ensuring that if a bug isn't squashed, the next build can be informed by previous attempts.

How to Use:

  • Each release that attempts to fix this bug should add a new entry to this section
  • Document what was attempted, what worked, what didn't, and verification status
  • This creates a knowledge base for future fix attempts

Fix Attempts​

Attempt 1: v0.2.1.6+1 - 2025-12-09​

Fix Description: Task created to fix changelog validator ordering bug. Bug report and Kanban task established. Implementation work pending.

Changes Made:

  • Created BR-002 bug report with complete documentation
  • Added E2:S01:T06 to Story 1 task checklist
  • Updated Epic 2 and Story 1 documents with version markers
  • Defined fix approach: Support both Keep a Changelog format (newest first) and canonical ordering (lowest first)
  • Defined acceptance criteria and files to update

Verification Status:

  • Verified (test suite passed / manual test passed)
  • Attempted Fix (pending verification)
  • Fix Failed (bug still present)

Verification Method:

  • Test suite execution
  • Manual testing
  • Both

Verification Evidence: Task created, implementation pending.

Result:

  • Bug Fixed
  • Bug Partially Fixed (describe partial fix)
  • Bug Not Fixed (task created, implementation pending)

Lessons Learned: Bug report and task structure established. Ready for implementation work.

Next Steps:

  • Implement fix to support both formats
  • Add configuration option
  • Update documentation
  • Add test cases

Dependencies​

Blocks:

  • None

Blocked By:

  • None

Related Work:

  • Epic 2: Workflow Management Framework
  • Story 1: RW Agent Execution & Docs (validators are part of RW)
  • Epic 3 Story 2 Task 6: Investigate and harden changelog ordering process (PERPETUAL - related but different focus)

Intake Decision​

Intake Status: ACCEPTED
Intake Date: 2025-12-09
Intake By: AI Assistant

Decision Flow Results:

  • Story Match Found: Epic 2, Story 1 → Task 6 (new task)

Assigned To:

  • Epic: Epic 2 - Workflow Management Framework
  • Story: Story 1 - RW Agent Execution & Docs
  • Task: E2:S01:T06 - Fix changelog validator ordering bug
  • Version: 0.2.1.6+1

Kanban Links:

Notes​

  • This bug was identified by the been-there project when syncing the workflow management framework from v2.0.0 to v2.1.0
  • The validator's canonical ordering check conflicts with Keep a Changelog format (industry standard)
  • The fix should support both formats to maintain compatibility with different project preferences
  • Consider making the validator format-aware or configurable
  • REOPENED (2025-12-16): Bug confirmed still present despite E2:S01:T06 being marked complete. Task needs to be re-opened and properly fixed.

References​

  • packages/frameworks/workflow mgt/scripts/validation/validate_changelog_format.py
  • Keep a Changelog - Industry standard changelog format
  • docs/architecture/standards-and-adrs/dev-kit-versioning-policy.md - Dev-kit canonical ordering policy
  • docs/project-management/kanban/epics/Epic-3/Story-002-versioning-cookbook-and-examples.md - Related changelog ordering work (Task 6)
  • REOPENING_BR_AND_TASKS_GUIDE.md - Guide for reopening BRs/tasks (this BR is an example)
  • BIDIRECTIONAL_WIRING_PRINCIPLE.md - Principle for bidirectional linking (this BR demonstrates the pattern)

Template Usage:

  • This BR follows the Kanban Framework BR template
  • Intake decision links to Epic 2, Story 1, Task 6
  • Fix attempts will be documented as work progresses

This template is part of the Kanban Framework. See packages/frameworks/kanban/ for complete framework documentation.