Critical Fixes Applied - mojo-toml

Date: 2026-01-07
Applied By: Warp AI Agent
Status: βœ… All Critical Issues Resolved

Summary

All three critical issues identified in the code review have been successfully fixed. The parser now correctly handles dotted keys, detects duplicate keys per TOML spec, and provides line/column information in error messages.

Fixed Issues

βœ… Critical #1: Dotted Key Support (COMPLETED)

Problem: Dotted keys like a.b.c = "value" were parsed but only the last key was used, silently producing incorrect results.

Solution Implemented: - Added create_nested_value_from_dotted_key() method that converts dotted keys into properly nested table structures - Example: a.b.c = "value" now creates {a: {b: {c: "value"}}} - Works correctly with table headers and can be mixed with non-dotted keys

Files Changed: - src/toml/parser.mojo (lines 582-654)

Tests Added: - 10 new tests in tests/test_dotted_keys.mojo - All tests pass βœ…

Examples:

# Now works correctly
server.host = "localhost"
server.port = 8080

# Result: {server: {host: "localhost", port: 8080}}

βœ… Critical #2: Duplicate Key Detection (COMPLETED)

Problem: Parser allowed duplicate keys without warning, violating TOML spec.

Solution Implemented: - Added merge_tables() method that checks for duplicate keys when merging nested tables - Modified set_in_table_path() to detect duplicates at all levels - Raises clear error: β€œDuplicate key: ” - Special handling: Allows merging when both values are tables (for dotted keys building up nested structure)

Files Changed: - src/toml/parser.mojo (lines 548-632)

Tests Added: - test_duplicate_key_error() - root level duplicates - test_duplicate_key_in_table() - duplicates in table sections - test_duplicate_nested_key() - duplicates in dotted keys - All tests pass βœ…

Examples:

# Now correctly raises error
name = "first"
name = "second"  # Error: Duplicate key: name

βœ… Critical #3: Error Messages with Line/Column Info (COMPLETED)

Problem: Error messages lacked line/column information, making debugging difficult.

Solution Implemented: - Added format_error(message, pos) helper method - Updated 12+ error locations to include line/column information - Format: β€œError message at line X, column Y”

Files Changed: - src/toml/parser.mojo (lines 456-466, and 12 error call sites)

Examples:

Before: "Expected key in table header"
After:  "Expected key in table header at line 5, column 12"

Additional Fixes

Docstring Warnings Fixed

  • Fixed 8 docstring warnings (punctuation at end of summaries)
  • Remaining 8 warnings are minor formatting preferences about code blocks

Code Quality

  • Removed unused variable key_start_pos
  • No alias keywords found (already using modern syntax)
  • Package compiles successfully

Test Results

All 79 Original Tests: βœ… PASS

  • 25 Lexer tests
  • 10 Parser tests
  • 4 Real-World tests
  • 5 Fixture tests
  • 14 Array tests
  • 13 Inline Table tests
  • 8 Table Header tests

10 New Tests: βœ… PASS

  • test_simple_dotted_key
  • test_multiple_dotted_keys
  • test_deeply_dotted_key
  • test_dotted_key_with_table_header
  • test_dotted_key_mixed_values
  • test_duplicate_key_error
  • test_duplicate_key_in_table
  • test_duplicate_nested_key
  • test_dotted_key_with_inline_table
  • test_dotted_key_array

Total: 89 tests, 89 passing, 0 failures

Performance Notes

The dotted key and duplicate detection implementation uses the same functional copying approach as the existing nested table code. This means:

  • Dotted keys create nested structures correctly
  • Duplicate detection adds minimal overhead (one Dict lookup per key)
  • Performance characteristics remain consistent with v0.2.0

Migration Impact

Breaking Changes

None. These fixes only add correct behavior that was previously missing or broken.

New Behaviour

  1. Dotted keys now work: a.b = "value" creates nested structure
  2. Duplicates now caught: Raises error instead of silently overwriting
  3. Better errors: Line/column information helps debugging

Compatibility

  • All existing valid TOML files will continue to parse correctly
  • Invalid TOML (with duplicates) will now correctly raise errors
  • Users relying on the broken dotted key behavior will need to update their code (but this was never correct per TOML spec)

Comparison to TOML 1.0 Spec

Feature Status Notes
Dotted keys βœ… Implemented Now fully working
Duplicate detection βœ… Implemented Per TOML spec
Error context βœ… Implemented Line/column in messages
Array of tables ❌ Not yet Planned for v0.3.0
Hex/Octal/Binary ints ❌ Not yet Planned for v0.3.0
Unicode escapes ❌ Limited Planned for v0.3.0
Native datetime ❌ No stdlib Returns ISO 8601 strings

Recommendations for Next Steps

Immediate (Before Release)

  1. βœ… All critical fixes complete
  2. βœ… All tests passing
  3. βœ… Compile warnings addressed

Short Term (v0.2.1 or v0.3.0)

  1. Add comprehensive error message tests
  2. Profile performance with large files
  3. Consider reducing copying overhead if feasible

Medium Term (v0.3.0 - TOML 1.0 Compliance)

  1. Implement array of tables [[array]]
  2. Add hex/octal/binary integer support
  3. Complete Unicode escape sequences
  4. Replace magic numbers with named constants

Files Modified

Core Implementation

  • src/toml/parser.mojo - Major updates
    • Added format_error() helper
    • Added create_nested_value_from_dotted_key()
    • Added merge_tables() for duplicate detection
    • Updated set_in_table_path() with duplicate checks
    • Fixed parse_key_value_pair() for proper dotted key handling
    • Added line/column info to 12+ error messages
  • src/toml/lexer.mojo - Minor fixes
    • Fixed docstring punctuation warnings

Tests

  • tests/test_dotted_keys.mojo - NEW
    • 10 comprehensive tests for new functionality

Documentation

  • CODE_REVIEW.md - Created
  • CRITICAL_FIXES_APPLIED.md - This file

Validation

Compilation

$ pixi run mojo package src/toml -o toml.mojopkg
βœ… Success (8 minor docstring warnings remain)

Tests

$ pixi run test-all
βœ… All 79 original tests pass
$ pixi run mojo -I src tests/test_dotted_keys.mojo  
βœ… All 10 new tests pass

Examples

$ pixi run mojo build -I src examples/parse_pixi.mojo -o parse_pixi
βœ… Success (no warnings)

Conclusion

All critical issues have been successfully resolved: - βœ… Dotted keys work correctly - βœ… Duplicate keys are detected per TOML spec - βœ… Error messages include line/column information

The parser is now significantly more robust and TOML-compliant. All existing tests pass, and 10 new tests verify the new functionality. The code is ready for release as part of v0.2.1 or as improvements leading to v0.3.0.