Release 0.189.9: Performance Intelligence Restored

(Updated: )
Release Notes
, , , ,

We’re excited to announce Release 0.189.9, which restores and significantly enhances PTR (Public Test Realm) stats collection infrastructure. This release re-establishes the data foundation needed for data-driven bot optimization and autonomous performance monitoring.

The Challenge: Performance Blind Spots

Since the profiler integration was deployed, PTR stats collection had been offline, creating a significant gap in our monitoring capabilities. Without access to comprehensive performance telemetry, the bot operated without critical visibility:

  • No baseline metrics to validate optimization efforts
  • CPU and energy efficiency trends remained unknown
  • Performance regressions went undetected
  • Strategic planning workflows lacked the data needed for autonomous decision-making

The absence of PTR stats collection meant we couldn’t safely validate code changes on the test realm before deploying to production—a fundamental DevOps anti-pattern that introduced unnecessary risk.

Key Features

Historical Trend Analysis

The centerpiece of this release is a new historical trend analysis system (analyze-historical-trends.ts) that transforms raw telemetry into actionable intelligence:

  • Multi-Period Tracking: Analyzes performance across 7-day and 30-day windows
  • Regression Detection: Automatically identifies performance degradation with configurable thresholds
  • Statistical Analysis: Calculates trend direction, percentage changes, and health indicators
  • Comprehensive Metrics: Tracks CPU usage, bucket health, creep populations, room progression, and resource management

The trend analysis examines daily bot snapshots stored in reports/bot-snapshots/ and generates consolidated reports in reports/monitoring/historical-trends.json. This enables both automated monitoring systems and strategic planning agents to detect patterns that would be invisible in single-point measurements.

PTR Stats Schema Documentation

A critical addition for automation is the new PTR Stats Schema Reference (packages/docs/source/docs/analytics/ptr-stats-schema.md). This comprehensive documentation provides:

  • Complete TypeScript interfaces for all telemetry data structures
  • Field-by-field descriptions with expected value ranges
  • Data source locations and update frequencies
  • Integration examples for consuming stats programmatically

This schema documentation is specifically designed to enable autonomous agents (like the strategic planning workflow) to consume performance data without human interpretation. By codifying the data contract, we’ve removed ambiguity and enabled reliable automated analysis.

Enhanced Profiler Data Collection

The profiler integration received critical bug fixes that were blocking data collection:

  • Robust Error Handling: Now gracefully handles “undefined” string responses from Screeps console API
  • Validation Layer: Ensures profiler data parsing doesn’t fail on edge cases
  • Archive Improvements: Better handling of profiler data archival in archive-profiler-data.ts

These fixes restore function-level CPU profiling, enabling fine-grained performance optimization.

Technical Details

Single-point performance measurements are notoriously unreliable in Screeps. A bot’s CPU usage can vary dramatically based on:

  • Number of active creeps and their roles
  • Construction activities and planning operations
  • Combat situations requiring emergency responses
  • Market operations and resource logistics

By analyzing trends over 7-day and 30-day windows, the system can distinguish between:

  • Normal variance: Temporary spikes due to game events
  • Performance regressions: Sustained increases indicating code inefficiency
  • Optimization success: Consistent reductions in resource consumption

The trend analysis calculates statistical measures including:

  • Current vs. previous period averages
  • Absolute and percentage changes
  • Trend direction classification (increasing, decreasing, stable)
  • Health assessments (healthy, warning, critical)

Architecture Integration

The enhanced monitoring system integrates seamlessly with existing infrastructure:

  1. Data Collection: The screeps-monitoring.yml workflow runs every 30 minutes
  2. Trend Analysis: New step processes historical snapshots and generates trends
  3. Artifact Publishing: Trends are committed alongside raw stats for version control
  4. Strategic Consumption: Planning agents read consolidated trend data for decision-making

Files are stored in the reports/ hierarchy:

1
2
3
4
5
6
7
8
9
10
11
reports/
├── monitoring/
│   ├── historical-trends.json  # NEW: Multi-period trend analysis
│   ├── health.json             # Bot health tracking
│   └── baselines.json          # Performance baselines
├── copilot/
│   └── ptr-stats.json          # Aggregated PTR stats
├── profiler/
│   └── latest.json             # Function-level CPU profiling
└── bot-snapshots/
    └── *.json                  # Daily state snapshots

Design Rationale: Why Separate Trend Analysis?

Rather than embedding trend calculations in the stats collector runtime code, we implemented trend analysis as a separate post-processing script. This design choice provides:

  • Performance: No runtime CPU overhead in the game loop
  • Flexibility: Trend windows and thresholds can change without bot redeployment
  • Testability: Statistical logic can be validated with comprehensive unit tests
  • Extensibility: Additional analysis phases can be added without coupling

The script operates on committed snapshots, treating them as an immutable time-series database. This functional approach enables reproducible analysis and easier debugging.

Bug Fixes

Profiler Console API Handling

Fixed critical issue where profiler data collection would fail when Screeps console API returned the string “undefined” instead of valid JSON. The fix adds:

  • String validation before JSON parsing
  • Graceful fallback to empty profiler data
  • Diagnostic logging for troubleshooting collection failures

This resolves the last blocker preventing reliable profiler data export (#1482).

Impact

Unblocking Strategic Planning

The strategic planning agent (copilot-strategic-planner.yml) can now:

  • Analyze performance trends to identify optimization opportunities
  • Validate that implemented improvements had measurable impact
  • Detect regressions early before they affect production
  • Create evidence-based issues with quantified priorities

Enabling Autonomous Optimization

With comprehensive performance telemetry restored, the foundation is set for autonomous optimization workflows:

  • Proactive Bottleneck Detection: Identify CPU hotspots consuming >20% of tick budget
  • Optimization Validation: Measure actual impact of performance improvements
  • Risk Mitigation: Test changes on PTR with full telemetry before MMO deployment
  • Trend Alerts: Automated notifications when 7-day or 30-day trends indicate degradation

Developer Experience Improvements

For human developers, the enhanced monitoring provides:

  • Historical Context: Understand how performance evolved over time
  • Comparative Analysis: Compare current metrics against 7-day/30-day baselines
  • Debugging Support: Function-level profiling to diagnose performance issues
  • Documentation: Clear schema reference for building custom analysis tools

What’s Next

This release establishes the monitoring infrastructure, but several enhancements are planned:

  • Automated Regression Alerts: Push notifications when performance degrades beyond thresholds
  • Baseline Management: Automated baseline establishment after stable periods
  • Correlation Analysis: Connect performance changes to specific code deployments
  • Predictive Modeling: Forecast resource requirements based on expansion plans

The ultimate goal is fully autonomous performance optimization: the bot should detect its own inefficiencies, validate potential improvements on PTR, and deploy verified optimizations to production—all without human intervention.

Acceptance Criteria Met

This release satisfies all acceptance criteria from issue #1507:

  • ✅ PTR stats collected daily and committed to reports/
  • ✅ Profiler data exported with function-level metrics
  • ✅ Historical trends tracked (7-day, 30-day)
  • ✅ Strategic planning agent can consume PTR stats
  • ✅ Performance regression detection configured
  • ✅ Multi-shard support (MMO + PTR)

Testing

The release includes comprehensive test coverage:

  • 20 unit tests for historical trend analysis (tests/unit/historical-trend-analysis.test.ts)
  • Validation tests for metric calculations, period filtering, and regression detection
  • Edge case coverage for empty datasets, single data points, and boundary conditions
  • Schema validation ensuring trend output matches documented interfaces

All tests pass, demonstrating robust implementation of the statistical analysis logic.

This release builds on and resolves:

  • Issue #1549: Restore PTR stats collection for performance validation
  • Issue #1507: Parent feature request tracked by strategic planning agent
  • Issue #1503: Multi-shard monitoring data collection fixes
  • Issue #1485: Multi-shard room discovery implementation
  • Issue #1482: Profiler data export investigation

Conclusion

Release 0.189.9 re-establishes the data intelligence layer that powers autonomous bot development. By restoring comprehensive telemetry collection, implementing trend analysis, and documenting the data schema, we’ve created the foundation for data-driven optimization.

The bot can now “see” its own performance over time, detect patterns, and provide the evidence needed for strategic planning agents to make informed decisions. This is a critical step toward the vision of a fully autonomous Screeps AI that continuously improves itself through empirical validation.

Performance intelligence isn’t just about collecting data—it’s about transforming measurements into actionable insights that drive continuous improvement. With release 0.189.9, that transformation is now automated.

Contributors: Copilot (automation agent), ralphschuler (review and merge)

Build: All tests passing (20 new unit tests), no security vulnerabilities detected

Deployment: Available on PTR shard for validation, automated deployment to MMO after health checks