Session Telemetry¶

Monitor P2P session performance with structured telemetry events. Track rollbacks, prediction misses, frame advances, and network stats in real time.

Table of Contents¶

Architecture
Quick Start
SessionTelemetry Trait
TelemetryEvent Enum
CollectingTelemetry (Built-in)
Custom Telemetry Observer
Pull-Based Metrics
Spec Violation Observability
ViolationObserver Trait
SpecViolation Struct
CollectingObserver
TracingObserver
ViolationKind Variants
ViolationSeverity Levels
Event Flow
Use Cases
Integration Tips
See Also

Architecture¶

graph TD
    P2P["P2PSession"] -->|calls| ST["SessionTelemetry trait"]

    ST --> R["on_rollback"]
    ST --> PM["on_prediction_miss"]
    ST --> NS["on_network_stats"]
    ST --> FA["on_frame_advance"]

    subgraph Built-in Implementations
        CT["CollectingTelemetry<br/><i>testing</i>"]
        CU["Custom impl<br/><i>your own</i>"]
    end

    ST -.->|impl| CT
    ST -.->|impl| CU

Quick Start¶

Rust

use fortress_rollback::telemetry::{CollectingTelemetry, SessionTelemetry};
use fortress_rollback::SessionBuilder;
use std::sync::Arc;

// 1. Create a telemetry observer
let telemetry = Arc::new(CollectingTelemetry::new());

// 2. Pass to session builder
// MyConfig: your Config impl (see user-guide.md)
let builder = SessionBuilder::<MyConfig>::new()
    .with_telemetry(telemetry.clone());

// 3. After running the session, inspect events
let rollbacks = telemetry.rollbacks();
let misses = telemetry.prediction_misses();
println!("Rollbacks: {}, Prediction misses: {}", rollbacks.len(), misses.len());

`SessionTelemetry` Trait¶

Rust

// With `sync-send` feature enabled:
pub trait SessionTelemetry: Send + Sync {
    fn on_rollback(&self, depth: usize, frame: Frame) { /* no-op */ }
    fn on_prediction_miss(&self, player: PlayerHandle, frame: Frame) { /* no-op */ }
    fn on_network_stats(&self, player: PlayerHandle, stats: &NetworkStats) { /* no-op */ }
    fn on_frame_advance(&self, frame: Frame) { /* no-op */ }
}

// Without `sync-send` feature:
pub trait SessionTelemetry {
    // same methods, no Send + Sync bounds
}

Note

All methods have default no-op implementations. Override only what you need. The Send + Sync supertraits are only required when the sync-send feature is enabled.

Method	Parameters	When Called
`on_rollback`	`depth: usize`, `frame: Frame`	State was rolled back
`on_prediction_miss`	`player: PlayerHandle`, `frame: Frame`	Predicted input was wrong
`on_network_stats`	`player: PlayerHandle`, `stats: &NetworkStats`	Network stats polled
`on_frame_advance`	`frame: Frame`	Frame advanced

`TelemetryEvent` Enum¶

Each variant captures the arguments from its corresponding trait method.

Variant	Fields	When
`Rollback`	`depth: usize`, `frame: Frame`	State was rolled back
`PredictionMiss`	`player: PlayerHandle`, `frame: Frame`	Predicted input was wrong
`NetworkStatsUpdate`	`player: PlayerHandle`, `stats: NetworkStats`	Network stats polled
`FrameAdvance`	`frame: Frame`	Frame advanced

`CollectingTelemetry` (Built-in)¶

Thread-safe observer that accumulates all events for later inspection.

Method	Returns
`new()`	Empty collector
`events()`	`Vec<TelemetryEvent>` -- all events
`rollbacks()`	`Vec<TelemetryEvent>` -- filtered rollback events
`prediction_misses()`	`Vec<TelemetryEvent>` -- filtered prediction misses
`network_stats_updates()`	`Vec<TelemetryEvent>` -- filtered network stats
`frame_advances()`	`Vec<TelemetryEvent>` -- filtered frame advances
`len()`	`usize` -- event count
`is_empty()`	`bool` -- no events collected?
`clear()`	Clear all collected events

Custom Telemetry Observer¶

Implement SessionTelemetry for your own metrics system:

Rust

use fortress_rollback::telemetry::SessionTelemetry;
use fortress_rollback::{Frame, PlayerHandle};
use fortress_rollback::NetworkStats;
use std::sync::atomic::{AtomicUsize, Ordering};

struct MetricsTelemetry {
    rollback_count: AtomicUsize,
    prediction_miss_count: AtomicUsize,
}

impl SessionTelemetry for MetricsTelemetry {
    fn on_rollback(&self, depth: usize, _frame: Frame) {
        self.rollback_count.fetch_add(1, Ordering::Relaxed);
        tracing::info!(depth, "rollback occurred");
    }

    fn on_prediction_miss(&self, player: PlayerHandle, frame: Frame) {
        self.prediction_miss_count.fetch_add(1, Ordering::Relaxed);
        tracing::debug!(%player, %frame, "prediction miss");
    }
}

Pull-Based Metrics¶

P2PSession::metrics() returns an always-on SessionMetrics snapshot. It complements the callback API with cumulative counters and high-water marks that remain available without a telemetry observer. peer_metrics(handle) returns encoded protocol counters and gauges for one remote player or spectator. Both snapshots are cheap to copy and support JSON serialization with the json feature.

Signal	Interpretation
`resimulated_frames / frames_advanced`	Fraction of simulation work spent repairing rollbacks
`rollback_depth_histogram` / `max_rollback_depth`	Distribution and worst rollback cost
`stall_count`	Advances blocked by a full prediction window
`confirmation_lag_current` / `max`	Speculative distance ahead of confirmed history
`checksums_mismatched`	Confirmed desync incidents
`event_queue_high_water` / `events_discarded_total`	Whether the application drains events fast enough
`unknown_source_packets`	Decoded traffic ignored because its source is not a configured endpoint
`pending_output_len`	Unacknowledged per-peer input backlog
`ping_ms`	Latest quality-report round-trip measurement
`average_frame_advantage`	Exact per-endpoint rolling average used by the session's max-aggregated wait controller
`portability_risk_messages_sent`	Messages at or above the conservative 1,200-byte path budget
`fragmentation_risk_messages_sent`	Messages at or above the common 1,472-byte IPv4/UDP payload ceiling

PeerMetrics::bytes_sent and bytes_received are exact encoded Fortress payload sizes. Sent values count protocol enqueue demand rather than observed transport throughput. Both exclude IP/UDP headers. NetworkStats::kbps_sent uses those serialized sizes plus an estimated header and reports UDP-equivalent offered demand in decimal kilobits per second, not adapter acceptance or observed transport throughput. Its ping and latest remote frame-advantage gauges refresh on the configured ProtocolConfig::quality_report_interval (200 ms by default). The rolling-average gauge advances when a local input is successfully queued, using that latest remote sample. P2PSession takes the maximum average_frame_advantage across connected endpoints when deciding whether to emit a wait recommendation.

Poll snapshots on a bounded interval and export deltas for monotonic counters. Do not sum gauges. Alert immediately on checksum mismatches or discarded events; choose workload-specific bounds for rollback rate, confirmation lag, stalls, pending output, and hot-join duration.

Spec Violation Observability¶

The telemetry module also provides a structured pipeline for specification violations -- internal invariant failures detected at runtime.

graph TD
    LIB["Library internals"] -->|report_violation!| VO["ViolationObserver trait"]

    subgraph Implementations
        TO["TracingObserver<br/><i>default, logs via tracing</i>"]
        CO["CollectingObserver<br/><i>testing</i>"]
        MO["CompositeObserver<br/><i>multiple observers</i>"]
    end

    VO -.->|impl| TO
    VO -.->|impl| CO
    VO -.->|impl| MO

`ViolationObserver` Trait¶

Rust

// With `sync-send` feature enabled:
pub trait ViolationObserver: Send + Sync {
    fn on_violation(&self, violation: &SpecViolation);
}

// Without `sync-send` feature:
pub trait ViolationObserver {
    // same method, no Send + Sync bounds
}

`SpecViolation` Struct¶

Each violation carries structured context:

Field	Type
`severity`	`ViolationSeverity`
`kind`	`ViolationKind`
`message`	`String`
`location`	`&'static str`
`frame`	`Option<Frame>`
`context`	`BTreeMap<String, String>`

Builder methods:

Method	Description
`new(severity, kind, message, location)`	Create a new violation
`with_frame(frame)`	Attach a frame reference
`with_context(key, value)`	Add a key-value context entry
`try_to_json()`	`Result<String, JsonSerializationError>` -- compact JSON with exact errors (requires `json` feature)
`try_to_json_pretty()`	`Result<String, JsonSerializationError>` -- pretty JSON with exact errors (requires `json` feature)
`to_json()`	Compatibility `Option<String>` that maps every JSON error to `None`
`to_json_pretty()`	Pretty compatibility wrapper with the same error-erasing fallback

The fallible helpers first count the exact encoded length without building an output buffer, reserve that buffer with try_reserve_exact, and then serialize without further growth. JsonSerializationError retains the original serializer or allocation error through std::error::Error::source. Compact and pretty output remain byte-for-byte identical to the corresponding serde_json helpers.

`CollectingObserver`¶

Thread-safe observer that accumulates all violations for later inspection.

Method	Returns
`new()`	Empty collector
`violations()`	`Vec<SpecViolation>` — all collected violations
`len()`	Number of violations
`is_empty()`	No violations collected?
`has_violation(kind)`	Any violation of this kind?
`has_severity(severity)`	Any violation at this severity?
`violations_of_kind(kind)`	Filtered by kind
`violations_at_severity(min)`	Filtered by minimum severity
`clear()`	Remove all collected violations

`TracingObserver`¶

Default observer that maps severity levels to tracing log levels: Warning → tracing::warn!, Error/Critical → tracing::error!. All fields are emitted as structured tracing fields.

Plugging In¶

Rust

use fortress_rollback::telemetry::CollectingObserver;
use fortress_rollback::SessionBuilder;
use std::sync::Arc;

let observer = Arc::new(CollectingObserver::new());
// MyConfig: your Config impl (see user-guide.md)
let builder = SessionBuilder::<MyConfig>::new()
    .with_violation_observer(observer.clone());

// After session operations
assert!(observer.violations().is_empty(), "unexpected violations");

`ViolationKind` Variants¶

Variant	Description
`FrameSync`	Frame synchronization invariant violated
`InputQueue`	Input queue invariant violated
`StateManagement`	State save/load invariant violated
`NetworkProtocol`	Network protocol invariant violated
`ChecksumMismatch`	Checksum or desync detection issue
`Configuration`	Configuration constraint violated
`InternalError`	Internal logic error (library bug)
`Invariant`	Runtime invariant check failed
`Synchronization`	Sync protocol issues
`ArithmeticOverflow`	Arithmetic overflow detected

`ViolationSeverity` Levels¶

Level	Meaning
`Warning`	Unexpected but recoverable -- operation continued with fallback
`Error`	Serious issue -- operation may have degraded behavior
`Critical`	Critical invariant broken -- state may be corrupted

Event Flow¶

sequenceDiagram
    participant Game as Game Loop
    participant Session as P2PSession
    participant Telemetry as SessionTelemetry

    Game->>Session: advance_frame()
    Session->>Telemetry: on_frame_advance(frame)

    Note over Session: Remote input arrives late
    Session->>Telemetry: on_prediction_miss(player, frame)
    Session->>Telemetry: on_rollback(depth, target_frame)

    Note over Session: Re-simulate frames
    loop For each re-simulated frame
        Session->>Telemetry: on_frame_advance(frame)
    end

    Game->>Session: poll_remote_clients()
    loop For each running remote endpoint
        Session->>Telemetry: on_network_stats(player, stats)
    end
    Note over Session: The public network_stats(player) getter<br/>returns stats to the caller and does not<br/>invoke the telemetry observer.

Use Cases¶

Performance monitoring -- Track rollback frequency and prediction accuracy over time
Network quality dashboards -- Aggregate NetworkStatsUpdate events per peer
Automated testing assertions -- Use CollectingTelemetry to assert rollback counts, prediction accuracy
Debug overlays -- Display rollback count, ping, and frame advantage in a HUD

Integration Tips¶

Performance

Keep observer callbacks fast -- they run inline during frame processing. Offload heavy work (file I/O, network sends) to a background thread.

Tip

Use Arc<CollectingTelemetry> for testing, a custom SessionTelemetry impl for production.

Safety

Both SessionTelemetry and ViolationObserver require Send + Sync when the sync-send feature is enabled. The sync-send feature is not a default feature and must be explicitly opted into. All built-in implementations are thread-safe regardless of feature flags.

Session Telemetry¶

Table of Contents¶

Architecture¶

Quick Start¶

SessionTelemetry Trait¶

TelemetryEvent Enum¶

CollectingTelemetry (Built-in)¶

Custom Telemetry Observer¶

Pull-Based Metrics¶

Spec Violation Observability¶

ViolationObserver Trait¶

SpecViolation Struct¶

CollectingObserver¶

TracingObserver¶

Plugging In¶

ViolationKind Variants¶

ViolationSeverity Levels¶

Event Flow¶

Use Cases¶

Integration Tips¶

See Also¶

`SessionTelemetry` Trait¶

`TelemetryEvent` Enum¶

`CollectingTelemetry` (Built-in)¶

`ViolationObserver` Trait¶

`SpecViolation` Struct¶

`CollectingObserver`¶

`TracingObserver`¶

`ViolationKind` Variants¶

`ViolationSeverity` Levels¶