Make a slow/lagging cluster node measurable, recognizable and reported (per-peer lag/heartbeat health + alert + metrics + Studio) #4812

Component: ha-raft (ClusterMonitor, GetClusterHandler, ClusterAlerts, RaftHAServer) + server metrics + Studio cluster tab
Type: enhancement

Summary

In a cluster, when one node is constantly slower than the others it silently throttles replication and eventually triggers leader churn / backpressure, but today this is hard to see: the per-replica lag/status the leader already computes (ClusterMonitor) is only written to the server log (the ASCII cluster table), not exposed in the GET /api/v1/cluster JSON that Studio consumes, and there is no structured alert that names the slow node. Operators have to grep logs to find the culprit.

This issue tracks making a slow/lagging follower measurable, recognizable and reported across the stack.

Background (what already exists)

• ClusterMonitor runs every 5s on the leader and classifies each replica as HEALTHY / CATCHING_UP / FALLING_BEHIND / STALLED, with throttled log warnings and a stall-streak for leader-driven resync (Replica stuck in STALLED #4728).
• RaftHAServer.getFollowerStates() exposes per-peer matchIndex, nextIndex, and lastRpcElapsedMs (heartbeat latency, the leading indicator of imminent election churn).
• Aggregate HA gauges arcadedb.ha.follower.max_last_contact_ms / max_replication_lag exist but collapse to a max, losing which node is slow.

Improvements

1. Sustained-lag tracking (ClusterMonitor): record per-replica laggingSince (set when over the warning threshold, cleared when healthy) and expose laggingForMs, so operators see persistence ("lagging for 45s") and not just a point-in-time snapshot.
2. Per-peer health in the cluster JSON (GetClusterHandler, GET /api/v1/cluster): enrich each follower peer (leader-only) with matchIndex, nextIndex, replicationLag, replicaStatus, lagging, lastContactMs, laggingForMs. The Studio cluster JS already references replicaStatus/matchIndex but never receives them today.
3. Lagging-follower alert (ClusterAlerts): a structured alert that names the slow node - warning for FALLING_BEHIND, critical for STALLED - including lag and lagging duration, with concrete remediation (check that node's CPU/disk/GC/network; reduce batch size; raise election timeouts).
4. Per-peer Prometheus gauges: arcadedb.ha.follower.last_contact_ms{peer=...} and arcadedb.ha.follower.replication_lag{peer=...} (via Micrometer MultiGauge, refreshed on membership change) so Grafana can alert on a specific node, complementing the aggregate-max gauges.
5. Studio cluster tab: render the per-peer lag number, heartbeat latency and lagging duration, and surface the lagging-follower alert banner.

Outcome

A constantly-slow node is visible at a glance in Studio (badge + lag/latency/duration), alertable in Grafana per node, and called out by a named cluster alert with remediation - instead of only being discoverable by reading logs.