Priority Engineering

Native WebRTC + H.264 Rebuild

Replacing SIPSorcery-based streaming with a native libwebrtc + Media Foundation H.264 pipeline. Maintaining existing C# service architecture (workers, signaling, tunnel).

Verified Working (Current Build)

Native DLL loads next to RemoteIQ.Agent.exe
C# P/Invoke matches native ABI (struct packing)
Native managed callbacks: signal, state, data, log
RemoteConsoleWorker forwards browser offers
Native stub emits synthetic answer SDP
rc.control verified (JSON preview echo)
Smoke test flag: --native-smoke

North Star Goal

Production connectivity across all networks via coturn STUN/TURN infrastructure.

C# WrapperSignaling & UI
Native DLLPC & Encoding

Protocol Specifications

Control Schema decision

RemoteConsoleControlProtocol

  • Standardize on 't' field (e.g. mouse.move)
  • Add browser-side shim if legacy 'type' is detected
DataChannel Semantics
rc.controlJSON Input
rc.inputTerminal stream
rc.logFuture logging
Create /native/RemoteIQ.NativeWebRtc/
Choose build system (CMake/MSBuild)
DLL distribution model confirmed
Finalize 't' vs 'type' alignment
Deliverables
C ABI header (remoteiq_rc.h)
NativeMethods.cs signatures
NativeWebRtcConsolePeer logic
DI injection via PeerFactory
Acceptance
Agent runs without crashing
Offer → Native → Answer flow
DataChannel callbacks arrive
Core Deliverables
Integrate real libwebrtc.dll
PeerConnectionFactory creation
Apply ICE servers from JSON config
Handle offer → create real answer
Add ICE candidates callback loop
Open DataChannels (rc.input)
Success Criteria
  • • Real WebRTC (DTLS/SRTP) connection confirmed
  • • Terminal roundtrip: browser → PowerShell → browser
Deploy coturn cluster
Open UDP/TCP 3478, TLS 5349 ports
Backend TURN REST HMAC credentials
Log candidate pair type logging

"Without TURN we’ll only ever have 'works on some networks'. Runs parallel with Phase 2."

Phase 3Video Pipeline

Capture + Encode + Track (H.264).

Phase 4Control Plane

Bitrate, Keyframes, Resolution logic.

Phase 5Cleanup

Remove SIPSorcery completely.

Phase 6Hardening

Full Telemetry + Resilience.

Definition of Done

Reliable connect across networks (TURN)
H.264 renders at stable FPS/Latency
rc.control reliably injects inputs
Terminal tunnel works flawlessly
Reconnect does not require agent restart
Zero SIPSorcery references remain
No memory leaks after sessions