Back to changelog/Edge runtime improvements
Jan 10, 2025v2025.01improvededgeperformance

Edge runtime improvements

Faster cold starts and better telemetry for edge deployments.

Edge runtime improvements

We made the edge runtime lighter and more observable.

Performance

  • Cold start time reduced by 23% on Xavier-class devices.
  • Streaming now defaults to chunked responses for chat completions.

Reliability

  • Added backpressure handling on low-bandwidth networks.
  • Improved retry heuristics for transient storage failures.

Tooling

  • unieai deploy --edge now surfaces GPU temperature and throttling warnings inline.