[serve] deflake backpressure test

[kamil-kaczmarek]

This pull request moves wrap_queued_request up one level to route_and_send_request in the router.py. It is to ensure that the num_queued_requests remains incremented across all routing retries, preventing oscillation when a replica rejects a request.

Details:

• num_queued_requests now reflects the requests this router is actively trying to place.
• In route_and_send_request, the wrap_queued_request context now wraps the entire retry loop, so the num_queued_requests stays incremented for the full duration of all retries instead of being reset on each retry.
• argument resolution must not be counted as queue time -> _resolve_request_arguments (along with the _objref_resolution_latency_ms observation and ActorDiedError → upstream_crash_error handling) is now called in route_and_send_request before the wrap_queued_request context is opened.

[gemini-code-assist]

Code Review

This pull request moves the wrap_queued_request context manager to wrap the entire retry loop in route_and_send_request to prevent metric oscillation during replica rejections. However, a review comment points out that this change introduces a regression in metrics accuracy because the counter now increments before request arguments are resolved. The reviewer suggests moving the argument resolution logic before the context manager to ensure that queue metrics accurately reflect pending work and do not include time spent waiting for upstream dependencies.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1c26d69. Configure here.}