What matters for your decision
Streaming may improve perceived wait without speeding up a slow integration. Measure representative fast and slow cases and keep long-tail performance and answer quality visible.
Diagnose before changing settings
- Time the retrieval span, first useful answer token, action calls and total completion on matched test prompts.
- Compare typical and slow-percentile cases by workflow; separate read-only FAQs from multi-step actions.
- Test one change at a time, such as a shorter response or removing an unnecessary serial call, and check accuracy before accepting it.