One agent, too many kinds of tickets
Quote from Bormind on August 19, 2026, 7:13 pmOur internal IT agent handles everything from password resets to debugging integration failures. Every request currently goes through the same model, temperature, prompt, and tool set, which makes simple tickets absurdly expensive and still doesn’t give the harder ones enough attention. We can score each run based on whether the ticket was resolved, reopened, or handed to a person, but we have no good way to turn those results into better choices for the next request. A few rules can’t be bent: contractors can’t trigger admin tools, web search is blocked for billing tickets, and code can run only in the sandbox. How are teams tuning the configuration for each request while making sure the system never tries a combination that breaks those rules?
Our internal IT agent handles everything from password resets to debugging integration failures. Every request currently goes through the same model, temperature, prompt, and tool set, which makes simple tickets absurdly expensive and still doesn’t give the harder ones enough attention. We can score each run based on whether the ticket was resolved, reopened, or handed to a person, but we have no good way to turn those results into better choices for the next request. A few rules can’t be bent: contractors can’t trigger admin tools, web search is blocked for billing tickets, and code can run only in the sandbox. How are teams tuning the configuration for each request while making sure the system never tries a combination that breaks those rules?
Quote from Palonius on August 19, 2026, 7:25 pmRight now you’re asking one configuration to be cheap, capable, and safe for every ticket, which it can’t do. The security rules also shouldn’t be part of the same experiment as model quality. Put a hard policy layer in front of the router, then let the router choose only from configurations already allowed for that user and ticket type. Start with three ticket groups, replay closed cases against a few configurations, and keep the ones that lower cost while reopen and handoff rates stay flat.
Right now you’re asking one configuration to be cheap, capable, and safe for every ticket, which it can’t do. The security rules also shouldn’t be part of the same experiment as model quality. Put a hard policy layer in front of the router, then let the router choose only from configurations already allowed for that user and ticket type. Start with three ticket groups, replay closed cases against a few configurations, and keep the ones that lower cost while reopen and handoff rates stay flat.
Quote from Starver on August 19, 2026, 7:28 pmFixed routing rules will get you started, but they’ll soon turn into a pile of exceptions as ticket types and model prices change. You already collect the feedback an optimizer needs through resolutions, reopened tickets, handoffs, and cost. Use those results for agent optimization while declaring the forbidden model and tool combinations as hard constraints here: https://eignex.com/ . Begin with model choice, temperature, and tool access, then add prompt style after the results settle down. Keep a fixed set of past tickets for comparison so a cheaper configuration doesn’t quietly make the difficult cases worse.
Fixed routing rules will get you started, but they’ll soon turn into a pile of exceptions as ticket types and model prices change. You already collect the feedback an optimizer needs through resolutions, reopened tickets, handoffs, and cost. Use those results for agent optimization while declaring the forbidden model and tool combinations as hard constraints here: https://eignex.com/ . Begin with model choice, temperature, and tool access, then add prompt style after the results settle down. Keep a fixed set of past tickets for comparison so a cheaper configuration doesn’t quietly make the difficult cases worse.