-
Notifications
You must be signed in to change notification settings - Fork 1k
Python: fix(ag-ui): Execute tools with approval_mode, fix shared state, code cleanup #3079
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes a bug where tools with approval_mode="always_require" were not executed after user approval. The issue occurred because approval payloads ({"accepted": true}) were converted to generic TextContent instead of FunctionApprovalResponseContent, preventing the agent framework from executing the approved tools.
Key Changes
- Modified approval payload handling to create
FunctionApprovalResponseContentobjects that the framework can execute - Added logic to filter out stale tool results when re-executing approved tools
- Updated message hygiene to recognize and properly handle approval responses
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 12 comments.
| File | Description |
|---|---|
python/packages/ag-ui/tests/test_agent_wrapper_comprehensive.py |
Added integration tests for approval and rejection flows to verify tools execute correctly |
python/packages/ag-ui/agent_framework_ag_ui/_orchestration/_message_hygiene.py |
Added logic to detect FunctionApprovalResponseContent and skip synthetic result injection for approved calls |
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py |
Modified to convert approval payloads to FunctionApprovalResponseContent and remove stale tool results |
Comments suppressed due to low confidence (2)
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py:125
- The new approval payload handling logic (lines 69-125) that converts tool results with
{"accepted": true/false}toFunctionApprovalResponseContentlacks direct unit test coverage intest_message_adapters.py. While integration tests exist intest_agent_wrapper_comprehensive.py, consider adding a focused unit test that verifies this conversion logic in isolation, including edge cases like:
- Approval payload with matching function call
- Approval payload without matching function call (confirm_changes fallback)
- Rejection payload (
{"accepted": false}) - Approval payload with existing tool result that gets filtered out
if is_approval:
# Look for the matching function call in previous messages to create
# a proper FunctionApprovalResponseContent. This enables the agent framework
# to execute the approved tool (fix for GitHub issue #3034).
parsed_approval = json.loads(result_content)
accepted = parsed_approval.get("accepted", False)
# Find the function call that matches this tool_call_id
matching_func_call = None
for prev_msg in result:
role_val = prev_msg.role.value if hasattr(prev_msg.role, "value") else str(prev_msg.role)
if role_val != "assistant":
continue
for content in prev_msg.contents or []:
if isinstance(content, FunctionCallContent):
if content.call_id == tool_call_id and content.name != "confirm_changes":
matching_func_call = content
break
if matching_func_call:
# Remove any existing tool result for this call_id since the framework
# will re-execute the tool after approval. Keeping old results causes
# OpenAI API errors ("tool message must follow assistant with tool_calls").
result = [
m
for m in result
if not (
(m.role.value if hasattr(m.role, "value") else str(m.role)) == "tool"
and any(
isinstance(c, FunctionResultContent) and c.call_id == tool_call_id
for c in (m.contents or [])
)
)
]
# Create FunctionApprovalResponseContent for the agent framework
approval_response = FunctionApprovalResponseContent(
approved=accepted,
id=str(tool_call_id),
function_call=matching_func_call,
)
chat_msg = ChatMessage(
role=Role.USER,
contents=[approval_response],
)
else:
# No matching function call found - this is likely a confirm_changes approval
# Keep the old behavior for backwards compatibility
chat_msg = ChatMessage(
role=Role.USER,
contents=[TextContent(text=str(result_content))],
additional_properties={"is_tool_result": True, "tool_call_id": str(tool_call_id or "")},
)
if "id" in msg:
chat_msg.message_id = msg["id"]
result.append(chat_msg)
continue
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py:287
- This import of module json is redundant, as it was previously imported on line 5.
import json
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py
Outdated
Show resolved
Hide resolved
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py
Outdated
Show resolved
Hide resolved
python/packages/ag-ui/agent_framework_ag_ui/_orchestration/_message_hygiene.py
Outdated
Show resolved
Hide resolved
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py
Outdated
Show resolved
Hide resolved
python/packages/ag-ui/agent_framework_ag_ui/_orchestration/_message_hygiene.py
Outdated
Show resolved
Hide resolved
eavanvalkenburg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One nit, otherwise looks good.
Motivation and Context
Tools with
approval_mode="always_require"were not executed after user approval. Instead, the response showed "Changes confirmed and applied successfully!" without actually invoking the tool.When AG-UI received an approval payload (
{"accepted": true}), it was:TextContentwithis_tool_resultflag, routed toHumanInTheLoopOrchestratorconfirm_changes(predictive state) approvals, not real function approvalssanitize_tool_historyinjected synthetic "skipped" results, causing model API errorsThe fixes are in:
_message_adapters.py: Convert approval payloads toFunctionApprovalResponseContent(which the agent framework knows how to execute). Remove any existing tool results for the call_id since the framework will re-execute._message_hygiene.py: RecognizeFunctionApprovalResponseContentand skip synthetic result injection for those call_ids.Additionally, for 3010:
Shared state wasn't properly handled when updated via the UI.
Description
Code Cleanup
Refactored the
agent_framework_ag_uipackage to reduce duplication and improve maintainability:New modules under
_orchestration/:_helpers.py: Extracted orchestration helper functions (message selection, tool call tracking, approval handling)_predictive_state.py: Consolidated predictive state logic intoPredictiveStateHandlerclassImprovements:
_utils.py:safe_json_parse,get_role_value, role constantsAnytypes with proper SDK types (ChatMessage,Contents) for better type safetyConfirmationStrategyproperly abstract with@abstractmethoddecorators on properties_orchestrators.pyAll existing tests pass (251 tests).
Contribution Checklist