Skip to content
This repository was archived by the owner on Apr 23, 2025. It is now read-only.

Docs: update testing workarounds #23

Merged
ipv1337 merged 1 commit into from
Apr 15, 2025
Merged

Docs: update testing workarounds #23

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion docs/TESTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,4 +170,17 @@ Recent work with the Google Generative AI (Gemini) API highlighted several key l

3. **Regular Updates**: Update tests when APIs change, focusing on the behavior rather than the exact implementation.

4. **Error Handling**: Include proper error handling in tests to make them more robust against changes.
4. **Error Handling**: Include proper error handling in tests to make them more robust against changes.

### Known Test Workarounds

1. **Gemini Agent Loop Issues**: The Gemini agent loop has limitations in handling sequences of tool calls.
- Several tests in `tests/models/test_gemini.py` have modified assertions to accommodate these limitations:
- `test_generate_simple_tool_call` has commented-out assertions for the second tool execution (`mock_task_complete_tool.execute`) and final result check.
- History count assertions are adjusted to reflect actual behavior rather than ideal behavior.
- When writing new tests that involve sequential tool calls, be aware of these limitations and adjust assertions accordingly.
- If you're improving the agent loop functionality, consult `TODO_gemini_loop.md` for details on remaining issues.
Comment on lines +177 to +182
Copy link

@gemini-code-assist gemini-code-assist bot Apr 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Consider adding a link to the TODO_gemini_loop.md file directly in the sentence. This would allow developers to quickly navigate to the file and understand the remaining issues. Also, consider adding a brief summary of the issues mentioned in the TODO file, to provide context without requiring the reader to open another file.


2. **Mock API Response Structure**: Some tests may have extra or adjusted mock structures to handle the model's specific response processing.
- Look for comments like `# Mock response adapted for agent loop` to identify these cases.
- When updating these tests, ensure you maintain the adjusted structure until the underlying issues are resolved.