-
Notifications
You must be signed in to change notification settings - Fork 569
feat: add per_turn parameter to SlidingWindowConversationManager #1374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: add per_turn parameter to SlidingWindowConversationManager #1374
Conversation
Allow conversation managers to act as hook providers and add an option to built-in conversation managers to proactively apply message management during the agent loop execution. Use that functionality to add an option to SlidingWindowConversationManager to allow per_turn management application Fixes strands-agents#509
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
src/strands/agent/conversation_manager/sliding_window_conversation_manager.py
Outdated
Show resolved
Hide resolved
|
|
||
| # Validate per_turn parameter | ||
| # Note: Must check bool before int since bool is a subclass of int in Python | ||
| if not isinstance(per_turn, bool) and isinstance(per_turn, int) and per_turn <= 0: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In our SDK I don't think we validate input parameters
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we against it or just that this is contrary to other places?
Personally I don't see a problem with enforcing it for new things, but can remove it if we want
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While validation wouldn't hurt, we intentionally avoid it. @pgrayy mentioned it will make SDK cumbersome, and yes, if we do it here, we can do else where.
Long term plan is to allow every ConversationManager to add hooks as needed
| self.hooks.add_hook(self._session_manager) | ||
|
|
||
| # Allow conversation_managers to subscribe to hooks | ||
| self.hooks.add_hook(self.conversation_manager) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
session manager uses conversation manager, right? Should we put conversation_manager hooks before it in case there is something the conversation manager needs to do earlier?
Also what are you thinking for documentation? It is a bit unclear now that conversation manager is now both called manually and via hooks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, what is the story for transitioning from "manual" conversation management, meaning conversation_manager.apply_management, to using only hooks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we put conversation_manager hooks before it in case there is something the conversation manager needs to do earlier?
I actually think we want to do it after, in case the conversation_manager wants to trim after the session-manager
Also what are you thinking for documentation?
I will have to do a docs PR for this; specifically I need to update it to account for the per_turn parameter AND conversation managers being able to register for hooks.
Also, what is the story for transitioning from "manual" conversation management, meaning conversation_manager.apply_management, to using only hooks.
I'll cut an issue - though IMHO it's low priority
Description
Allow conversation managers to act as hook providers and add an option to built-in conversation managers to proactively apply message management during the agent loop execution.
Use that functionality to add an option to SlidingWindowConversationManager to allow per_turn management application
Addresses #509
Public API Changes
Default
per_turn=Falsemaintains current behaviorParameter Options
per_turn=False(default): Current behavior - management only at endper_turn=True: Apply management before every model callper_turn=N(int): Apply management before every N model callsConversationManager as HookProviders
ConversationManagernow provides aregister_hooks()method that subclasses can override to integrate with the agent's event lifecycle and at initialization time, it will have it's hook subscribed to.SlidingWindowConversationManagerleverages this underneath the hood.Added
@runtime_checkabletoHookProviderprotocol to enable proper isinstance checks.Before:
After:
We guide folks to call
super().register_hooks(registry, **kwargs)because in the future I'd like to re-implement the default conversation-management behavior (where apply_management and reduce_context is called) using hooks - if we do that and folks aren't callingsuper().register_hooks(registry, **kwargs)then once we migrate to hooks then derived classes that do not callsuper().register_hookswould lose the default behavior which would be a breaking change.Related Issues
#509
Type of Change
New feature
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.