-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Adding-Tests-Quoting-issue [Issue # 40996][WIP] #41982
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Hi @w-javed. Thank you for your interest in helping to improve the Azure SDK experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment. Otherwise, we'll close this out in 7 days. |
|
Hi @w-javed. Thank you for your contribution. Since there hasn't been recent engagement, we're going to close this out. Feel free to respond with a comment containing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds a test to verify that the evaluation framework correctly handles quoted strings in JSON input data. The test validates that quotes within JSON string values (e.g., "\"test\"") are preserved correctly when passed to evaluators, addressing issue #40996.
Key Changes:
- Added a new test evaluator class (
QuotationFixEval) that compares input strings - Added test data file with quoted string values
- Added a test case to validate quote preservation behavior
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| sdk/evaluation/azure-ai-evaluation/tests/unittests/test_evaluators/test_inputs_evaluators.py | Adds QuotationFixEval class for testing string comparison with quoted values |
| sdk/evaluation/azure-ai-evaluation/tests/unittests/test_evaluate.py | Adds test fixture and test method to validate quote preservation in evaluation inputs |
| sdk/evaluation/azure-ai-evaluation/tests/unittests/data/quotation_fix_test_data.jsonl | Adds test data with quoted strings to verify quote handling |
PR for #40996