Fix: Improve coverage reporting in SonarCloud #15
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hello @ipv1337, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
This pull request addresses the issue of inaccurate coverage reporting in SonarCloud. The changes include modifications to the .coveragerc file to improve coverage pattern matching, updates to the run_coverage_ci.sh script to use --cov-append for accumulating coverage data, adjustments to sonar-project.properties to include the reference branch and Jacoco coverage paths, and the removal of the pr_description.md file. The goal is to ensure that SonarCloud accurately reflects the actual code coverage.
Highlights
- Coverage Reporting: Fixes coverage reporting issues in SonarCloud, ensuring accurate coverage metrics.
- Configuration Updates: Enhances
.coveragercwith better patterns for including source files and excludesollama.pyto match SonarCloud exclusions. - Script Modification: Updates
run_coverage_ci.shto use--cov-appendfor coverage data accumulation and setsCI_EXIT_ON_TEST_FAILURE=1to enforce code quality. - Sonar Properties: Updates
sonar-project.propertiesto include the reference branch and Jacoco coverage paths.
Changelog
- .coveragerc
- Modified the
includesection to use broader patterns for including Python files, specifically adding*/src/cli_code/*.py,*/src/cli_code/*/*.py, and*/src/cli_code/*/*/*.pyon lines 4-6. - Added an exclusion for
src/cli_code/models/ollama.pyto match the exclusion insonar-project.propertieson line 17.
- Modified the
- pr_description.md
- Removed the entire
pr_description.mdfile.
- Removed the entire
- scripts/run_coverage_ci.sh
- Changed
CI_EXIT_ON_TEST_FAILUREfrom0to1on line 13 to ensure the script exits on test failures. - Added the
--cov-appendflag to thepytestcommand on line 260 to accumulate coverage data across multiple test runs.
- Changed
- sonar-project.properties
- Added
sonar.newCode.referenceBranch=origin/mainon line 21 to force SonarCloud to see all files as new code for proper coverage metrics.
- Added
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
A script's changed decree,
Coverage clear for all to see,
Sonar's eye now bright,
Catching errors in the night,
Code quality takes flight.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This PR aims to improve coverage reporting in SonarCloud by addressing issues with coverage calculation and configuration. The changes include modifications to the coverage script, coverage configuration file, and SonarCloud properties. Overall, the changes seem reasonable and address the stated problem. However, there are a few areas that could be improved.
Summary of Findings
- Inconsistent use of CI_EXIT_ON_TEST_FAILURE: The script initially sets CI_EXIT_ON_TEST_FAILURE to 0 and then changes it to 1. This inconsistency might lead to unexpected behavior during CI runs.
- Coverage exclusions: The .coveragerc and sonar-project.properties files both exclude
src/cli_code/models/ollama.py. It's important to ensure that this exclusion is intentional and justified.
Merge Readiness
The pull request is not yet ready for merging. The inconsistent use of CI_EXIT_ON_TEST_FAILURE and the coverage exclusions should be addressed before merging. I am unable to approve this pull request, and recommend that others review and approve this code before merging. There are no critical or high severity issues, but the medium severity issues should be addressed before merging.
.coveragerc
Outdated
| */venv/* | ||
| */.pytest_cache/* | ||
| */site-packages/* | ||
| */src/cli_code/models/ollama.py # Match the exclusion in sonar-project.properties |
There was a problem hiding this comment.
This line excludes src/cli_code/models/ollama.py from coverage. It's good that you're matching the exclusion in sonar-project.properties, but is it intentional to exclude this file from coverage? If so, please add a comment explaining why.
*/src/cli_code/models/ollama.py # Match the exclusion in sonar-project.properties. Excluding due to...
| # Force SonarCloud to see all files as new code to get proper coverage metrics | ||
| sonar.newCode.referenceBranch=origin/main |
There was a problem hiding this comment.
Adding sonar.newCode.referenceBranch=origin/main forces SonarCloud to see all files as new code. While this might be useful for initial coverage analysis, it could skew future coverage reports. Consider removing this line once the initial coverage baseline is established, or add a comment explaining the long-term implications.
# Force SonarCloud to see all files as new code to get proper coverage metrics
# This line should be removed after the initial coverage baseline is established
sonar.newCode.referenceBranch=origin/main
Code Coverage Report📊 Current Coverage: 38.35% Detailed coverage analysis is available in SonarCloud Coverage Change DetailsThis shows code coverage for changes in this PR. To improve coverage, consider adding tests for new or modified code. |
|
This PR addresses the ongoing coverage reporting issues in SonarCloud where coverage is being reported as 0.9% instead of the actual coverage. Changes include fixing run_coverage_ci.sh to use --cov-append, enhancing .coveragerc with better patterns, updating sonar properties with reference branch, and adding jacoco coverage paths.