Skip to content

Conversation

@firewave
Copy link
Collaborator

This will prevent duplicated workflows from being invoked for pushes into pull requests made from branches in the root (danmar) repository branches.

@firewave
Copy link
Collaborator Author

With these changes branches in your repo will no longer trigger CI builds if you were leveraging that.

It is possible there is a different approach but I didn't find any. I was thinking about checking if the PR branch is the same as the push branch and then dropping the push invocation but I am not sure if that is actually possible.

@danmar
Copy link
Owner

danmar commented Sep 1, 2023

@firewave Related question..
I would like to speed up our CI. I believe we can pay for extra speed. Do you know what we can do?
There is for instance the https://buildjet.com/for-github-actions does that seem interesting? Would you care to take a look?

@firewave
Copy link
Collaborator Author

firewave commented Sep 1, 2023

@firewave Related question.. I would like to speed up our CI. I believe we can pay for extra speed. Do you know what we can do? There is for instance the https://buildjet.com/for-github-actions does that seem interesting? Would you care to take a look?

I don't think we need to pay for speed - it would mainly help with the build times but those aren't our bottleneck (ccache helped a lot there), it's a lot of other things. The CI is considerably fast - a PR can be filed, reviewed and merged within 30 minutes.

There's several jobs we can shed and things still to speed up:

  • qmake builds are on their way out
  • Qt 5.x is EOL and I am in the midst of switching to Qt6 and we can unsupported/deprecate it with leaving the code in but not actively testing it, removing it down the road - fixed #12042 - deprecated building with Qt5 / build with Qt6 in CI / updated latest Qt in CI to 6.5.3 #4957
  • MinGW is very slow because of of the setup and linking - I have been working on that as well - CI-mingw.yml: use pre-installed MinGW / also use lld and ccache for faster builds #4400
  • I would like to deprecate/remove support for the very old compilers and move to the minimum compilers for C++14 (not using it yet) - that would also get rid of some jobs
  • clang-tidy is the slowest job but that is on it. There have been big improvements upstream but I need to collect timing information again to report that and see what is causing that.
  • we can also speed up some of the runs which already use optimized by using LTO/-O3 - also working on that
  • builds were extremely fast around 1.75 and I think if we can figure out which change caused that and refactor a bit we might also get a speed-up. I think this is caused by all the STL containers in Library which are instantiated for the compilation of each TU
  • splitting up big files could also help - see ValueFlow: start splitting it into multiple files #4748
  • not invoking ValueFlow if not necessary will also speed up tests - added (optional) lazy execution of ValueFlow #4521
  • Windows builds do debug and release builds - we are not doing that for any other platform (because it has totally separate configurations in the project) - wanted to address that by code sharing and having the release job run on main only. The two different builds would also be unnecessary if we would dump the solution from the repo and just use it from CMake as there's no risk of some breaking). We could split it into separate steps though - CI-windows.yml: perform debug and release builds in separate jobs #5392. It might also make sense to drop the 32-bit builds as we don't do them for any other platform - fixed #11910 - removed Visual Studio x86 targets from project files and builds from CI #5397
  • testruns are slow because we are using debug builds - we can probably use some minor optimization level to have compilation time unaffected and save on those runs
  • there's also a bit more room for ccache (just a little though) - see use ccache for MSBuild builds #4709 / CI-mingw.yml: use pre-installed MinGW / also use lld and ccache for faster builds #4400
  • also implementing automatic cancelling existing jobs if you push again would help (currently it's lots of manual work to do that) - unfortunately that does not work as necessary. I filed a support request about that (https://github.com/orgs/community/discussions/31040) but haven't gotten a reply yet (don't have the link handy).
  • find a way to trigger the longest running builds first - might not be possible though
  • move up the compilation of the long-running files. The slowest compiling files are the very end of the list because of the alphabet. I wanted to introduce a ordered list to improve that but that was shot down because it would get rid of the globbing in CMake (although we do the same for the tests). - fixed #11925 (build slowest files first) #5419
  • we could also bump the minimum required compiler and CMake version to get rid of some very outdated platforms in the CI - bumped minimum requirements to GCC 5.1 / Clang 3.5 / Visual Studio 2015 / CMake 3.5 #5398
  • use Clang instead of GCC for shorter build times. That might not help much in case the changes are mostly covered by ccache
  • evaluate if ninja improves a full build. Seems unlikely and would also not help in case we are covered by the cache
  • drop testing of Python versions which are EOL (from 2.7 and up to 3.7 at the time of writing). Do not actively remove support (e.g. doing batch changes which move to newer features) for them but no longer test them and treat as as "untested and unmaintained" allowing the scripts to mostly still work - see https://devguide.python.org/versions/
  • use Boost for more builds

MinGW (#4400), Cygwin and clang-tidy are the big outliers.

I will try to file tickets about those.

@firewave firewave marked this pull request as ready for review September 1, 2023 16:22
@danmar danmar merged commit bf5e633 into danmar:main Sep 1, 2023
@firewave firewave deleted the ci-push-pr branch September 1, 2023 16:53
@firewave
Copy link
Collaborator Author

firewave commented Sep 1, 2023

Thanks. This might just be temporarily. I will monitor it and check if things are actually working as expected. I will also do some tests over the next days and try to find a more targeted approach. But give it a few days as I am still not well and will be out a bit over the weekend.

@chrchr-github
Copy link
Collaborator

Starting around this time frame, actions are no longer triggered if I push changes to a branch of my fork. Is anyone else seeing that?

@firewave
Copy link
Collaborator Author

firewave commented Sep 6, 2023

Starting around this time frame, actions are no longer triggered if I push changes to a branch of my fork. Is anyone else seeing that?

Yes, that is what this does. I was fearing people were using that in their forks (I don't)...

I was looking for a more targeted solution but it does not seems possible. I am quite baffled by that as this behavior would render repos with very long-running CI jobs totally unusable (not to speak about the extra costs we would encounter if you would pay for the runners).

@chrchr-github
Copy link
Collaborator

Is it possible to trigger the actions manually? Or do I need to create a PR just to see if the CI passes?

@firewave
Copy link
Collaborator Author

firewave commented Sep 6, 2023

Is it possible to trigger the actions manually? Or do I need to create a PR just to see if the CI passes?

Only individually (also a thing I don't understand - which is a pain if you want to test things manually or need to cancel them) and if they are configured to (workflow_dispatch).

The only way is to revert the change. That would be fine with me.

I made this change because if someone works on the main repo or files a PR from there the CI is basically clogged up for hours because each individual push as well as the pull request itself will trigger the workflows. Which is very disruptive to say the least but no one's fault.

I was looking into a different approach which would detect in the push that it is a pull request but I am very doubtful that will actually work at all.

@chrchr-github
Copy link
Collaborator

How do I trigger an action for a specific branch of mine?

danmar added a commit that referenced this pull request Sep 6, 2023
…from root repository branches (#5377)"

This reverts commit bf5e633.
@chrchr-github
Copy link
Collaborator

My fork is up to date, but still no actions after the revert: https://github.com/chrchr-github/cppcheck/actions

@firewave
Copy link
Collaborator Author

firewave commented Sep 8, 2023

This was only reverted in the 2.12.x branch and not in main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants