-
Notifications
You must be signed in to change notification settings - Fork 12
[Update Deps] Fix Release #418
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
5f26629
d01e82f
6679822
2cba1a9
3be78f7
b29bdba
728827f
b012908
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -49,7 +49,7 @@ jobs: | |
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup Machine for bumping version | ||
| uses: snapchat/gigl/.github/actions/setup-python-tools@main | ||
| uses: ./.github/actions/setup-python-tools | ||
| with: | ||
| setup_gcloud: "true" | ||
| install_dev_deps: "true" | ||
|
|
@@ -61,7 +61,7 @@ jobs: | |
| - name: Get current version | ||
| id: get_version | ||
| run: | | ||
| CURRENT_VERSION=$(python -m scripts.bump_version --get_current_version) | ||
| CURRENT_VERSION=$(uv run python -m scripts.bump_version --get_current_version) | ||
| echo "current_version=$CURRENT_VERSION" >> $GITHUB_OUTPUT | ||
| echo "Current version: $CURRENT_VERSION" | ||
|
|
||
|
|
@@ -70,23 +70,23 @@ jobs: | |
| id: set_vars | ||
| run: | | ||
| if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then | ||
| echo "bump_type=${{ github.event.inputs.bump_type }}" >> $GITHUB_OUTPUT | ||
| echo "bump_type=${{ inputs.bump_type }}" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "bump_type=nightly" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| # Perform the version bump | ||
| - name: Bump version | ||
| run: | | ||
| python -m scripts.bump_version \ | ||
| uv run python -m scripts.bump_version \ | ||
| --bump_type ${{ steps.set_vars.outputs.bump_type }} \ | ||
| --project ${{ vars.GCP_PROJECT_ID }} | ||
|
|
||
| # Capture new version and create release branch name | ||
| - name: Get new version | ||
| id: get_new_version | ||
| run: | | ||
| NEW_VERSION=$(python -m scripts.bump_version --get_current_version) | ||
| NEW_VERSION=$(uv run python -m scripts.bump_version --get_current_version) | ||
| RELEASE_BRANCH_NAME="release/v${NEW_VERSION}" | ||
| echo "new_version=$NEW_VERSION" >> $GITHUB_OUTPUT | ||
| echo "release_branch_name=$RELEASE_BRANCH_NAME" >> $GITHUB_OUTPUT | ||
|
|
@@ -118,7 +118,7 @@ jobs: | |
| ref: ${{ needs.bump_version.outputs.release_branch_name }} | ||
|
|
||
| - name: Setup Machine for release | ||
| uses: snapchat/gigl/.github/actions/setup-python-tools@main | ||
| uses: ./.github/actions/setup-python-tools | ||
| with: | ||
| setup_gcloud: "true" | ||
| install_dev_deps: "false" | ||
|
|
@@ -139,37 +139,38 @@ jobs: | |
| service_account: ${{ secrets.gcp_service_account_email }} | ||
| project: ${{ vars.GCP_PROJECT_ID }} | ||
|
|
||
| release_dev_workbench_image: | ||
| needs: bump_version | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| id-token: write | ||
|
|
||
| steps: | ||
| - name: Checkout release branch | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ needs.bump_version.outputs.release_branch_name }} | ||
|
|
||
| - name: Setup Machine for workbench image release | ||
| uses: snapchat/gigl/.github/actions/setup-python-tools@main | ||
| with: | ||
| setup_gcloud: "true" | ||
| install_dev_deps: "false" | ||
| gcp_project_id: ${{ vars.GCP_PROJECT_ID }} | ||
| workload_identity_provider: ${{ secrets.WORKLOAD_IDENTITY_PROVIDER }} | ||
| gcp_service_account_email: ${{ secrets.GCP_SERVICE_ACCOUNT_EMAIL }} | ||
|
|
||
| - name: Release Dev Workbench Image | ||
| uses: snapchat/gigl/.github/actions/run-cloud-run-command-on-active-checkout@main | ||
| with: | ||
| cmd: | | ||
| bash ./requirements/install_scala_deps.sh --download-only | ||
| make push_dev_workbench_docker_image | ||
| service_account: ${{ secrets.gcp_service_account_email }} | ||
| project: ${{ vars.GCP_PROJECT_ID }} | ||
| # TODO: (svij) Let's deprecate this as no one uses it beyond it being used for the TUT. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why not just delete?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can, if folks are okay with it.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yeah let's remove
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. actually let me do it as a follow up as to completely deprecate this is a wider change.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we choose not to deprecate this - it will induce extra work in fixing this - which I am unsure if valuable. |
||
| # release_dev_workbench_image: | ||
| # needs: bump_version | ||
| # runs-on: ubuntu-latest | ||
| # permissions: | ||
| # contents: write | ||
| # pull-requests: write | ||
| # id-token: write | ||
|
|
||
| # steps: | ||
| # - name: Checkout release branch | ||
| # uses: actions/checkout@v4 | ||
| # with: | ||
| # ref: ${{ needs.bump_version.outputs.release_branch_name }} | ||
|
|
||
| # - name: Setup Machine for workbench image release | ||
| # uses: ./.github/actions/setup-python-tools | ||
| # with: | ||
| # setup_gcloud: "true" | ||
| # install_dev_deps: "false" | ||
| # gcp_project_id: ${{ vars.GCP_PROJECT_ID }} | ||
| # workload_identity_provider: ${{ secrets.WORKLOAD_IDENTITY_PROVIDER }} | ||
| # gcp_service_account_email: ${{ secrets.GCP_SERVICE_ACCOUNT_EMAIL }} | ||
|
|
||
| # - name: Release Dev Workbench Image | ||
| # uses: snapchat/gigl/.github/actions/run-cloud-run-command-on-active-checkout@main | ||
| # with: | ||
| # cmd: | | ||
| # bash ./requirements/install_scala_deps.sh --download-only | ||
| # make push_dev_workbench_docker_image | ||
| # service_account: ${{ secrets.gcp_service_account_email }} | ||
| # project: ${{ vars.GCP_PROJECT_ID }} | ||
|
|
||
|
|
||
| # TODO: (svij) Also release the dev image | ||
|
|
@@ -179,7 +180,7 @@ jobs: | |
| needs: | ||
| - bump_version | ||
| - release_gigl_kfp_pipeline | ||
| - release_dev_workbench_image | ||
| # - release_dev_workbench_image | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can delete here too
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. actually did it as a separate change since its a bit big to delete everything. |
||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: write | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -94,21 +94,17 @@ Below we provide two ways to bootstrap an environment for using and/or developin | |
|
|
||
| ### Install Wheel | ||
|
|
||
| 1. Create a python environment | ||
|
|
||
| ```bash | ||
| conda create -y --override-channels --channel conda-forge --name gigl python=3.9 | ||
| conda activate gigl | ||
| ``` | ||
| 1. Create a python virtual environment w/ `python==3.11.*` | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This doesn't really explain much, if we want people to use UV, should we just say what UV command to run? Aren't "virtual envs" abstracted away from users now?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is public documentation. We are not constrained to people using conda anymore. For dev purposes we can maintain seperate documentation.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd say it's best to provide some recommendation that works for us, e.x.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure, I can do this as a follow up as the best way is a little more involved. |
||
|
|
||
| 2. Install GiGL | ||
|
|
||
| #### Install GiGL + necessary tooling for Torch 2.5 + Cuda12.1 | ||
| #### Install GiGL + necessary tooling for PyG 2.7 + Torch 2.8 on Cuda12.8 | ||
|
|
||
| ```bash | ||
| pip install "gigl[torch25-cuda-121,transform]==0.0.9" \ | ||
| --index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \ | ||
| --extra-index-url=https://pypi.org/simple | ||
| pip install "gigl[pyg27-torch28-cu128, transform]==0.1.0" \ | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
| --extra-index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \ | ||
| --extra-index-url=https://download.pytorch.org/whl/cu128 \ | ||
| --extra-index-url=https://data.pyg.org/whl/torch-2.8.0+cu128.html | ||
| ``` | ||
|
|
||
| Currently, building/using wheels for GLT is error prone, thus we opt to install from source every time. Run post-install | ||
|
|
@@ -118,12 +114,13 @@ script to setup GLT dependency: | |
| gigl-post-install | ||
| ``` | ||
|
|
||
| #### Install GiGL + necessary tooling for Torch 2.5 + CPU | ||
| #### Install GiGL + necessary tooling for PyG 2.7 + Torch 2.8 on CPU | ||
|
|
||
| ```bash | ||
| pip install "gigl[torch25-cpu,transform]==0.0.9" \ | ||
| --index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \ | ||
| --extra-index-url=https://pypi.org/simple | ||
| pip install "gigl[pyg27-torch28-cpu, transform]==0.1.0" \ | ||
| --extra-index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \ | ||
| --extra-index-url=https://download.pytorch.org/whl/cpu \ | ||
| --extra-index-url=https://data.pyg.org/whl/torch-2.8.0+cpu.html | ||
| ``` | ||
|
|
||
| Currently, building/using wheels for GLT is error prone, thus we opt to install from source every time. Run post-install | ||
|
|
@@ -135,20 +132,18 @@ gigl-post-install | |
|
|
||
| ### Install from source | ||
|
|
||
| There are various ways to use GiGL. Firstly, clone the repo locally. | ||
|
|
||
| ```bash | ||
| git clone https://github.com/Snapchat/GiGL.git | ||
| ``` | ||
|
|
||
| From the root directory: | ||
| If you are just using (not developing) GiGL, from the root directory: | ||
|
|
||
| ```bash | ||
| make install_deps | ||
| ``` | ||
|
|
||
| If you *instead* want to contribute and/or extend GiGL. You can install the developer deps which includes some extra | ||
| tooling useful for contributions: | ||
| tooling: | ||
|
|
||
| ```bash | ||
| make install_dev_deps | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,8 +6,10 @@ This document outlines the steps required to add a new Python package as a depen | |
|
|
||
| Open `pyproject.toml` file --> Add the package to its correct location (dev, transform, etc.). Guidance: | ||
|
|
||
| 1. Freeze (set explicit versions e.g. `my-lib==1.2.3`) versions of doc deps as version updates changes how the doc pages are rendered, files are formatted, etc. | ||
| 2. Freeze versions for deps that don't do a great job of maintaining in their wheels what libs / libraries they are compatible with - this is a problem with packages like tensorflow. | ||
| 1. Freeze (set explicit versions e.g. `my-lib==1.2.3`) versions of doc deps as version updates changes how the doc pages | ||
| are rendered, files are formatted, etc. | ||
| 2. Freeze versions for deps that don't do a great job of maintaining in their wheels what libs / libraries they are | ||
| compatible with - this is a problem with packages like tensorflow. | ||
|
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. formatting - realize I forgot to do this in prior PR and merge to main will fail otherwise. |
||
| 3. Everything else, let it auto resolve so we can keep flexibility for GiGL lib users. | ||
|
|
||
| ### 2. Hash Generation | ||
|
|
||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any reason there aren't new releases in this PR as well?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will be done after so we have a clean release of what is in main.
Test releases were done here:
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| __version__ = "0.0.11" | ||
| __version__ = "0.1.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are we planning on reverting these before submitting?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No,
#414 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Once the PR is merged in these changes aren't needed anymore right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
correct, plan is to follow up reverting: