Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 39 additions & 38 deletions .github/workflows/create_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup Machine for bumping version
uses: snapchat/gigl/.github/actions/setup-python-tools@main
uses: ./.github/actions/setup-python-tools
with:
setup_gcloud: "true"
install_dev_deps: "true"
Expand All @@ -61,7 +61,7 @@ jobs:
- name: Get current version
id: get_version
run: |
CURRENT_VERSION=$(python -m scripts.bump_version --get_current_version)
CURRENT_VERSION=$(uv run python -m scripts.bump_version --get_current_version)
echo "current_version=$CURRENT_VERSION" >> $GITHUB_OUTPUT
echo "Current version: $CURRENT_VERSION"

Expand All @@ -70,23 +70,23 @@ jobs:
id: set_vars
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
echo "bump_type=${{ github.event.inputs.bump_type }}" >> $GITHUB_OUTPUT
echo "bump_type=${{ inputs.bump_type }}" >> $GITHUB_OUTPUT
else
echo "bump_type=nightly" >> $GITHUB_OUTPUT
fi

# Perform the version bump
- name: Bump version
run: |
python -m scripts.bump_version \
uv run python -m scripts.bump_version \
--bump_type ${{ steps.set_vars.outputs.bump_type }} \
--project ${{ vars.GCP_PROJECT_ID }}

# Capture new version and create release branch name
- name: Get new version
id: get_new_version
run: |
NEW_VERSION=$(python -m scripts.bump_version --get_current_version)
NEW_VERSION=$(uv run python -m scripts.bump_version --get_current_version)
RELEASE_BRANCH_NAME="release/v${NEW_VERSION}"
echo "new_version=$NEW_VERSION" >> $GITHUB_OUTPUT
echo "release_branch_name=$RELEASE_BRANCH_NAME" >> $GITHUB_OUTPUT
Expand Down Expand Up @@ -118,7 +118,7 @@ jobs:
ref: ${{ needs.bump_version.outputs.release_branch_name }}

- name: Setup Machine for release
uses: snapchat/gigl/.github/actions/setup-python-tools@main
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we planning on reverting these before submitting?

Copy link
Collaborator Author

@svij-sc svij-sc Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No,

#414 (comment)

Post Merge:
Things that will need to be done after PR is merged in

  • Github action workflows comment - Cannot test till merged.
  • Post PR merge we will need to update back ./.github/actions/setup-python-tools --> snapchat/gigl/.github/actions/setup-python-tools@main
  • Formal release of gigl v0.1.0

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the PR is merged in these changes aren't needed anymore right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, plan is to follow up reverting:

Post PR merge we will need to update back ./.github/actions/setup-python-tools --> snapchat/gigl/.github/actions/setup-python-tools@main

uses: ./.github/actions/setup-python-tools
with:
setup_gcloud: "true"
install_dev_deps: "false"
Expand All @@ -139,37 +139,38 @@ jobs:
service_account: ${{ secrets.gcp_service_account_email }}
project: ${{ vars.GCP_PROJECT_ID }}

release_dev_workbench_image:
needs: bump_version
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
id-token: write

steps:
- name: Checkout release branch
uses: actions/checkout@v4
with:
ref: ${{ needs.bump_version.outputs.release_branch_name }}

- name: Setup Machine for workbench image release
uses: snapchat/gigl/.github/actions/setup-python-tools@main
with:
setup_gcloud: "true"
install_dev_deps: "false"
gcp_project_id: ${{ vars.GCP_PROJECT_ID }}
workload_identity_provider: ${{ secrets.WORKLOAD_IDENTITY_PROVIDER }}
gcp_service_account_email: ${{ secrets.GCP_SERVICE_ACCOUNT_EMAIL }}

- name: Release Dev Workbench Image
uses: snapchat/gigl/.github/actions/run-cloud-run-command-on-active-checkout@main
with:
cmd: |
bash ./requirements/install_scala_deps.sh --download-only
make push_dev_workbench_docker_image
service_account: ${{ secrets.gcp_service_account_email }}
project: ${{ vars.GCP_PROJECT_ID }}
# TODO: (svij) Let's deprecate this as no one uses it beyond it being used for the TUT.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just delete?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can, if folks are okay with it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah let's remove

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually let me do it as a follow up as to completely deprecate this is a wider change.
Commenting here prevents failures but removing all references is a 5+ file change reserved for a seperate PR.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we choose not to deprecate this - it will induce extra work in fixing this - which I am unsure if valuable.

# release_dev_workbench_image:
# needs: bump_version
# runs-on: ubuntu-latest
# permissions:
# contents: write
# pull-requests: write
# id-token: write

# steps:
# - name: Checkout release branch
# uses: actions/checkout@v4
# with:
# ref: ${{ needs.bump_version.outputs.release_branch_name }}

# - name: Setup Machine for workbench image release
# uses: ./.github/actions/setup-python-tools
# with:
# setup_gcloud: "true"
# install_dev_deps: "false"
# gcp_project_id: ${{ vars.GCP_PROJECT_ID }}
# workload_identity_provider: ${{ secrets.WORKLOAD_IDENTITY_PROVIDER }}
# gcp_service_account_email: ${{ secrets.GCP_SERVICE_ACCOUNT_EMAIL }}

# - name: Release Dev Workbench Image
# uses: snapchat/gigl/.github/actions/run-cloud-run-command-on-active-checkout@main
# with:
# cmd: |
# bash ./requirements/install_scala_deps.sh --download-only
# make push_dev_workbench_docker_image
# service_account: ${{ secrets.gcp_service_account_email }}
# project: ${{ vars.GCP_PROJECT_ID }}


# TODO: (svij) Also release the dev image
Expand All @@ -179,7 +180,7 @@ jobs:
needs:
- bump_version
- release_gigl_kfp_pipeline
- release_dev_workbench_image
# - release_dev_workbench_image
Copy link
Collaborator

@yliu2-sc yliu2-sc Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can delete here too

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually did it as a separate change since its a bit big to delete everything.

runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/nightly_release_&_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ permissions:
# Calls create_release w/ nightly and then run the api tests
jobs:
create-release:
uses: snapchat/gigl/.github/workflows/create_release.yml@main
uses: ./.github/workflows/create_release.yml
secrets: inherit
with:
bump_type: nightly
Expand All @@ -28,14 +28,14 @@ jobs:
with:
ref: ${{ needs.create-release.outputs.release_branch_name }}
- name: Setup development environment
uses: snapchat/gigl/.github/actions/setup-python-tools@main
uses: ./.github/actions/setup-python-tools
with:
setup_gcloud: "true"
gcp_project_id: ${{ vars.GCP_PROJECT_ID }}
workload_identity_provider: ${{ secrets.workload_identity_provider }}
gcp_service_account_email: ${{ secrets.gcp_service_account_email }}
- name: Run API Tests
uses: snapchat/gigl/.github/actions/run-cloud-run-command-on-active-checkout@main
uses: ./.github/actions/run-cloud-run-command-on-active-checkout
with:
cmd: "make run_api_test"
service_account: ${{ secrets.gcp_service_account_email }}
Expand Down
15 changes: 5 additions & 10 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
uses: actions/checkout@v4

- name: Setup Python deps and gcloud
uses: snapchat/gigl/.github/actions/setup-python-tools@main
uses: ./.github/actions/setup-python-tools
with:
install_dev_deps: "true"
setup_gcloud: "true"
Expand All @@ -37,24 +37,19 @@ jobs:
# And: https://cloud.google.com/artifact-registry/docs/python/authentication
- name: Setup environment for publishing Python package
run: |
python -m pip install --upgrade build==1.3.0 twine==6.2.0 keyrings.google-artifactregistry-auth==1.1.2
# Pre-install keyring and Artifact Registry plugin from the public PyPI
uv tool install keyring --with keyrings.google-artifactregistry-auth==1.1.2

- name: Build Whl Distribution
run: (cd python && python -m build)
run: uv build

- name: Publish Package 🚀
env:
PYPIRC_CONTENTS: ${{ secrets.PYPIRC_CONTENTS }}
PIP_CONF_CONTENTS: ${{ secrets.PIP_CONF_CONTENTS }}
# We upload the build whls to Google Artifact Registry.
run: |
echo "$PYPIRC_CONTENTS" > ~/.pypirc
mkdir -p ~/.pip
echo "$PIP_CONF_CONTENTS" > ~/.pip/pip.conf
gcloud config set artifacts/location us-central1
gcloud config set artifacts/repository gigl
python -m twine check python/dist/*
python -m twine upload --repository-url https://us-central1-python.pkg.dev/${PROJECT_ID}/gigl python/dist/*
uv publish --index gcp-release-registry --username oauth2accesstoken --keyring-provider subprocess

- name: Post Publish Package
if: always()
Expand Down
31 changes: 13 additions & 18 deletions docs/user_guide/getting_started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,21 +94,17 @@ Below we provide two ways to bootstrap an environment for using and/or developin

### Install Wheel

1. Create a python environment

```bash
conda create -y --override-channels --channel conda-forge --name gigl python=3.9
conda activate gigl
```
1. Create a python virtual environment w/ `python==3.11.*`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't really explain much, if we want people to use UV, should we just say what UV command to run?

Aren't "virtual envs" abstracted away from users now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is public documentation. We are not constrained to people using conda anymore.
They should be able to use whatever they want to generate their python virtual env i.e. virtualenv, conda, uv, etc.

For dev purposes we can maintain seperate documentation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say it's best to provide some recommendation that works for us, e.x. uv it'll make getting started easier

Copy link
Collaborator Author

@svij-sc svij-sc Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can do this as a follow up as the best way is a little more involved.
Specifically, I think providing a template pyproject.toml for uv will be more helpful.

#422


2. Install GiGL

#### Install GiGL + necessary tooling for Torch 2.5 + Cuda12.1
#### Install GiGL + necessary tooling for PyG 2.7 + Torch 2.8 on Cuda12.8

```bash
pip install "gigl[torch25-cuda-121,transform]==0.0.9" \
--index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \
--extra-index-url=https://pypi.org/simple
pip install "gigl[pyg27-torch28-cu128, transform]==0.1.0" \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pip uv now right? ditto later

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--extra-index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \
--extra-index-url=https://download.pytorch.org/whl/cu128 \
--extra-index-url=https://data.pyg.org/whl/torch-2.8.0+cu128.html
```

Currently, building/using wheels for GLT is error prone, thus we opt to install from source every time. Run post-install
Expand All @@ -118,12 +114,13 @@ script to setup GLT dependency:
gigl-post-install
```

#### Install GiGL + necessary tooling for Torch 2.5 + CPU
#### Install GiGL + necessary tooling for PyG 2.7 + Torch 2.8 on CPU

```bash
pip install "gigl[torch25-cpu,transform]==0.0.9" \
--index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \
--extra-index-url=https://pypi.org/simple
pip install "gigl[pyg27-torch28-cpu, transform]==0.1.0" \
--extra-index-url=https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/ \
--extra-index-url=https://download.pytorch.org/whl/cpu \
--extra-index-url=https://data.pyg.org/whl/torch-2.8.0+cpu.html
```

Currently, building/using wheels for GLT is error prone, thus we opt to install from source every time. Run post-install
Expand All @@ -135,20 +132,18 @@ gigl-post-install

### Install from source

There are various ways to use GiGL. Firstly, clone the repo locally.

```bash
git clone https://github.com/Snapchat/GiGL.git
```

From the root directory:
If you are just using (not developing) GiGL, from the root directory:

```bash
make install_deps
```

If you *instead* want to contribute and/or extend GiGL. You can install the developer deps which includes some extra
tooling useful for contributions:
tooling:

```bash
make install_dev_deps
Expand Down
6 changes: 4 additions & 2 deletions docs/user_guide/trouble_shooting/adding_new_package.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ This document outlines the steps required to add a new Python package as a depen

Open `pyproject.toml` file --> Add the package to its correct location (dev, transform, etc.). Guidance:

1. Freeze (set explicit versions e.g. `my-lib==1.2.3`) versions of doc deps as version updates changes how the doc pages are rendered, files are formatted, etc.
2. Freeze versions for deps that don't do a great job of maintaining in their wheels what libs / libraries they are compatible with - this is a problem with packages like tensorflow.
1. Freeze (set explicit versions e.g. `my-lib==1.2.3`) versions of doc deps as version updates changes how the doc pages
are rendered, files are formatted, etc.
2. Freeze versions for deps that don't do a great job of maintaining in their wheels what libs / libraries they are
compatible with - this is a problem with packages like tensorflow.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formatting - realize I forgot to do this in prior PR and merge to main will fail otherwise.

3. Everything else, let it auto resolve so we can keep flexibility for GiGL lib users.

### 2. Hash Generation
Expand Down
7 changes: 7 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,13 @@ url = "https://data.pyg.org/whl/torch-2.8.0+cu128.html"
explicit = true
format = "flat"

# =============================== Google Artifact Registry Index ========================
[[tool.uv.index]]
name = "gcp-release-registry"
url = "https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl/simple/"
publish-url = "https://us-central1-python.pkg.dev/external-snap-ci-github-gigl/gigl"
explicit = true

# ========== Mapping individual packages to their appropriate index =============
[tool.uv.sources]
# ============= PyTorch hosted Package Index Mappings
Expand Down
2 changes: 1 addition & 1 deletion python/gigl/__init__.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason there aren't new releases in this PR as well?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be done after so we have a clean release of what is in main.
#414 (comment)

Post Merge:
Things that will need to be done after PR is merged in

  • Github action workflows comment - Cannot test till merged.
  • Post PR merge we will need to update back ./.github/actions/setup-python-tools --> snapchat/gigl/.github/actions/setup-python-tools@main
  • Formal release of gigl v0.1.0

Test releases were done here:
#418 (comment)

Testing Plan:

Ran nightly release + api test here: https://github.com/Snapchat/GiGL/actions/runs/20198180696/job/57985597587
Released an actual wheel and subsequently installed it locally to validate it works here: https://github.com/Snapchat/GiGL/actions/runs/20198990587/job/57987110578

Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.0.11"
__version__ = "0.1.0"