Skip to content

Remove duplicate database code in provider logic #277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
pcallewaert wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Remove duplicate database code in provider logic #277

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 95 additions & 37 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Manage external PostgreSQL databases in Kubernetes with ease—supporting AWS RD
- [Features](#features)
- [Supported Cloud Providers](#supported-cloud-providers)
- [Configuration](#configuration)
- [Dedicated Operator Role](#dedicated-operator-role)
- [Installation](#installation)
- [Custom Resources (CRs)](#custom-resources-crs)
- [Multiple Operator Support](#multiple-operator-support)
Expand Down Expand Up @@ -66,27 +67,83 @@ Set `POSTGRES_CLOUD_PROVIDER` to `AWS` via environment variable, Kubernetes Secr

Set environment variables in [`config/manager/operator.yaml`](config/manager/operator.yaml):

| Name | Description | Default |
| --- | --- | --- |
| `WATCH_NAMESPACE` | Namespace to watch. Empty string = all namespaces. | (all namespaces) |
| `POSTGRES_INSTANCE` | Operator identity for multi-instance deployments. | (empty) |
| `KEEP_SECRET_NAME` | Use user-provided secret names instead of auto-generated ones. | disabled |
| Name | Description | Default |
| ------------------- | -------------------------------------------------------------- | ---------------- |
| `WATCH_NAMESPACE` | Namespace to watch. Empty string = all namespaces. | (all namespaces) |
| `POSTGRES_INSTANCE` | Operator identity for multi-instance deployments. | (empty) |
| `KEEP_SECRET_NAME` | Use user-provided secret names instead of auto-generated ones. | disabled |

> **Note:**
> If enabling `KEEP_SECRET_NAME`, ensure there are no secret name conflicts in your namespace to avoid reconcile loops.

## Dedicated Operator Role

The operator connects to PostgreSQL using the credentials configured via the `POSTGRES_*` environment variables / Secret (see below). In many setups these credentials are the _server admin_ or _master user_.

You can also run the operator using a **dedicated operator login role** (recommended for production), for better separation of duties and easier auditing/rotation.

### What privileges are required?

This operator manages databases and roles, and also runs some operations inside the created databases. Your operator login role must be able to:

- Create databases and set database owners (`CREATE DATABASE`, `ALTER DATABASE ... OWNER TO ...`)
- Grant database-level privileges (the operator runs `GRANT CREATE ON DATABASE ...`)
- Create roles/users and manage role membership (`CREATE ROLE`, `DROP ROLE`, `GRANT <role> TO <grantee>`, `REVOKE ...`)
- Connect to managed databases and:
- Create schemas (`CREATE SCHEMA ... AUTHORIZATION ...`)
- Create extensions (`CREATE EXTENSION ...`)
- Grant privileges / alter default privileges within schemas

The operator also grants each created role to itself, so it can later revoke privileges, reassign ownership, and drop roles cleanly.

### Example: creating an operator role

The exact SQL depends on how your PostgreSQL instance is managed. In plain PostgreSQL (self-hosted), you can often do something like:

```sql
-- Create a dedicated login for the Kubernetes operator
CREATE ROLE pgoperator WITH
PASSWORD 'YourSecurePassword123!'
LOGIN
CREATEDB
CREATEROLE;
```

For managed services, you typically create `ext_postgres_operator` while connected as the platform-provided admin and grant only the capabilities supported by that platform.

### Cloud provider notes

Because this is an _external / managed PostgreSQL_ operator, the feasibility of least-privilege depends on your provider.

- **AWS RDS (PostgreSQL)**
- The initial “master user” is a member of the `rds_superuser` role.
- A dedicated operator role is usually possible: create a login role with `CREATEDB`/`CREATEROLE`, then grant it any extra permissions you need for extensions/schemas.
- Docs: <https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.Roles.html>

- **GCP Cloud SQL (PostgreSQL)**
- Cloud SQL does not expose true `SUPERUSER`. The default `postgres` user is a member of `cloudsqlsuperuser` and has `CREATEROLE` and `CREATEDB`.
- You can create other users/roles with reduced privileges (for example, an operator role with `CREATEROLE`/`CREATEDB`), but some operations (notably certain extensions) may require `cloudsqlsuperuser`.
- Docs: <https://cloud.google.com/sql/docs/postgres/users>

- **Azure Database for PostgreSQL  Flexible Server**
- The server admin user is a member of `azure_pg_admin` and has `CREATEDB` and `CREATEROLE`; the `azuresu` superuser role is reserved for Microsoft.
- A dedicated operator role is supported: create a user with `CREATEDB`/`CREATEROLE`, optionally add it to `azure_pg_admin` if you need additional capabilities.
- Docs: <https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/security-manage-database-users>

## Installation

### Install Using Helm (Recommended)

The Helm chart for this operator is located in the `charts/ext-postgres-operator` subdirectory. Follow these steps to install:

1. Add the Helm repository:

```bash
helm repo add ext-postgres-operator https://movetokube.github.io/postgres-operator/
```

2. Install the operator:

```bash
helm install -n operators ext-postgres-operator ext-postgres-operator/ext-postgres-operator
```
Expand Down Expand Up @@ -121,11 +178,13 @@ To install the operator using Kustomize, follow these steps:
1. Configure Postgres credentials for the operator in `config/default/secret.yaml`.

2. Deploy the operator:

```bash
kubectl kustomize config/default/ | kubectl apply -f -
```

Alternatively, use [Kustomize](https://github.com/kubernetes-sigs/kustomize) directly:

```bash
kustomize build config/default/ | kubectl apply -f -
```
Expand All @@ -149,11 +208,11 @@ spec:
dropOnDelete: false # Set to true if you want the operator to drop the database and role when this CR is deleted (optional)
masterRole: test-db-group (optional)
schemas: # List of schemas the operator should create in database (optional)
- stores
- customers
- stores
- customers
extensions: # List of extensions that should be created in the database (optional)
- fuzzystrmatch
- pgcrypto
- fuzzystrmatch
- pgcrypto
```

This creates a database called `test-db` and a role `test-db-group` that is set as the owner of the database.
Expand All @@ -173,14 +232,14 @@ metadata:
postgres.db.movetokube.com/instance: POSTGRES_INSTANCE
spec:
role: username
database: my-db # This references the Postgres CR
database: my-db # This references the Postgres CR
secretName: my-secret
privileges: OWNER # Can be OWNER/READ/WRITE
annotations: # Annotations to be propagated to the secrets metadata section (optional)
privileges: OWNER # Can be OWNER/READ/WRITE
annotations: # Annotations to be propagated to the secrets metadata section (optional)
foo: "bar"
labels:
foo: "bar" # Labels to be propagated to the secrets metadata section (optional)
secretTemplate: # Output secrets can be customized using standard Go templates
foo: "bar" # Labels to be propagated to the secrets metadata section (optional)
secretTemplate: # Output secrets can be customized using standard Go templates
PQ_URL: "host={{.Host}} user={{.Role}} password={{.Password}} dbname={{.Database}}"
```

Expand All @@ -191,22 +250,22 @@ This creates a user role `username-<hash>` and grants role `test-db-group`, `tes
Two `Postgres` referencing the same database can exist in more than one namespace. The last CR referencing a database will drop the group role and transfer database ownership to the role used by the operator.
Every PostgresUser has a generated Kubernetes secret attached to it, which contains the following data (i.e.):

| Key | Comment |
|----------------------|---------------------|
| `DATABASE_NAME` | Name of the database, same as in `Postgres` CR, copied for convenience |
| `HOST` | PostgreSQL server host (including port number) |
| `URI_ARGS` | URI Args, same as in `Postgres` CR, copied for convenience |
| `PASSWORD` | Autogenerated password for user |
| `ROLE` | Autogenerated role with login enabled (user) |
| `LOGIN` | Same as `ROLE`. In case `POSTGRES_CLOUD_PROVIDER` is set to "Azure", `LOGIN` it will be set to `{role}@{serverName}`, serverName is extracted from `POSTGRES_USER` from operator's config. |
| `POSTGRES_URL` | Connection string for Posgres, could be used for Go applications |
| `POSTGRES_JDBC_URL` | JDBC compatible Postgres URI, formatter as `jdbc:postgresql://{POSTGRES_HOST}/{DATABASE_NAME}` |
| `HOSTNAME` | The PostgreSQL server hostname (without port) |
| `PORT` | The PostgreSQL server port |

| Functions | Meaning |
|----------------|-------------------------------------------------------------------|
| `mergeUriArgs` | Merge any provided uri args with any set in the `Postgres` CR |
| Key | Comment |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `DATABASE_NAME` | Name of the database, same as in `Postgres` CR, copied for convenience |
| `HOST` | PostgreSQL server host (including port number) |
| `URI_ARGS` | URI Args, same as in `Postgres` CR, copied for convenience |
| `PASSWORD` | Autogenerated password for user |
| `ROLE` | Autogenerated role with login enabled (user) |
| `LOGIN` | Same as `ROLE`. In case `POSTGRES_CLOUD_PROVIDER` is set to "Azure", `LOGIN` it will be set to `{role}@{serverName}`, serverName is extracted from `POSTGRES_USER` from operator's config. |
| `POSTGRES_URL` | Connection string for Posgres, could be used for Go applications |
| `POSTGRES_JDBC_URL` | JDBC compatible Postgres URI, formatter as `jdbc:postgresql://{POSTGRES_HOST}/{DATABASE_NAME}` |
| `HOSTNAME` | The PostgreSQL server hostname (without port) |
| `PORT` | The PostgreSQL server port |

| Functions | Meaning |
| -------------- | ------------------------------------------------------------- |
| `mergeUriArgs` | Merge any provided uri args with any set in the `Postgres` CR |

### Multiple operator support

Expand All @@ -227,7 +286,7 @@ meeting the specific needs of different applications.
Available context:

| Variable | Meaning |
|-------------|------------------------------|
| ----------- | ---------------------------- |
| `.Host` | Database host |
| `.Role` | Generated user/role name |
| `.Database` | Referenced database name |
Expand All @@ -243,12 +302,11 @@ can be found [here](https://github.com/kubernetes/client-go/blob/master/README.m
Postgres operator compatibility with Operator SDK version is in the table below

| | Operator SDK version | apiextensions.k8s.io |
|---------------------------|----------------------|----------------------|
| `postgres-operator 0.4.x` | v0.17 | v1beta1 |
| `postgres-operator 1.x.x` | v0.18 | v1 |
| `postgres-operator 2.x.x` | v1.39 | v1 |
| `HEAD` | v1.39 | v1 |

| ------------------------- | -------------------- | -------------------- |
| `postgres-operator 0.4.x` | v0.17 | v1beta1 |
| `postgres-operator 1.x.x` | v0.18 | v1 |
| `postgres-operator 2.x.x` | v1.39 | v1 |
| `HEAD` | v1.39 | v1 |

## Contributing

Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (

require (
cel.dev/expr v0.24.0 // indirect
github.com/DATA-DOG/go-sqlmock v1.5.2 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
Expand Down
3 changes: 3 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
cel.dev/expr v0.24.0 h1:56OvJKSH3hDGL0ml5uSxZmz3/3Pq4tJ+fb1unVLAFcY=
cel.dev/expr v0.24.0/go.mod h1:hLPLo1W4QUmuYdA72RBX06QTs6MXw941piREPl3Yfiw=
github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU=
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/antlr4-go/antlr/v4 v4.13.0 h1:lxCg3LAv+EUK6t1i0y1V6/SLeUi0eKEKdhQAlS8TVTI=
Expand Down Expand Up @@ -86,6 +88,7 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kisielk/sqlstruct v0.0.0-20201105191214-5f3e10d3ab46/go.mod h1:yyMNCyc/Ib3bDTKd379tNMpB/7/H5TjM2Y9QJ5THLbE=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
Expand Down
80 changes: 0 additions & 80 deletions pkg/postgres/aws.go
View file
Open in desktop

This file was deleted.

50 changes: 0 additions & 50 deletions pkg/postgres/azure.go
View file
Open in desktop

This file was deleted.

Loading