Skip to content

Added high level architecture diagram for ice-rest-catalog with etcd … #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
subkanthi wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Added high level architecture diagram for ice-rest-catalog with etcd … #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fa3d75a
Added high level architecture diagram for ice-rest-catalog with etcd …
subkanthi Dec 1, 2025
a7af569
updated architecture.md
subkanthi Dec 1, 2025
4516e55
Added k8s manifest and documentation on how to replace minio with s3.
subkanthi Dec 2, 2025
e8abb76
Removed service and linked documentation to the examples/eks folder.
subkanthi Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions doc/.ice.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
uri: http://localhost:8181
s3:
endpoint: http://localhost:9000
pathStyleAccess: true
accessKeyID: minio
secretAccessKey: minio123
region: us-east-1
47 changes: 47 additions & 0 deletions doc/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# ICE REST Catalog Architecture

![ICE REST Catalog Architecture](ice-rest-catalog-architecture.drawio.png)

## Components

- **ice-rest-catalog**: Stateless REST API service (Kubernetes Deployment)
- **etcd**: Distributed key-value store for catalog state (Kubernetes StatefulSet)
- **Object Storage**: S3-compatible storage for data files
- **Clients**: ClickHouse or other Iceberg-compatible engines

## Design Principles

### Stateless Catalog

The `ice-rest-catalog` is completely stateless and deployed as a Kubernetes Deployment with multiple replicas.
It can be scaled horizontally without coordination. The catalog does not store any state locally—all metadata is persisted in etcd.

### State Management

All catalog state (namespaces, tables, schemas, snapshots, etc.) is maintained in **etcd**, a distributed, consistent key-value store.
Each etcd instance runs as a StatefulSet pod with persistent storage, ensuring data durability across restarts.

### Service Discovery

`ice-rest-catalog` uses the k8s service to access the cluster.
The catalog uses jetcd library to interact with etcd https://github.com/etcd-io/jetcd.
In the etcd cluster, the data is replicated in all the nodes of the cluster.
The service provides a round-robin approach to access the nodes in the cluster.

### High Availability

- Multiple `ice-rest-catalog` replicas behind a load balancer
- etcd cluster.
- Persistent volumes for etcd data
- S3 for durable object storage


### k8s Manifest Files

Kubernetes deployment manifests and configuration files are available in the [`examples/eks`](../examples/eks/) folder:

- [`etcd.eks.yaml`](../examples/eks/etcd.eks.yaml) - etcd StatefulSet deployment
- [`ice-rest-catalog.eks.envsubst.yaml`](../examples/eks/ice-rest-catalog.eks.envsubst.yaml) - ice-rest-catalog Deployment (requires envsubst)
- [`eks.envsubst.yaml`](../examples/eks/eks.envsubst.yaml) - Combined EKS deployment template

See the [EKS README](../examples/eks/README.md) for detailed setup instructions.
Loading