Skip to content

Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud #2696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
odidev wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud #2696

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
---
title: Deploy ClickHouse on Google Cloud C4A Arm virtual machines
title: Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud Axion (Arm-based C4A VMs)

minutes_to_complete: 30
minutes_to_complete: 50

who_is_this_for: This is an introductory topic for developers deploying and optimizing ClickHouse on Arm-based Linux environments using Google Cloud C4A virtual machines powered by Axion processors, to evaluate ClickHouse performance and behaviour on Arm-based infrastructure.
who_is_this_for: This learning path is intended for software developers, data engineers, and platform engineers who want to build and benchmark a real-time analytics pipeline using ClickHouse on Linux/Arm64 environments, specifically Google Cloud C4A virtual machines powered by Axion processors.

learning_objectives:
- Provision an Arm-based SUSE SLES virtual machine on Google Cloud using C4A instances powered by Axion processors
- Install and start a ClickHouse server on a SUSE Arm64 (C4A) virtual machine
- Verify ClickHouse functionality by connecting to the server and running basic insert and query operations
- Run baseline ClickHouse performance tests to produce throughput and query latency results for evaluating Arm-based deployments on Google Cloud
- Provision an Arm-based SUSE SLES virtual machine on Google Cloud using C4A (Axion processors)
- Configure Google Cloud Pub/Sub for real-time log ingestion
- Deploy and validate ClickHouse on a SUSE Linux Arm64 (Axion) VM
- Build a streaming ETL pipeline using Apache Beam and Google Dataflow
- Ingest real-time Pub/Sub data into ClickHouse using Dataflow
- Validate end-to-end data flow from Pub/Sub to ClickHouse
- Perform baseline and analytical query benchmarking on ClickHouse running on Arm64
- Measure and report query latency (including p95) on Axion processors

prerequisites:
- A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled
- Basic familiarity with [ClickHouse](https://clickhouse.com/)
- Basic understanding of databases and SQL

author: Pareena Verma

Expand All @@ -27,7 +32,10 @@ armips:

tools_software_languages:
- ClickHouse
- clickhouse-benchmark
- Apache Beam
- Google Dataflow
- Google Cloud Pub/Sub
- Python 3.11

operatingsystems:
- Linux
Expand Down
3 changes: 2 additions & 1 deletion content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/baseline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
title: Establish a ClickHouse baseline on Arm
weight: 5
weight: 7


### FIXED, DO NOT MODIFY
layout: learningpathall
Expand Down
Loading