Internal Data Flow

This document describes the interaction paths, communication protocols, and the basic flow of data among the components of SynxDB Cloud.

SynxDB Cloud is a cloud-native data warehouse built on the Kubernetes (K8s) platform. Its architecture decouples computing resources (query execution) from data storage (persistence), allowing each to be scaled independently and on-demand. All core business components run as containerized Pods in the Kubernetes cluster, where their resource scheduling and lifecycle are managed by Kubernetes.

Data Flow Diagram

The following sections explain the interaction paths, communication protocols, and basic data flow among the components, as illustrated in the diagram above.

Platform user management flow

  • Platform users send requests to the dbaas-integration component via HTTPS.

  • The dbaas-integration component interacts with the Coordinator component via TCP.

  • The dbaas-integration component can also interact with an optional external PostgreSQL database via TCP.

Database user query flow

The system provides two query paths for database users:

Query flow via the Web UI

  1. A user’s HTTP request first reaches the cloudbeaver component.

  2. cloudbeaver forwards the request to the coordinator-proxy component via HTTP.

  3. coordinator-proxy then passes the request to the Coordinator component via TCP.

Query flow via a direct client connection

  • Database users or applications use the PostgreSQL Frontend/Backend Protocol (based on TCP) to connect directly to the Coordinator component for data transfer.

Internal query execution and data exchange flow

  1. After receiving a query, the Coordinator component communicates with one or more Segment components in a Warehouse via libpq/TCP to distribute tasks.

  2. Within the same Warehouse, Segment components exchange data directly with each other using Interconnect (based on UDP or TCP).

  3. The Coordinator component interacts with components inside the UnionStore Cluster via TCP.

Data persistence and read flow

  • Components within the UnionStore Cluster (pageserver, safekeepers) communicate with each other through an internal protocol.

  • UnionStore Cluster and Warehouse segments interact with an external OSS (Object Storage Service) via HTTP(S) to perform final data reads and writes.