After configuring a new stack of Data Flow Platform by following this, you can customize the stack.
Test / Proof of Concept (POC) Stack
To create a simple test Data Flow stack on cloud prividers, set the following parameters.
AWS and K8S Component:
Provider Key Name: Choose the AWS Provider Access name which is assigned to you. See this for creating an AWS Provider Access method.
Regions: Choose the AWS region where you want to deploy the stack.
VPC: Choose VPC where you want to deploy the stack into
Subnets: Choose at least 2 public and at least 1 private Subnets.
Desired Capacity: 6 nodes
Maximum Size: 6 nodes
Minimum Size: 6 nodes
Instance Type: t2.large
On GCP & K8S component:
Provider Key Name: Choose the cloud provider key name which is assigned to you. See here on creating a Provider Key Name. Location Type: Choose between Regional and Zonal (Regions are independent geographic areas that consist of zones. A zone is a deployment area for Google Cloud resources within a region. Zones should be considered a single failure domain within a region.)
Regions: Choose the GCP region where you want to deploy the stack.
Node Location: Choose the Zone where nodes will be located. Recommend to select 3 or more zones for the high availability.
VPC Network: Choose the VPC Network where you want to deploy the Data Platform.
VPC Sub Network: Choose the VPC Sub Network where you want to deploy the Data Platform.
Desired Capacity: 5 nodes
Maximum Size: 5 nodes
Minimum Size: 5 nodes
Instance Type: t2.large
Other Components:
Production Stack
[TBD] - To create a scalable and highly available Data Flow stack, set each component's settings accordingly.
What's Next?
After customizing all components, you can deploy the stack following this.