Choose one of two options to integrate StreamSets Data Collector with the Data Platform Stack.
Create a new StreamSets Data Collector cluster within the Data Platform Stack
Integrate externally created StreamSets Data Collector cluster with Data Platform Stack
(Option 1) Create a new StreamSets Data Collector cluster within the Data Platform (recommended option)
Choose this option to create a new StreamSets Data Collector cluster managed by the Data Platform Kubernetes infrastructure and integrate seamlessly with other Data Platform components without affecting external applications outside the Data Platform.
Istio is an open platform for providing a uniform way to integrate microservices, manage traffic flow across microservices, enforce policies, and aggregate telemetry data. Istio's control plane provides an abstraction layer over the underlying cluster management platform, such as Kubernetes. Istio reduces the complexity of managing microservice deployments by providing a uniform way to secure, connect, and monitor microservices.
Istio is composed of these components:
Envoy - Sidecar proxies per microservice to handle ingress/egress traffic between services in the cluster and from a service to external services. The proxies form a secure microservice mesh providing a rich set of functions like discovery, rich layer-7 routing, circuit breakers, policy enforcement and telemetry recording/reporting functions.
Note: The service mesh is not an overlay network. It simplifies and enhances how microservices in an application talk to each other over the network provided by the underlying platform.
Istiod - The Istio control plane. It provides service discovery, configuration and certificate management. It consists of the following sub-components:
Pilot - Responsible for configuring the proxies at runtime.
Citadel - Responsible for certificate issuance and rotation.
Galley - Responsible for validating, ingesting, aggregating, transforming and distributing config within Istio.
Operator - The component provides user friendly options to operate the Istio service mesh.
Istio Settings:
Istio Version: Select the Istio version for the stack from the currently Supporting Version.
(Option 2) Integrate externally created StreamSets Data Collector cluster with Data Platform Stack
Choose this option if the StreamSets Data Collector cluster is external and the StreamSets Data Collector cluster is within the Data Platform. Be aware that any changes and usages that occur while running the Data Platform may impact external applications (systems) that depend on this StreamSets Data Collector cluster.