What is StreamSets DC+ Platform

What is StreamSets DC+ Platform

StreamSets Data Collector+ Platform provides data ingestion pipelines integrated with ETL processing for streaming, batch, and change data capture (CDC). This allows a user to use snapblocs dashboard UI to create a data ingestion pipeline stack by configuring the data ingestion options such as input source and target destination settings and operational configuration for target runtime environment such as Kubernetes on Cloud platform (AWS, etc.). Therefore, you can focus your business applications without managing StreamSets DC clusters.

snapblocs dashboard allows a user to manage the lifecycle of StreamSets Data Collector stack (start, update, terminate) and monitor the runtime behavior of the stack through build-in Observability features that measure how well data are moving how internal states of the components are healthy. Also, it allows you to perform overall administrative tasks like scaling up the cluster to increase its capacity.



StreamSets DC+ Platform includes the following software components:
  1. AWS EKS is used to provision a StreamSets Data Collector+ Platform stacks using the customer's AWS account.
    Google GKE is used to provision a StreamSets Data Collector+ Platform stacks using the customer's Google account.
    snapblocs provisions a StreamSets Data Collector+ following the well-architecture guides (i.e., AWS Well-Architected for AWS, Google Cloud Architecture Framework, etc.) for provisioning and configuring production-grade Kubernetes, and deploying workloads into the clusters. It provides benefits from patterns that worked for many customers who have gone to production. Also, snapblocs make it easy to get started and easy to do the right thing. 
  2. StreamSets Data Collector is a low-latency ingest infrastructure tool that lets you create continuous data ingest pipelines using a drag and drop UI within an integrated development environment (IDE). It is used to ingest source data in-stream or batch to other data platforms such as Data Lake or different on-prem or cloud datacenter.
  3. Elastic Stack is used to provide observability (monitoring, alerting, APM) for answering questions about what's happening inside the system just by observing the outside of the system.
  4. Grafana is open-source visualization and analytics software. It allows you to query, visualize, and explore your metrics. It also allows you to set an alert for quickly identifying problems in your system moments so that you can minimize disruption to your services.