How to configure an AWS and K8S component
AWS is one of the most comprehensive and broadly adopted Cloud platforms providing on-demand cloud computing platforms. It offers over 175 fully-featured services from data centers globally.
Kubernetes is a system for automating the deployment, scaling, and management of containerized applications.
AWS EKS, a fully-managed Kubernetes service, is used with the Kubernetes platform for the AWS environment. EKS is a certified Kubernetes conformant, leveraging all benefits of open source Kubernetes tooling from the community and integrated with AWS services for a scalable and highly available control plane that runs across multiple availability zones to eliminate a single point of failure.
Use snapblocs to provision Data Platforms on AWS following AWS Well-Architected guidelines for provisioning and configuring production-grade Kubernetes clusters and deploying workloads into the clusters. snapblocs provides benefits from patterns that worked for many customers who have gone to production quickly and easily.
Once creating a stack that has an AWS and K8S component, you can customize the AWS and K8S components with the following settings.
Provider Key Name
Use the Provider Access Key to deploy a stack to a specific AWS account.
Provider Access Keys created at the account level are available for use by all projects and their stacks. In essence, they have shared keys.
Stacks can only use Provider Access Keys created at the project level within that project.
Stacks can use a key from their project or an account key.
Delete Provider Access Keys when not in use by stacks.
VPC
Amazon Virtual Private Cloud (Amazon VPC) is a logically isolated section of the AWS Cloud where snapblocs platforms, with all needed AWS resources as a single unit in a defined virtual network, are deployed.
Each AWS account comes with a default VPC and default Subnets in each AWS Region. A default VPC is ready to use. There is no need to create and configure a custom VPC.
Use a default VPC to get started quickly or a prototype and launch public instances such as a blog or simple website.
However, as the default VPC is not ideal for a secure production system, a custom VPC is required.
Kubernetes API server Endpoint Access
Amazon EKS creates an endpoint for the managed Kubernetes API server to communicate with your cluster (Kubernetes management tools such as kubectl). By default, this API server endpoint is public to the internet, and access to the API server is secured using a combination of AWS Identity and Access Management (IAM) and native Kubernetes Role-Based Access Control (RBAC). The private-only access option is currently not supported for now.
Kubernetes Configuration
A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications. The worker node(s) host the Pods that are the components of the application workload.
Enter the desired number of nodes to scale to when creating the stack.
Enter the minimum number of nodes that the node Auto Scaling group can scale into when creating the stack.
Enter the maximum number of nodes that the node Auto Scaling group can scale out to when creating the stack.
Min Size <= Desired Capacity <= Max Size
To enable secure access to the worker nodes, use EC2 Key Pair.
Execute a custom bootstrap script stored on the S3 object when launching worker nodes to customize the Kubernetes worker nodes for installing required libraries, etc.
Reference
1.1-JPE
Related Articles
How to configure an AWS and K8S for DaaS component
Amazon Web Services (AWS) AWS is one of the most comprehensive and broadly adopted cloud platforms providing on-demand cloud computing platforms. It offers over 175 fully-featured services from data centers globally. Kubernetes is a system for ...
How to configure a Kafka component
Kafka is a distributed streaming platform used to publish and subscribe to streams of records. Kafka gets used for fault-tolerant storage. Kafka replicates topic log partitions to multiple servers. Use it to stream data to other data platforms such ...
Configure Components
Before proceeding, click here to configure a stack. Each component must be configured before deploying the stack. As the most of the components come with reasonable default values, Complete the following configuration settings: AWS and K8S component ...
How to add other IAM users to provide access to AWS EKS
After deploying a stack, if someone tries to access AWS EKS using kubectl, (s)he will get the following authorization error: "error: You must be logged in to the server (Unauthorized)." This happens when the Amazon EKS cluster is created by an ...
How to configure a Dremio component
Dremio is a data lake engine that liberates big data with live, interactive queries directly on cloud data lake storage. Dremio delivers secure, self-service data access and lightning-fast queries directly on cloud providers AWS, GCP, or Azure. In ...