The Importance of Container Orchestration

Containers have revolutionized how we distribute applications by allowing replicated test environments, portability, resource efficiency, scalability and unmatched isolation capabilities. While containers help us package applications for easier deployment and updating, we need a set of specialized tools to manage them.

To help with this, orchestration tools provide the framework through which we automate containerized workloads. Such tools help DevOps teams to manage the lifecycle of containers, and therefore implement their networking, load balancing, provisioning, scaling and more. As a result, orchestration tools help teams unlock the full benefits of containerization by offering application resilience, improved security and simplified operations.

Tasks performed using container orchestration tools include:

• Allocating resources among containers
• Scaling containers up and down based on workloads
• Routing traffic and balancing loads
• Assigning services and applications to specific containers
• Deployment and Provisioning

In this article, let us find the some of the popular container orchestration tools that an organization can take use of.

List of Top Container Orchestration Tools

1. Kubernetes

Kubernetes was developed by Google in 2008 and handed over to the Cloud Native Computing Foundation in 2014. As one of the most popular open-source container orchestration tool, Kubernetes offers a wide array of benefits, including auto-scaling and automated load balancing.

The Kubernetes framework consists of four main components:

• Node - In Kubernetes, a node is is responsible to run containerized workloads, and could either be physical or virtual. These machines serve as hosts for container runtimes, and also facilitate communication between containers and the Kubernetes service.
• Cluster - This is a set of nodes that share resources and run containerized applications.
• Replication Controllers - Intelligent agents responsible for scheduling and resource allocation among containers.
• Labels - These are tags that the Kubernetes service uses to identify containers that are members of a pod.

Kubernetes continues to be a popular choice among developers being open-source platform of extensive tools that offers flexibility and ease of use by improving workflows and maximizing productivity. The platform also offers a large library of functionalities developed by communities all over the world, giving it unmatched microservice management capabilities. As a result, plenty of managed out-of-the-box orchestration solutions are developed based on the Kubernetes.

2. Red Hat OpenShift

OpenShift was developed by Red Hat to provide a hybrid, enterprise-grade platform that extends Kubernetes functionalities to companies that require managed orchestration. The framework is built on an enterprise-grade Linux Operating System that lets you automate the lifecycle of your containerized application.  This lets you easily manage all your workloads using a container to virtualize every host. More so, with its various templates and pre-built images, OpenShift lets you create databases, frameworks and other application services easily. As a result, you get a highly-optimized platform that standardizes production workflows, enables continuous integration and helps companies automate the management of releases. As an added advantage, the Red Hat Marketplace lets you purchase certified applications that can help in a range of areas, such as, billing, visibility, governance and responsive support.

OpenShift offers both Platform-as-a-Service(PaaS) and Container-as-a-Service(CaaS) cloud service computing models. This essentially lets you either define your application source code in a Dockerfile or convert your source code to a container using a Source-to-Image builder.

Key features of Redhat OpenShift include:

• Built-in Jenkins pipelines streamline workflows, allowing faster production
• Comes with an Integrated Container Runtime (CoreOS), but also integrates well with Standard CRI-O and Docker Runtimes
• Supports SDN and validates integration with various networking solutions
• Integrates various development and operations tools to offer Self-Service Container Orchestration
• Its Embedded Operator Hub grants administrators easy access to services such as Kubernetes Operators, third-party solutions and direct access to cloud service providers, such as AWS
• OpenShift is an Open-Source, vendor-agnostic platform, without a vendor lock-in commitment

3. Apache Mesos

Mesos is a cluster management tool developed by Apache that can efficiently perform container orchestration. The Mesos framework is open-source, and can easily provide resource sharing and allocation across distributed frameworks. It enables resource allocation using modern kernel features, such as Zones in Solaris and CGroups in Linux. Additionally, Mesos uses Chronos Scheduler to start and stop services, and the Marathon API to scale services and balance loads. To let developers define inter-framework policies, Mesos uses a pluggable application module .

More details on the Mesos architecture can be found here.

Key features of Apache Mesos include:

• Linear scalability, allowing the deployment of 10,000s of nodes
• Zookeeper integration for fault-tolerant master replication
• APIs for developing new applications in Java, C++, etc.
• Graphical User Interface for monitoring the state of your clusters
• LXC isolation between tasks

Advantages of using Mesos seem apparent as Apache claims to have build a number of software projects on Mesos, which include: Long running services such as Aurora, Marathon & Singularity, Big Data Processing Solutions, Batch Scheduling and Data Storage Solutions.

4. Mirantis Kubernetes Engine

Formerly known as Docker Enterprise Edition, Mirantis is an orchestration tool that lets you manage Kubernetes Clusters and Docker Swarms interchangeably to provide ultimate runtime flexibility. The solution offers multiple layers of security that include Role Based Access Control (RBAC) and built-in encryption, providing advanced authentication and access control. With its node-based isolation, Mirantis enables efficient multi-tenant architecture by offering a clear separation of resources. The Mirantis Engine uses Calico for Kubernetes networking and includes an Istio Ingress, enhancing load balancing and providing streamlined gateway controls. Mirantis allows development teams to ship code faster by providing a simple, consistent experience across all major cloud platforms, and offers a choice of various tools and frameworks you can use to improve application portability.

Mirantis allows organizations to innovate and scale applications using its Drivetrain lifecycle management system. The tool integrates into the Mirantis Cloud platform to deliver regular updating and improvement of open-source software. Incremental updates are enabled via a Git repo, which reduces the cost and time of upgrades. Mirantis Drivetrain lifecycle enables a DevOps approach to development while also enabling a Continuous Delivery pipeline.

Key features of Mirantis Kubernetes Engine include:

• Easier management of containers using standard image builds
• Mirantis isolation ensures security by having all applications running independently in different containers
• Allows application portability across various platforms
• Mirantis Drivetrain enables continuous integration for faster deployment

5. Helios

Developed by Spotify, Helios helps developers orchestrate Docker containers by deploying them across distributed servers. Helios is particularly popular with developers due to its pragmatic nature and its functionalities that enhance CI/CD pipelines. The platform also fits seamlessly with  most DevOps workflows, that doesn't require specific Operating Systems, Cloud Services or Network Topologies to manage containers. As an added advantage, Helios documents cluster history, with a log of events such as restarts, deployments and version changes. This essentially helps developers identify root causes or security vulnerabilities efficient using either a HTTP API Client or Command Line Interface.

Key features of Helios include:

• Easily integrates with DevOps philosophy
• Vendor-agnostic, works well with any platform or network
• Can run both single and multi-node instances
• Does not require Apache Mesos as a prerequisite to run Helios
• Does not depend on prescribed load balancers and routers

6. Amazon Elastic Container Service (Amazon ECS)

With Amazon ECS, organizations can easily deploy and run container clusters on Amazon’s Elastic Container (EC2) instances. Amazon ECS offers a secure, reliable, and highly scalable orchestration platform, making it appropriate for sensitive and mission critical applications without wasting compute resources. Amazon ECS easily integrates with Amazon Fargate, a serverless computing tool, that lets developers specify resource requirements and eliminates the need for server provision. This lets organizations to focus more on streamlining applications rather than managing infrastructure. It is also easy to cost-optimize your application using Fargate Spot tasks and EC2 Spot instances, cutting off up to 90% of your infrastructure provision fees.

ECS allows you to use Network Access Control Lists (ACLs) and Amazon Virtual Private Clouds (VPCs) for resource isolation and security. Possibly one of the key features of ECS is that it is available in 69 availability zones and 22 regions globally, guaranteeing peace of mind regarding uptime, reliability, and low latency.

Key features of AWS ECS include:

• ECS Supports Fargate, a serverless AWS offering, that eliminates the need to manage servers
• Includes Capacity Providers which dynamically determine compute resources required to run your application
• Help optimize costs using spot instances for non-persistent workloads
• ECS creates Amazon VPCs for your containers, ensuring no sharing of resources between tenants
• Container Registry makes applications compatible within multiple environments

7. Google Kubernetes Engine (GKE)

GKE is a managed orchestration service that provides an easy-to-use environment to deploy, manage and scale Docker containers on the Google Cloud Platform. While doing so, the service engine lets you create agile and serverless applications without compromising security. With multiple release channels offering different node upgrade cadences, GKE makes it easier to streamline operations based on application needs. Through its enterprise-ready, pre-built deployment templates GKE enables enhanced developer productivity across multiple layers of a DevOps workflow.

For developers, the service engine helps streamline every stage of the SDLC using native CI/CD tooling accelerators, while Site Reliability Engineers (SREs) may utilize GKE for ease of infrastructure management by monitoring resource usage, clusters and networks.

Key features of GKE include:

• GKE offers rapid, regular and stable release channels, allowing developers to streamline operations
• The platform sets up the baseline functionality and automates cluster management for ease of use
• Integrates native Kubernetes tooling so organizations can develop applications faster without compromising security
• Google Site Reliability Engineers offer support in the management of infrastructure
• Google consistently improves the GKE platform with new features and enhancements, making it robust and reliable
• Well-documented platform, making all its features easy to learn and use

8. Azure Service Fabric

Microsoft's Azure Service Fabric is a Platform-as-a-Service solution that lets developers focus on business logic and application development by making container deployments, packaging and management a lot easier. Service Fabric lets companies deploy and manage microservices across distributed machines, allowing the management of both stateful and stateless services. It also integrates seamlessly with CI/CD tools to help manage application life cycles while letting you create and manage clusters across different environments, including Linux, Windows Server, Azure on-premises and other public cloud offerings.

Service Fabric uses a .NET SDK to integrate with popular Windows Software Development Kits, such as PowerShell and Visual Studio. To integrate with Linux development solutions, such as Eclipse, it uses a Java SDK. Service Fabric is available across all Azure regions, and is included on all Azure Compliance Certifications.

Key features of Azure Service Fabric include:

• Service Fabric allows management of containerized applications on both stateful and stateless services
• Can be used for lift & shift migration using guest executables for legacy applications
• Enables a Serverless Compute experience, so organizations don’t have to worry about backend provisioning
• The Azure platform is data-aware, improving workload performance while reducing latency
• Makes applications resilient by running different tracks for different servers hosting different microservices

Azure Fabric Service can be teamed up with CI/CD services such as the Visual Studio Team Services to ensure successful migration of existing apps to the cloud. This makes it easy to debug applications remotely and seamless monitoring using the Operations Management Suite.

9. Amazon Elastic Kubernetes Service (EKS)

Amazon EKS helps developers create, deploy and scale Kubernetes applications on-premises or in the AWS cloud. EKS automates tasks such as patching, updates and node provisioning, thereby helping organizations to ship reliable, secure and highly scalable clusters. While doing so, EKS takes away all the tedium and manual configuration tasks to manage Kubernetes clusters, helping to cut-down on efforts of performing repetitive tasks to run your applications.

Since EKS is an upstream offering of Kubernetes, you can use all existing Kubernetes plugins and tools for your application. This service automatically deploys Kubernetes with three master nodes across multiple availability zones for ultimate reliability and resilience. With Role Based Access Control (RBAC) and Amazon’s Identity and Access Management (IAM) entities, you can easily manage security in your AWS clusters using Kubernetes tools, such as kubectl. As one of its core features, EKS allows launching and managing Kubernetes clusters easy using a few easy steps.

Key features of AWS EKS include:

• EKS provides a flexible Kubernetes Control Plane available across all regions. This makes Kubernetes applications hosted on EKS highly available and scalable.
• You can directly manage your applications from Kubernetes using AWS Controllers for Kubernetes
• Extending the functionality of your Kubernetes cluster is simple thanks to EKS Add-ons
• Easily scale, create, update and terminate nodes from your EKS cluster using a single command
• Compatibility between EKS and Kubernetes clusters ensures a simple, code-free migration to AWS cloud
• EKS implements automatic patches and identifies non-functioning masters, ensuring application reliability

Amazon EKS prevents single failure points of Kubernetes cluster by running it across multiple availability zones. This makes the application reliable, resilient and secure since by reducing the Mean-time-to-Recovery (MTTR). Additionally, as a Managed Kubernetes platform, Amazon’s EKS makes your application optimized and scalable through a rich ecosystem of services that eases container management.

10. Docker Swarm

Swarm is the native container orchestration platform for Docker applications. In Docker, a Swarm is a group of machines (physical or virtual) that work together to run Docker applications. A Swarm Manager controls activities of the swarm, and helps manage the interactions of containers deployed on different host machines (nodes). Docker Swarm fully leverages the benefits of containers, allowing highly portable and agile applications while offering redundancy to guarantee high availability for your applications. Swarm managers also assign workloads to the most appropriate hosts, ensuring proper load balancing of applications. While doing so, the Swarm Manager ensures proper scaling by adding and removing worker tasks to help maintain a cluster’s desired state.

Key features of Docker Swarm include:

• Manager nodes help with load balancing by assigning tasks to the most appropriate hosts
• Docker Swarm uses redundancy to enable high service availability
• Swarm containers are lightweight and portable
• Tightly integrated into the Docker Ecosystem, allowing easier management of containers
• Does not require extra plugins for setup
• Ensures high scalability by balancing loads and bringing up worker nodes when workload increases
• Docker Swarm’s distributed environment allows for decentralized access and collaboration

As Docker remains one of the most used container runtimes, Docker Swarm proves to be an efficient container orchestration tool. Swarm makes it easy to scale, update applications and balance workloads. This makes it perfect for application deployment and management even when dealing with extensive clusters.

Conclusion

As more cloud deployment technologies emerge, container orchestration tools will keep evolving. As with every technology, each option has its benefits and drawbacks. Managed Service Platforms, such as GKE, EKS and Mirantis, provide unmatched functionality while incurring little costs. Other offerings, like Kubernetes and Docker Swarm should be evaluated for the trade-offs in performance, complexity and flexibility before adoption.

With that in mind, it is equally important to note that orchestration tools are cost and effort intensive. An efficient alternative to this is appfleet's global edge platform for deploying and hosting containerized applications. With appfleet, you do not have to spend time building a team to manage a mammoth clustering framework, learning a new technology, or digging through thousands of lines of documentation.

First of all what is appfleet? appfleet is an edge compute platform that allows people to deploy their web applications globally. Instead of running your code in a single centralized location you can now run it everywhere, at the same time.

In simpler terms appfleet is a next-gen CDN, instead of being limited to only serving static content closer to your users you can now do the same thing for your whole codebase. Run the whole thing where just your cache used to be.

Check out our video explainer too https://www.youtube.com/watch?v=7n617ZF-oT4

This results in drastic performance improvement, lower latency, better uptime and an enormous amount of new use-cases and possibilities.

In part it's because we are not limiting our users to HTTP services. You have complete freedom to run any kind of service over any protocol you want, on any port.

Do you want to build your own global nameservers? Deploy a container running a DNS server over UDP port 53. It takes only a few clicks. A globally distributed database? Sure thing. Or something simple like image optimization on the edge? No problem. How about a monstrous container running a web service, redis, ssh, DNS, MySQL and an admin service all on different ports, all at the same time? Who are we to stop you, go ahead!

We launched our closed beta quite a while ago. And since then we have worked with many developers and business owners to improve our platform and support the many exciting use-cases that people kept coming up with.

And while the system was technically ready to accept real customers months ago, we decided it was best to stay in beta and ensure the stability of the platform.

Since then we have polished our user-experience, redesigned many parts of our UI multiple times and made sure our backend is ready for whatever people throw at it.

Do you know what is the first thing people run when given a container and asked to test it? They run a fork bomb.  Of course we were ready for that and while the specific instance became unavailable it had no impact on our system or other clients. All thanks to the multiple layers of isolation we have built. And nowadays the instance won't even go down, so fork-bomb away.

Each container runs in a virtualized box of its own, with its own resources, filesystem and security in place. Security and stability were top priorities for us and everything we built had that in mind.

Whenever we were working on a new feature we kept asking ourselves "What if?". "What if this system goes down?", "What if the user does something unexpected?"

appfleet is based on multiple services and modules interacting with each other. We made sure that even if something breaks, like our whole API goes down, or our DB, or anything else really, the already running applications would not feel a thing. Everything is built to run in standalone mode and if the worst happens to wait until things get better.

We also drew from our experience building and maintaining jsDelivr, a free CDN for open source projects that currently serves 100 billion requests every month and more than 3 Petabytes of traffic! It is used by millions of websites all over the world and all of them trust us to ensure it never goes down. We integrated multiple levels of failover with multiple checks on every step to ensure the system can automatically handle different kinds of issues and fix itself.

The appfleet platform was built to be as simple as possible and make edge compute accessible to everyone, from open source projects, to solo developers and even big enterprises that need something that works without relying on an army of DevOps engineers. This is one of the reasons we decided to build on top of containers, this allows our users to easily migrate to or from appfleet and to even run legacy applications.

Today March 1st 2021 is the beginning of a long an exciting journey to make the web faster and accessible to all!

Register now and get $10 of free credits to use as you see fit. No need to enter your credit card until you are ready for production workloads.

And if you are a non-profit or an open source project let us know to get sponsored with free services. We even offer free design services

Join us and make sure you send us your feedback and ideas! Even better email the founder directly at d@appfleet.com

Today, most organizations, large or small, are hosting their SaaS application on the cloud using multi-tenant architecture. There are multiple reasons for this, but the most simple and straightforward reasons are cost and scalability. In a multi-tenant architecture, one instance of a software application is shared by multiple tenants (clients). These tenants share various resources such as databases, web servers, computing resources, etc., that make multi-tenant architecture cost-efficient while keeping scalability in mind.

Amazon EKS (Elastic Kubernetes Service) is one of the most popular container orchestration platforms offered by AWS, which is popularly used host multi-tenant SaaS applications. However, while adopting a multi-tenancy framework, it is also important to note the challenges that might prevail due to cluster resource sharing.

Let us delve into the best practices and considerations for multi-tenant SaaS applications using Amazon EKS in this article.

Practice 1: Separate Namespaces for Each Tenant (Compute Isolation)

Having separate namespaces remains an essential consideration while deploying a multi-tenant SaaS application, as it essentially divides a single cluster resource across multiple clients. Namespaces in a multi-tenant architecture are the primary unit of isolation in Kubernetes. As one of its core features, Amazon EKS helps you create a separate namespace for each tenant running the SaaS application. This attribute aids in isolating every tenant and its own environment within the corresponding Kubernetes cluster, thereby enforcing data privacy in the sense that you don't have to create different clusters per tenant. This aspect eventually brings substantial cost reduction in compute resources and AWS hosting costs.

Practice 2: Setting ResourceQuota on Resource Consumption

A multi-tenant SaaS application serves several tenants, with each of them accessing the same Kubernetes cluster resources concurrently. Often there would be scenarios where a particular tenant consume resources disproportionately to exhaust all the cluster resources on its own, without leaving resources to be used by other tenants. To avoid such instances of capacity downtime the ResourceQuota concept comes to rescue, with which you can set limit on the resources the containers (hosting the SaaS application) can use.

Here is an example:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: resourceQuotaSetting
  namespace: tenant1
spec:
  hard:
    requests.cpu: "2"
    requests.memory: "1Gi"
    limits.cpu: "4"
    limits.memory: "2Gi"

To get the above in context, once you set up the above configuration on Amazon EKS for resource quota, a container can only request 2 CPUs at a time while in total it can have only 4 CPUs concurrently. Additionally, it can request 1 Gi memory at a time, with the maximum memory limit defined as 2Gi. With this you essentially limit the usage of resources while ensuring that a particular tenant does not end up consuming all resources.

Practice 3: Network Isolation using Network Policies

By default, the Kubernetes production cluster allows namespaces to talk to each other. But if your SaaS application is running in a multi-tenant architecture, you would like to avoid that to bring in the isolation between different namespaces. To do that, you can use tenant isolation network policy and network segmentation on Amazon EKS. As a best practice, you can install Calico on Amazon EKS and assign network policies to pods as shown below.

For reference, the following policy allows traffic only from the same-namespace while restricting other traffic. The pods with label app: api in tenant-a namespace will only receive traffic from same-namespace. The communication of tenant-a with other tenants and vice-versa is being denied here to achieve network isolation.

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: same-namespace
  namespace: tenant-a
spec:
  podSelector:
	matchLabels:
  	app: api
  policyTypes:
  - ingress
  - egress
  ingress:
  - from:
	- namespaceSelector:
    	    matchLabels:
      	 nsname: tenant-a
  egress:
  - to:
	- namespaceSelector:
    	    matchLabels:
      	 nsname: tenant-a

Practice 4: Storage Isolation using PersistentVolume and PersistentVolumeClaim

As opposed to a single-tenant framework, a multi-tenant framework requires a different approach to managing application storage. On Amazon EKS, you can assign and manage storage for different tenants seamlessly using PersistentVolume (PV). On the other hand, a storage request sent by a tenant is called referred as PersistentVolumeClaim (PVC). Since PVC is a namespaced resource, you can bring isolation of storage among different tenants easily.

In the example below for tenant1, we have configured PVC with ReadWriteOnce access mode and storage space of 2 Gi.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-storage
  namespace: tenant1
spec:
  accessModes:
	- ReadWriteOnce
  resources:
	requests:
  	storage: 2Gi 

Practice 5: Amazon IAM Integration with Amazon EKS for Setting RBAC

Just like any other AWS service, EKS integrates with AWS IAM to administer Role-Based Access Control (RBAC) on a Kubernetes cluster. Through the AWS IAM authenticator, you can authenticate any tenant namespace to a Kubernetes cluster. To use IAM on multi-tenancy, you need to add the tenant's (user) IAM role inside aws-auth configmap for authenticating the tenant's namespace. Once the authentication is successful by AWS IAM, defined Role (namespaced resource) and/or ClusterRole (non-namespaced resource) for a particular tenant's namespace will be implemented on the cluster. By provisioning ClusterRole and Role policies on the cluster, you can adopt a hardened security posture on a multi-tenant SaaS application.

Practice 6: Manage Tenant Placement on Kubernetes nodes

Similar to the default Kubernetes service, Amazon EKS provides Node Affinity and Pod Affinity for managing tenant placement on Kubernetes nodes through which you can use taint and toleration. Using Node Affinity, you can decide on which node you want to run a particular tenant's pod. On the other hand, using Pod Affinity you can decide if tenant 1 and tenant 2 should be on the same or on different nodes.

With the command below, no pods will be scheduled on node 1 until it matches the toleration arguments, while the key  value (client) must be tenant1. With this method, you can run pods on a particular node just for a particular tenant.

kubectl taint nodes node1 client=tenant1:NoSchedule

This is how a pod configuration looks like:

apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
  labels:
   env: prod
spec:
  containers:
  - name: ubuntu
	image: ubuntu
	imagePullPolicy: IfNotPresent
tolerations:
  - key: “client”
	operator: "Equal"
	value: “tenant1”
	effect: "NoSchedule"

Conclusion

That was all about the best practices and considerations for running a multi-tenant SaaS application on Amazon EKS. A multi-tenant SaaS application on EKS provides you with multiple possibilities for compute, network, storage isolations, while keeping the workflow secured. Go ahead and try out these practices and let us know your experience.

Alternately, if you have any additional best practices to share, do let us know.

In this guide, we will use Ansible as a Deployment tool in a Continuous Integration/Continuous Deployment process using Jenkins Job.

In the world of CI/CD process, Jenkins is a popular tool for provisioning development/production environments as well as application deployment through pipeline flow. Still, sometimes, it gets overwhelming to maintain the application's status, and script reusability becomes harder as the project grows.

To overcome this limitation, Ansible plays an integral part as a shell script executor, which enables Jenkins to execute the workflow of a process.

Let us begin the guide by installing Ansible on our Control node.

Install and Configure Ansible

Installing Ansible:
Here we are using CentOS 8 as our Ansible Control Node. To install Ansible, we are going to use python2-pip, and to do so, first, we have to install python2. Use the below-mentioned command to do so:

# sudo yum update
# sudo yum install python2

After Python is installed on the system, use pip2 command to install Ansible on the Control Node:

# sudo pip2 install ansible
# sudo pip2 install docker

It might take a minute or two to complete the installation, so sit tight. Once the installation is complete, verify:

# ansible --version
 
ansible 2.9.4
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.16 (default, Nov 17 2019, 00:07:27) [GCC 8.3.1 20190507 (Red Hat 8.3.1-4)]

Through the above command, we notice that the config file path is missing, which we will create and configure later. For now, let’s move to the next section.

Configuring Ansible Control Node User:
The first thing we are going to do is create a user named ansadmin, as it is considered the best practice. So let’s create a user, by using the command adduser, which will create a new user to our system:

# useradd ansadmin

Now, use the passwd command to update the ansadmin user’s password. Make sure that you use a strong password.

# passwd ansadmin

Changing password for user ansadmin.
New password: 
Retype new password: 
passwd: all authentication tokens updated successfully.

Copy the password for user ansadmin and save it somewhere safe.

Once we have created the user, it's time to grant sudo access to it, so it doesn't ask for a password when we log in as root. To do so, follow the below-mentioned steps:

# nano /etc/sudoers

Go to the end of the file and paste the below-mentioned line as it is:

...
ansadmin ALL=(ALL)       NOPASSWD: ALL
...

Before moving forward, we have one last thing to do. By default, SSH password authentication is disabled in our instance. To enable it, follow the below-mentioned steps:

# nano /etc/ssh/sshd_config

Find PasswordAuthentication, uncomment it and replace no with yes, as shown below:

...
PasswordAuthentication yes
...

You will see why we are doing this in the next few steps. To reflect changes, reload the ssh service:

# service sshd reload

Now, log in as an ansadmin user on your Control Node and generate ssh key, which we will use to connect with our remote or managed host. To generate the private and public key, follow the below-mentioned commands:

# su - ansadmin

Use ssh-keygen command to generate key:

# ssh-keygen

Enter file in which to save the key (/home/ansadmin/.ssh/id_rsa): ansible-CN   
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in ansible-CN.
Your public key has been saved in ansible-CN.pub.
The key fingerprint is:
SHA256:6G0xzIrIsmsBwCakACI8CVr8AOuRR8v5F1p2+CsB6EY ansadmin@ansible-host
The key's randomart image is:
+---[RSA 3072]----+
|&+o.             |
|OO* +   .        |
|Bo.E . = .       |
|o = o =++        |
|.. o o.oS.       |
| o.. o.o.o.      |
|. + . o.o.       |
| +     ..        |
|+.               |
+----[SHA256]-----+

Usually, keys are generated in the .ssh/ directory. In our case, you can find keys at /home/ansadmin/.ssh/. Now let us configure our Managed Host for Ansible.

Configuring Ansible Managed Host User:
First, we will create a user on our managed host, so log in to your host and create a user with the same name and password.

As our managed host is an Ubuntu machine, therefore here we have to use the adduser command. Please make sure that the password for the username ansadmin is the same for Control and Managed Host.

# adduser ansadmin
# su - ansadmin

Other than this, it is also an excellent thing to cross-check if password authentication is enabled on the Managed Host as we need to copy the ssh public key from the control node to the Managed Host.

Switch to Control Node machine; to copy the public key to our Managed Host machine, we will use the command ssh-copy-id:

$ su - ansadmin
$ ssh-copy-id -i .ssh/ansible-CN.pub ansadmin@managed-host-ip-here 

For the first time, it will ask for the password. Enter the password for ansadmin, and you are done. Now, if you wish, you can disable Password Authentication on both machines.

Setting Ansible Inventory:
Ansible allows us to manage multiple nodes or hosts at the same time. The default location for the inventory resides in /etc/ansible/hosts. In this file, we can define groups and sub-groups.

If you remember, earlier, the hosts' file was not created automatically for our Ansible. So let's create one:

# cd /etc/ansible
# touch hosts && nano hosts

Add the following lines in your hosts' file and save it:

[docker_group]
docker_host ansible_host=your-managed-host-ip ansible_user=ansadmin ansible_ssh_private_key_file=/home/ansadmin/.ssh/ansible-CN ansible_python_interpreter=/usr/bin/python3
ansible_CN ansible_connection=local

Make sure that you replace your-managed-host-ip with your host IP address.

Let's break down the basic INI format:

• docker_group - Heading in brackets is your designated group name.
• docker_host & ansible_CN - The first hostname is docker_host, which points to our Managed Host. While the second hostname is ansible_CN, which is pointing towards our localhost, to be used in Ad-Hoc commands and Playbooks.
• ansible_host - Here, you need to specify the IP address of our Managed Host.
• ansible_user - We mentioned our Ansible user here.
• ansible_ssh_private_key_file - Add the location of your private key.
• ansible_python_interpreter - You can specify which Python version you want to use; by default, it will be Python2.
• ansible_connection - This variable helps Ansible to understand that we are connecting the local machine. It also helps to avoid the SSH error.

It is time to test our Ansible Inventory, which can be done through the following command. Here we are going to use a simple Ansible module PING:

# ansible all -m ping
ansible_CN | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    }, 
    "changed": false, 
    "ping": "pong"
}
docker_host | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    }, 
    "changed": false, 
    "ping": "pong"
}

It looks like the Ansible system can now communicate with our Managed Host as well as with the localhost.

Install Docker:

We need a Docker ready system to manage our process; for this, we have to install Docker on both systems. So follow the below-mentioned steps:

For CentOS (Control Node):
Run the following command on your Control Node:

# sudo yum install -y yum-utils device-mapper-persistent-data lvm2
 
# sudo yum-config-manager --add-repo \  https://download.docker.com/linux/centos/docker-ce.repo
 
# sudo yum install docker-ce docker-ce-cli containerd.io

In case you encounter the below-mentioned error during installation:

Error: 
 Problem: package docker-ce-3:19.03.5-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed

Next, run the following command:

# sudo yum install docker-ce docker-ce-cli containerd.io --nobest

For Ubuntu OS (Managed Host):
Run the following command on your Managed Host, which is a Ubuntu-based machine:

$ sudo apt-get remove docker docker-engine docker.io containerd runc
 
$ sudo apt-get update && sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

$ sudo apt-get update && sudo apt-get install docker-ce docker-ce-cli containerd.io

That’s it for this section. Next, we are going to cover how to integrate Ansible with Jenkins.

Integrating Ansible with Jenkins:

In this section, we will integrate Ansible with Jenkins. Fire up your Jenkins, go to Dashboard > Manage Jenkins > Manage Plugins > Available and then search for Publish Over SSH as shown in the image below:

Now, go to Configure System and find Publish over SSH; under this section, go to SSH Servers and click on the Add button. Here we are going to add our Docker Server as well as Ansible Server, as shown in the image:

SSH server setting for Docker:

SSH server setting for Ansible:

In the Hostname field, add your IP address or domain name of Docker and Ansible server. Before saving the setting, make sure that you test the connection before saving the configuration, by clicking on the Test Configuration button as shown in the image below:

Create Jenkins Job

The next step is to create Jenkins jobs. The sole propose of this Job is to build, test, and upload the artifact to our Ansible Server. Here we are going to create Job as a Maven Project, as shown in the image below:

Next in Job setting page, go to the Source Code Management section and add your Maven project repo URL, as shown in the image below:

Find the Build section, and in Root POM field enter your pom.xml file name. Additionally in the Goals and options field enter clean install package:

After successful build completion, your goal is to send the war file to the specified directory to your Ansible server with the right permissions so that it doesn't give us the writing permission by assigning ansadmin to the directory.

Right now, we don't have such a directory, so let us create one. Follow the below-mentioned steps:

# sudo su
# mkdir /opt/docker
# chown ansadmin:ansadmin /opt/docker -R
# ls -al /opt/docker/

total 0
drwxr-xr-x. 2 ansadmin ansadmin  6 Jan 31 16:57 .
drwxr-xr-x. 4 root     root     38 Jan 31 17:10 ..

Directory /opt/docker will be used as our workspace, where Jenkins will upload the artifacts to Ansible Server.

Now, go to the Post-build Actions section and from the drop-down menu, select Send build artifacts over SSH, as shown in the image below:

Make sure that in the Remote Directory field, you enter the pattern //opt//docker as it doesn’t support special characters. Apart from this, for now, we are going to leave the Exec Command field empty so that we can test whether our existing configuration works or not.

Now Build the project, and you will see the following output in your Jenkins’s console output:

Go to your Ansible Server terminal and see if the artifact was sent with right user privileges:

# ls -al /opt/docker/

total 4
drwxr-xr-x. 2 ansadmin ansadmin   24 Feb  3 10:54 .
drwxr-xr-x. 4 root     root       38 Jan 31 17:10 ..
-rw-rw-r--. 1 ansadmin ansadmin 2531 Feb  3 10:54 webapp.war

It looks like our webapp.war file was transferred successfully. In the following step, we will create an Ansible Playbook and Dockerfile.

Creating Dockerfile and Ansible Playbook:

To create a Docker Image with the webapp.war file, first, we will create a DockerFile. Follow the below-mentioned steps:

First, log in to your Ansible Server and go to directory /opt/docker and create a file named as Dockerfile:

# cd /opt/docker/
# touch Dockerfile

Now open the Dockerfile in your preferred editor, and copy the below-mentioned lines and save it:

FROM tomcat:8.5.50-jdk8-openjdk
MAINTAINER Your-Name-Here
COPY ./webapp.war /usr/local/tomcat/webapps

Here instructions are to pull a Tomcat image with tag 8.5.50-jdk8-openjdk and copying the webapp.war file to Tomcat default webapp directory., which is /usr/local/tomcat/webapps

With the help of this Dockerfile, we will create a Docker container. So let us create the Ansible Playbook, which will enable us to automate the Docker image build process and later run the Docker container out of it.

We are creating a Ansible Playbook, which does two tasks for us:

1. Pull Tomcat’s latest version and build an image using webapp.war file.
2. Run the built image on the desired host.

For this, we are going to create a new YAML format file for your Ansible Playbook:

# nano simple-ansible.yaml

Now copy the below-mentioned line into your simple-ansible.yaml file:

---
#Simple Ansible Playbook to build and run a Docker containers
 
- name: Playbook to build and run Docker
  hosts: all
  become: true
  gather_facts: false
 
  tasks:
    - name: Build a Docker image using webapp.war file
      docker_image:
        name: simple-docker-image
        build:
          path: /opt/docker
          pull: false
        source: build
 
    - name: Run Docker container using simple-docker-image
      docker_container:
        name: simple-docker-container
        image: simple-docker-image:latest
        state: started
        recreate: yes
        detach: true
        ports:
          - "8888:8080"

You can get more help here: docker_image and docker_container. Now, as our Playbook is created, we can run a test to see if it works as planned:

# cd /opt/docker
# ansible-playbook simple-ansible-playbook.yaml --limit ansible_CN

Here we have used the --limit flag, which means it will only run on our Ansible Server (Control Node). You might see the following output, in your terminal window:

PLAY [Playbook to build and run Docker] 
***************************************************************************
 
TASK [Build Docker image using webapp.war file] 
***************************************************************************
changed: [ansible_CN]
 
TASK [Run Docker image using simple-docker-image]
***************************************************************************
changed: [ansible_CN]
 
PLAY RECAP 
***************************************************************************
ansible_CN                 : ok=2    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Look's like Playbook ran sccessfully and no error was detected during the Ansible Playbook check, so now we can move to Jenkins to complete our CI/CD process using Ansible.

Run Ansible Playbook using Jenkins

In this step, we would execute our Ansible Playbook (i.e., simple-ansible-playbook.yaml) file, and to do so let us go back to the Project Configuration page in Jenkins and find Post-build Actions there.

In this section, copy the below-mentioned command in the Exec command field:

sudo ansible-playbook --limit ansible_CN /opt/docker/simple-ansible-playbook.yaml;

Now, let us try to build the project and see the Jenkins Job's console output:

In the output, you can see that our Ansible playbook ran successfully. Let us verify if at Ansible Server the image is created and the container is running:

For Docker Image list:

# docker images
 
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
simple-docker-image   latest              d47875d99095        32 seconds ago      507MB
tomcat                latest              5692d26ea179        15 hours ago        507MB

For Docker Container list:

# docker ps

CONTAINER ID        IMAGE                        COMMAND             CREATED             STATUS              PORTS                    NAMES
5a824d0a43d5        simple-docker-image:latest   "catalina.sh run"   15 seconds ago      Up 14 seconds       0.0.0.0:8888->8080/tcp   simple-docker-container

It looks like Jenkins was able to run the Ansible Playbook successfully. Next, we are going to push Docker Image to Docker Hub.

Pushing Docker Image to Docker Hub Using Ansible

We are going to use Docker Hub public repository for this guide; in case you want to work on a live project, then you should consider using the Docker Hub private registry.

For this step, you have to create a Docker Hub account if you haven’t had one yet.

Our end goal for this step is to publish the Docker Image to Docker Hub using Ansible Playbook. So go to your Ansible Control Node and follow the below-mentioned steps:

# docker login
 
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: your-docker-hub-user
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
 
Login Succeeded

Make sure that you enter the right username and password.

Now it’s time to create a new Ansible Playbook which will build and push the Docker image to your Docker Hub account. Note that this image will be publicly available, so be cautious.

# nano build-push.yaml

Create a new Ansible Playbook, which will build a Docker image and push it to our Docker Hub account:

---
#Simple Ansible Playbook to build and push Docker image to Registry
 
- name: Playbook to build and run Docker
  hosts: ansible_CN
  become: true
  gather_facts: false
 
  tasks:
    - name: Delete existing Docker images from the Control Node
      shell: docker rmi $(docker images -q) -f 
      ignore_errors: yes
 
    - name: Push Docker image to Registry
      docker_image:
        name: simple-docker-image
        build:
          path: /opt/docker
          pull: true
        state: present
        tag: "latest"
        force_tag: yes
        repository: gauravsadawarte/simple-docker-image:latest
        push: yes
        source: build

Let us run the playbook now and see what we get:

# ansible-playbook --limit ansible_CN build-push.yaml
 
PLAY [Playbook to build and run Docker] 
*****************************************************************************************
 
TASK [Push Docker image to Registry] 
*****************************************************************************************
changed: [ansible_CN]
 
PLAY RECAP 
*****************************************************************************************
ansible_CN                 : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Go to your Docker Hub account and see if the image was pushed successfully, as shown in the image below:

Next, let us modify our simple-ansible-playbook.yaml playbook, which we created earlier, as from here on, we are going to pull the Docker image from Docker Hub Account and create a container out of it.

---
#Simple Ansible Playbook to pull Docker Image from the registry and run a Docker containers
 
- import_playbook: build-push.yaml
 
- name: Playbook to build and run Docker
  hosts: docker_host
  gather_facts: false
 
  tasks:
    - name: Run Docker container using simple-docker-image
      docker_container:
        name: simple-docker-container
        image: gauravsadawarte/simple-docker-image:latest
        state: started
        recreate: yes
        detach: true
        pull: yes
        ports:
          - "8888:8080"

Note that we have used the import_playbook statement at the top of the existing playbook, which means that we want to run the build-push.yaml playbook first along with our main playbook, and this way, we don’t have to run multiple playbooks manually.

Let us break the whole process into steps:

1. With the help of build-push.yaml playbook, we are asking Ansible to build an image with the artifacts sent by Jenkins to our Control Node, and later push the built image (i.e., simple-docker-image) to our Docker Hub’s account or any other private registry like AWS ECR or Google’s Container Registry.
2. In the simple-ansible-playbook.yaml file, we have imported the build-push.yaml file, which is going to run prior to any statement present within the simple-ansible-playbook.yaml file.
3. Once build-push.yaml playbook is executed, Ansible will launch a container into our Managed Docker Host by pulling our image from our defined registry.

Now, it's time to build our job. So in the next step, we will deploy the artifact to our Control Node, where Ansible Playbook will build an image, push to Docker Hub and run the container in Managed Host. Let us get started!

Jenkins Jobs to Deploy Docker Container Using Ansible

To begin, go to JenkinstoDockerUsingAnsible configure page and change the Exec command in the Post-build Actions section.

Copy the below-mentioned command and add it as shown in the image below:

sudo ansible-playbook /opt/docker/simple-ansible-playbook.yaml;

Save the configuration and start the build; you will see the following output:

Now go to your Control Node and verify if our images were built:

# docker images
REPOSITORY                            TAG                   IMAGE ID            CREATED             SIZE
gauravsadawarte/simple-docker-image   latest                9ccd91b55796        2 minutes ago       529MB
simple-docker-image                   latest                9ccd91b55796        2 minutes ago       529MB
tomcat                                8.5.50-jdk8-openjdk   b56d8850aed5        5 days ago          529MB

It looks like Ansible Playbook was successfully executed on our Control Node. It’s time to verify if Ansible was able to launch containers on our Managed Host or not.

Go to your Managed Host and enter the following command:

# docker ps
CONTAINER ID        IMAGE                                        COMMAND                  CREATED             STATUS              PORTS                               NAMES
6f5e18c20a68        gauravsadawarte/simple-docker-image:latest   "catalina.sh run"        4 minutes ago       Up 4 minutes        0.0.0.0:8888->8080/tcp              simple-docker-container

Now visit the following URL http://your-ip-addr:8888/webapp/ in your browser. Note that, Tomcat Server may take some time before you can see the output showing your project is successfully setup.

And you are done!

You successfully managed to deploy your application using Jenkins, Ansible, and Docker. Now, whenever someone from your team pushes code to the repository, Jenkins will build the artifact and send it to Ansible, from there Ansible will be responsible for publishing the application to the desired machine.

There is no doubt about the fact that Docker makes it very easy to deploy multiple applications on a single box. Be it different versions of the same tool, different applications with different version dependencies - Docker has you covered. But then nothing comes free. This flexibility comes with some problems - like high disk usage and large images. With Docker, you have to be careful about writing your Dockerfile efficiently in order to reduce the image size and also improve the build times.

Docker provides a set of standard practices to follow in order to keep your image size small - also covers multi-stage builds in brief.

Multi-stage builds are specifically useful for use cases where we build an artifact, binary or executable. Usually, there are lots of dependencies required for building the binary - for example - GCC, Maven, build-essentials, etc., but once you have the executable, you don’t need those dependencies to run the executable. Multi-stage builds use this to skim the image size. They let you build the executable in a separate environment and then build the final image only with the executable and minimal dependencies required to run the executable.

For example, here’s a simple application written in Go. All it does is to print “Hello World!!” as output. Let’s start without using multi-stage builds.

Dockerfile
FROM golang
ADD . /app
WORKDIR /app
RUN go build # This will create a binary file named app
ENTRYPOINT /app/app

• Build and run the image

docker build -t goapp .
~/g/helloworld ❯❯❯ docker run -it --rm goapp
Hello World!!

• Now let us check the image size

~/g/helloworld ❯❯❯ docker images | grep goapp
goapp                                          latest              b4221e45dfa0        18 seconds ago      805MB

• New Dockerfile

# Build executable stage
FROM golang
ADD . /app
WORKDIR /app
RUN go build
ENTRYPOINT /app/app
# Build final image
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /app/app .
CMD ["./app"]

• Re-build and run the image

docker build -t goapp .
~/g/helloworld ❯❯❯ docker run -it --rm goapp
Hello World!!

• Let us check the image again

~/g/helloworld ❯❯❯ docker images | grep goapp
goapp                                          latest              100f92d756da        8 seconds ago       8.15MB
~/g/helloworld ❯❯❯

We can see a massive reduction in the image size -> From 805 MB we are down to 8.15MB. This is mostly because the Golang image has lots of dependencies which our final executable doesn’t even require for running.

What’s happening here?

We are building the image in two stages. First, we are using a Golang base image, copying our code inside it and building our executable file App. Now in the next stage, we are using a new Alpine base image and copying the binary which we built earlier to our new stage. Important point to note here is that the image built at each stage is entirely independent.

• Stage 0

# Build executable stage
FROM golang
ADD . /app
WORKDIR /app
RUN go build
ENTRYPOINT /app/app

• Stage 1

# Build final image
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /app/app .
CMD ["./app”]

Note the line COPY —from=0 /app/app  -> this lets you access data from inside the image built in previous image.

How multi-stage builds work?

If you look at the process carefully multi-stage builds are not much different from actual Docker builds. The only major difference is that you build multiple independent images (1 per stage) and you get the capability to copy artifacts/files from one image to another easily. The feature which multi-stage build is providing right now was earlier achieved through scripts. People used to create the build image - copy artifact from it manually - and then copy it to a new image with no additional dependencies. In the above example, we build one image in Stage 0 and then in Stage 1 we build another image, to which we copy files from the older image - nothing complicated.

NOTE: We are copying /app from one image to another - not one container to another.

This can speed up deployments and save cost in multiple ways

• You build efficient lightweight images - hence you ship lesser data during deployment which saves both cost and time.
• You can stop a multi-stage build at any stage - so you can use it to avoid the builder pattern with multi-stage builds. Have a single Dockerfile for both dev, staging and deployment.

The above was just a small example, multi-stage builds can be used to improve Docker images of applications written in other languages as well. Also, multi-stage builds can help to avoid writing multiple Dockerfiles (builder pattern) - instead a single Dockerfile with multiple stages can be adapted to streamline the development process. If you haven't explored it already - go ahead and do it.

As the cloud-native ecosystem continues to evolve, many alternative solutions are popping-up, that challenges the status quo of application deployment methodologies. One of these solutions that is quickly gaining traction is Unikernels, which are executable images that can run natively on a hypervisor without the need for a separate operating system.

For cloud-native platforms to integrate unikernels into their ecosystem, they are required to provide unikernels with the same services they provide for containers. In this post, we are going to introduce UniK, an open-source orchestration system for unikernels.

UniK (pronounced you-neek) is a tool for simplifying compilation and orchestration of unikernels. Similar to the way Docker builds and orchestrates containers, UniK automates compilation of popular languages (C/C++, Golang, Java, Node.js. Python) into unikernels. UniK deploys unikernels as virtual machines on various cloud platforms.

UniK Design

The UniK Daemon consists of 3 major components.

• The API server
• Compilers
• Providers

The API Server handles requests from the CLI or any HTTP Client, then determines which is the appropriate provider and/or compiler to service the request.

When the API Server receives a build request (POST /images/:image_name/create), it calls the specified compiler to build the raw image, and then passes the raw image to the specified provider, which processes the raw image with the Stage() method, turning it into an infrastructure-specific bootable image (e.g. an Amazon AMI on AWS)

Let's go ahead and try spinning up a unikernel ourselves.

Deploying a GO HTTP daemon with UniK

In this tutorial, we are going to be:

1. Installing UniK
2. Writing a simple HTTP Daemon in Go
3. Compiling to a unikernel and launching an instance on Virtualbox

Install, configure, and launch UniK

Prerequisites

Ensure that each of the following are installed

• Docker installed and running with at least 6GB available space for building images
• jq
• make
• Virtualbox

Install UniK

$ git clone https://github.com/solo-io/unik.git
$ cd unik
$ make binary

Note: make will take quite a few minutes the first time it runs. The UniK Makefile is pulling all of the Docker images that bundle UniK's dependencies.

Then, place the unik executable in your $PATH to make running UniK commands easier:

$ mv _build/unik /usr/local/bin/

Configure a Host-only network on VirtualBox

1. Open Virtualbox.
2. Open Preferences > Network > Host-only Networks.
3. Click the green Add button on the right side of the UI.
4. Record the name of the new Host-only adapter (e.g. "vboxnet0"). You will need this in your UniK configuration.
5. Ensure that the Virtualbox DHCP Server is Enabled for this Host-only Network
6. With the Host-only Network selected, Click the edit button (screwdriver image)
7. In the Adapter tab, note the IPv4 address and netmask of the adapter.
8. In the DHCP Server tab, check the Enable Server box.
9. Set Server Address an IP on the same subnet as the Adapter IP. For example, if the adapter IP is 192.168.100.1, make set the DHCP server IP as 192.168.100.X, where X is a number between 2-254.
10. Set Server Mask to the netmask you just noted.
11. Set Upper/Lower Address Bound to a range of IPs on the same subnet. We recommend using the range X-254 where X is one higher than the IP you used for the DHCP server itself. E.g., if your DHCP server is 192.168.100.2, you can set the lower and upper bounds to 192.168.100.3 and 192.168.100.254, respectively.

Configure UniK daemon

UniK configuration files are stored in $HOME/.unik. Create this directory, if it is not present:

$mkdir $HOME/.unik

Using a text editor, create and save the following to $HOME/.unik/daemon-config.yaml:

providers:
  virtualbox:
    - name: my-vbox
      adapter_type: host_only
      adapter_name: NEW_HOST_ONLY_ADAPTER

Replacing NEW_HOST_ONLY_ADAPTER with the name of the network adapter you created.

Launch UniK and automatically deploy the Virtualbox Instance Listener

• Open a new terminal window/tab. This terminal will be where we leave the UniK daemon running.
• cd to the _build directory created by make
• run unik daemon --debug (the --debug flag is optional, if you want to see more verbose output)
• UniK will compile and deploy its own 30 MB unikernel. This unikernel is the Unik Instance Listener. The Instance Listener uses udp broadcast to detect (the IP address) and bootstrap instances running on Virtualbox.
• After this is finished, UniK is running and ready to accept commands.
• Open a new terminal window and type unik target --host localhost to set the CLI target to the your local machine.

Write a Go HTTP server

Open a new terminal window, but leave the window with the daemon running. This window will be used for running UniK CLI commands.

1. Create httpd.go file

Create a file httpd.go using a text editor. Copy and paste the following code in that file:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "my first unikernel!")
}

2. Run the code

Try running this code with go run httpd.go. Visit http://localhost:8080/ to see that the server is running.

3. Create a dummy Godeps file

We need to create a dummy Godeps file. This is necessary to tell the Go compiler how Go projects and their dependencies are structured. Fortunately, with this example, our project has no dependencies, and we can just fill out a simple Godeps file without installing godep.

Note: For Go projects with imported dependencies, and nested packages, you will need to install Godeps and run GO15VENDOREXPERIMENT=1 godep save ./... in your project. See Compiling Go Apps with UniK for more information.

To create the dummy Godeps file, create a folder named Godeps in the same directory as httpd.go. Inside, create a file named Godeps.json and paste the following inside:

{
	"ImportPath": "my_httpd",
	"GoVersion": "go1.6",
	"GodepVersion": "v63",
	"Packages": [
		"./.."
	],
	"Deps": [
		{
			"ImportPath": "github.com/solo-io/unik/docs/examples",
			"Rev": "f8cc0dd435de36377eac060c93481cc9f3ae9688"
		}
	]
}

For the purposes of this example, what matters here is my_httpd. It instructs the Go compiler that the project should be installed from $GOPATH/src/my_httpd.

4. Great! Now we're ready to compile this code to a unikernel.

Compile an image to run on VirtualBox

1. Compile sources

Run the following command from the directory where your httpd.go is located:

unik build --name myImage --path ./ --base rump --language go --provider virtualbox

This command will instruct UniK to compile the sources found in the working directory (./) using the rump-go-virtualbox compiler.

2. Watch output

You can watch the output of the build command in the terminal window running the daemon.

3. Locate disk image

When build finishes, the resulting disk image will reside at $HOME/.unik/virtualbox/images/myImage/boot.vmdk

4. Run instance of disk image

Run an instance of this image with:

unik run --instanceName myInstance --imageName myImage

5. Check IP

When the instance finishes launching, let's check its IP and see if it is running our application. Run unik instances. The instance IP Address should be listed.

6. View the instance!

Direct your browser to http://instance-ip:8080 and see that your instance is running!

Finishing up

To clean up your image and the instance you created

unik rmi --force --image myImage

And we're done. We hope you will benefit from this post and tutorial and stay tuned for future posts!

Intro

Having a proper backup recovery plan is vital to any organization's IT operation. However, when you begin to distribute workloads across data centers and regions, that process begins to become more and more complex. Container orchestration platforms such as Kubernetes have begun to ease this burden and enabled the management of distributed workloads in areas that were previously very challenging.

In this post, we are going to introduce you to a Kubernetes-native tool for taking backups of your disks, helping with the crucial recovery plan. Stash is a Restic Operator that accelerates the task of backing up and recovering your Kubernetes infrastructure. You can read more about the Operator Framework via this blog post.

How does Stash work?

Using Stash, you can backup Kubernetes volumes mounted in following types of workloads:

• Deployment
• DaemonSet
• ReplicaSet
• ReplicationController
• StatefulSet

At the heart of Stash is a Kubernetes controller which uses Custom Resource Definition (CRD) to specify targets and behaviors of the backup and restore process in a Kubernetes native way. A simplified architecture of Stash is shown below:

Installing Stash

Using Helm 3

Stash can be installed via Helm using the chart from AppsCode Charts Repository. To install the chart with the release name stash-operator:

$ helm repo add appscode https://charts.appscode.com/stable/
$ helm repo update
$ helm search repo appscode/stash --version v0.9.0-rc.6
NAME            CHART          VERSION      APP VERSION DESCRIPTION
appscode/stash  v0.9.0-rc.6    v0.9.0-rc.6  Stash by AppsCode - Backup your Kubernetes Volumes

$ helm install stash-operator appscode/stash \
  --version v0.9.0-rc.6 \
  --namespace kube-system

Using YAML

If you prefer to not use Helm, you can generate YAMLs from Stash chart and deploy using kubectl:

$ helm repo add appscode https://charts.appscode.com/stable/
$ helm repo update
$ helm search repo appscode/stash --version v0.9.0-rc.6
NAME            CHART VERSION APP VERSION DESCRIPTION
appscode/stash  v0.9.0-rc.6    v0.9.0-rc.6  Stash by AppsCode - Backup your Kubernetes Volumes

$ helm template stash-operator appscode/stash \
  --version v0.9.0-rc.6 \
  --namespace kube-system \
  --no-hooks | kubectl apply -f -

Installing on GKE Cluster

If you are installing Stash on a GKE cluster, you will need cluster admin permissions to install Stash operator. Run the following command to grant admin permission to the cluster.

$ kubectl create clusterrolebinding "cluster-admin-$(whoami)" \
  --clusterrole=cluster-admin \
  --user="$(gcloud config get-value core/account)"

In addition, if your GKE cluster is a private cluster, you will need to either add an additional firewall rule that allows master nodes access port 8443/tcp on worker nodes, or change the existing rule that allows access to ports 443/tcp and 10250/tcp to also allow access to port 8443/tcp. The procedure to add or modify firewall rules is described in the official GKE documentation for private clusters mentioned above.

Verify installation

To check if Stash operator pods have started, run the following command:

$ kubectl get pods --all-namespaces -l app=stash --watch

NAMESPACE     NAME                              READY     STATUS    RESTARTS   AGE
kube-system   stash-operator-859d6bdb56-m9br5   2/2       Running   2          5s

Once the operator pods are running, you can cancel the above command by typing Ctrl+C.

Now, to confirm CRD groups have been registered by the operator, run the following command:

$ kubectl get crd -l app=stash

NAME                                 AGE
recoveries.stash.appscode.com        5s
repositories.stash.appscode.com      5s
restics.stash.appscode.com           5s

With this, you are ready to take your first backup using Stash.

Configuring Auto Backup for Database

To keep everything isolated, we are going to use a separate namespace called demo throughout this tutorial.

$ kubectl create ns demo
namespace/demo created

Prepare Backup Blueprint

We are going to use GCS Backend to store the backed up data. You can use any supported backend you prefer. You just have to configure Storage Secret and spec.backend section of BackupBlueprint to match with your backend. Visit here to learn which backends are supported by Stash and how to configure them.

For GCS backend, if the bucket does not exist, Stash needs Storage Object Admin role permissions to create the bucket. For more details, please check the following guide.

Create Storage Secret:

At first, let’s create a Storage Secret for the GCS backend,

$ echo -n 'changeit' > RESTIC_PASSWORD
$ echo -n '<your-project-id>' > GOOGLE_PROJECT_ID
$ mv downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY
$ kubectl create secret generic -n demo gcs-secret \
    --from-file=./RESTIC_PASSWORD \
    --from-file=./GOOGLE_PROJECT_ID \
    --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY
secret/gcs-secret created

Create BackupBlueprint:

Next, we have to create a BackupBlueprint CRD with a blueprint for Repository and BackupConfiguration object.

Below is the YAML of the BackupBlueprint object that we are going to create:

apiVersion: stash.appscode.com/v1beta1
kind: BackupBlueprint
metadata:
  name: postgres-backup-blueprint
spec:
  # ============== Blueprint for Repository ==========================
  backend:
    gcs:
      bucket: appscode-qa
      prefix: stash-backup/${TARGET_NAMESPACE}/${TARGET_APP_RESOURCE}/${TARGET_NAME}
    storageSecretName: gcs-secret
  # ============== Blueprint for BackupConfiguration =================
  task:
    name: postgres-backup-${TARGET_APP_VERSION}
  schedule: "*/5 * * * *"
  retentionPolicy:
    name: 'keep-last-5'
    keepLast: 5
    prune: true

Note that we have used few variables (format: ${<variable name>}) in the spec.backend.gcs.prefix field. Stash will substitute these variables with values from the respective target. To learn which variables you can use in the prefix field, please visit here.

Let’s create the BackupBlueprint that we have shown above.

$ kubectl apply -f https://github.com/stashed/docs/raw/v0.9.0-rc.6/docs/examples/guides/latest/auto-backup/database/backupblueprint.yaml
backupblueprint.stash.appscode.com/postgres-backup-blueprint created

With this, automatic backup is configured for PostgreSQL database. We just have to add an annotation to the AppBinding of the targeted database.

Required Annotation for Auto-Backup Database:

You have to add the following annotation to the AppBinding CRD of the targeted database to enable backup for it:

stash.appscode.com/backup-blueprint: <BackupBlueprint name>

This annotation specifies the name of the BackupBlueprint object where a blueprint for Repository and BackupConfiguration has been defined.

Prepare Databases

Next, we are going to deploy two sample PostgreSQL databases of two different versions using KubeDB. We are going to backup these two databases using auto-backup.

Deploy First PostgreSQL Sample:

Below is the YAML of the first Postgres CRD:

apiVersion: kubedb.com/v1alpha1
kind: Postgres
metadata:
  name: sample-postgres-1
  namespace: demo
spec:
  version: "11.2"
  storageType: Durable
  storage:
    storageClassName: "standard"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  terminationPolicy: Delete

Let’s create the Postgres we have shown above:

$ kubectl apply -f https://github.com/stashed/docs/raw/v0.9.0-rc.6/docs/examples/guides/latest/auto-backup/database/sample-postgres-1.yaml
postgres.kubedb.com/sample-postgres-1 created

KubeDB will deploy a PostgreSQL database according to the above specification and it will create the necessary secrets and services to access the database. It will also create an AppBinding CRD that holds the necessary information to connect with the database.

Verify that an AppBinding has been created for this PostgreSQL sample:

$ kubectl get appbinding -n demo
NAME                AGE
sample-postgres-1   47s

If you view the YAML of this AppBinding, you will see it holds service and secret information. Stash uses this information to connect with the database.

$ kubectl get appbinding -n demo sample-postgres-1 -o yaml

apiVersion: appcatalog.appscode.com/v1alpha1
kind: AppBinding
metadata:
  name: sample-postgres-1
  namespace: demo
  ...
spec:
  clientConfig:
    service:
      name: sample-postgres-1
      path: /
      port: 5432
      query: sslmode=disable
      scheme: postgresql
  secret:
    name: sample-postgres-1-auth
  secretTransforms:
  - renameKey:
      from: POSTGRES_USER
      to: username
  - renameKey:
      from: POSTGRES_PASSWORD
      to: password
  type: kubedb.com/postgres
  version: "11.2"

Deploy Second PostgreSQL Sample:

Below is the YAML of the second Postgres object:

apiVersion: kubedb.com/v1alpha1
kind: Postgres
metadata:
  name: sample-postgres-2
  namespace: demo
spec:
  version: "10.6-v2"
  storageType: Durable
  storage:
    storageClassName: "standard"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 1Gi
  terminationPolicy: Delete

Let’s create the Postgres we have shown above.

$ kubectl apply -f https://github.com/stashed/docs/raw/v0.9.0-rc.6/docs/examples/guides/latest/auto-backup/database/sample-postgres-2.yaml
postgres.kubedb.com/sample-postgres-2 created

Verify that an AppBinding has been created for this PostgreSQL database:

$ kubectl get appbinding -n demo
NAME                AGE
sample-postgres-1   2m49s
sample-postgres-2   10s

Here, we can see AppBinding sample-postgres-2 has been created for our second PostgreSQL sample.

Backup

Next, we are going to add auto-backup specific annotation to the AppBinding of our desired database. Stash watches for AppBinding CRD. Once it finds an AppBinding with auto-backup annotation, it will create a Repository and a BackupConfiguration CRD according to respective BackupBlueprint. Then, rest of the backup process will proceed as normal database backup as described here.

Backup First PostgreSQL Sample

Let’s backup our first PostgreSQL sample using auto-backup.

Add Annotations:

At first, add the auto-backup specific annotation to the AppBinding sample-postgres-1:

$ kubectl annotate appbinding sample-postgres-1 -n demo --overwrite \
  stash.appscode.com/backup-blueprint=postgres-backup-blueprint

Verify that the annotation has been added successfully:

$ kubectl get appbinding -n demo sample-postgres-1 -o yaml

apiVersion: appcatalog.appscode.com/v1alpha1
kind: AppBinding
metadata:
  annotations:
    stash.appscode.com/backup-blueprint: postgres-backup-blueprint
  name: sample-postgres-1
  namespace: demo
  ...
spec:
  clientConfig:
    service:
      name: sample-postgres-1
      path: /
      port: 5432
      query: sslmode=disable
      scheme: postgresql
  secret:
    name: sample-postgres-1-auth
  secretTransforms:
  - renameKey:
      from: POSTGRES_USER
      to: username
  - renameKey:
      from: POSTGRES_PASSWORD
      to: password
  type: kubedb.com/postgres
  version: "11.2"

Following this, Stash will create a Repository and a BackupConfiguration CRD according to the blueprint.

Verify Repository:

Verify that the Repository has been created successfully by the following command:

$ kubectl get repository -n demo
NAME                         INTEGRITY   SIZE   SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-1                                                                2m23s

If we view the YAML of this Repository, we are going to see that the variables ${TARGET_NAMESPACE}, ${TARGET_APP_RESOURCE} and ${TARGET_NAME} has been replaced by demo, postgres and sample-postgres-1 respectively.

$ kubectl get repository -n demo postgres-sample-postgres-1 -o yaml

apiVersion: stash.appscode.com/v1beta1
kind: Repository
metadata:
  creationTimestamp: "2019-08-01T13:54:48Z"
  finalizers:
  - stash
  generation: 1
  name: postgres-sample-postgres-1
  namespace: demo
  resourceVersion: "50171"
  selfLink: /apis/stash.appscode.com/v1beta1/namespaces/demo/repositories/postgres-sample-postgres-1
  uid: ed49dde4-b463-11e9-a6a0-080027aded7e
spec:
  backend:
    gcs:
      bucket: appscode-qa
      prefix: stash-backup/demo/postgres/sample-postgres-1
    storageSecretName: gcs-secret

Verify BackupConfiguration:

Verify that the BackupConfiguration CRD has been created by the following command:

$ kubectl get backupconfiguration -n demo
NAME                         TASK                   SCHEDULE      PAUSED   AGE
postgres-sample-postgres-1   postgres-backup-11.2   */5 * * * *            3m39s

Notice the TASK field. It denotes that this backup will be performed using postgres-backup-11.2 task. We had specified postgres-backup-${TARGET_APP_VERSION} as task name in the BackupBlueprint. Here, the variable ${TARGET_APP_VERSION} has been substituted by the database version.

Let’s check the YAML of this BackupConfiguration.

$ kubectl get backupconfiguration -n demo postgres-sample-postgres-1 -o yaml

apiVersion: stash.appscode.com/v1beta1
kind: BackupConfiguration
metadata:
  creationTimestamp: "2019-08-01T13:54:48Z"
  finalizers:
  - stash.appscode.com
  generation: 1
  name: postgres-sample-postgres-1
  namespace: demo
  ownerReferences:
  - apiVersion: v1
    blockOwnerDeletion: false
    kind: AppBinding
    name: sample-postgres-1
    uid: a799156e-b463-11e9-a6a0-080027aded7e
  resourceVersion: "50170"
  selfLink: /apis/stash.appscode.com/v1beta1/namespaces/demo/backupconfigurations/postgres-sample-postgres-1
  uid: ed4bd257-b463-11e9-a6a0-080027aded7e
spec:
  repository:
    name: postgres-sample-postgres-1
  retentionPolicy:
    keepLast: 5
    name: keep-last-5
    prune: true
  runtimeSettings: {}
  schedule: '*/5 * * * *'
  target:
    ref:
      apiVersion: v1
      kind: AppBinding
      name: sample-postgres-1
  task:
    name: postgres-backup-11.2
  tempDir: {}

Notice that the spec.target.ref is pointing to the AppBinding sample-postgres-1 that we have just annotated with auto-backup annotation.

Wait for BackupSession:

Now, wait for the next backup schedule. Run the following command to watch BackupSession CRD:

$ watch -n 1 kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-1

Every 1.0s: kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-1  workstation: Thu Aug  1 20:35:43 2019

NAME                                    INVOKER-TYPE          INVOKER-NAME                 PHASE       AGE
postgres-sample-postgres-1-1564670101   BackupConfiguration   postgres-sample-postgres-1   Succeeded   42s

Note: Backup CronJob creates BackupSession CRD with the following label stash.appscode.com/backup-configuration=<BackupConfiguration crd name>. We can use this label to watch only the BackupSession of our desired BackupConfiguration.

Verify Backup:

When backup session is completed, Stash will update the respective Repository to reflect the latest state of backed up data.

Run the following command to check if a snapshot has been sent to the backend:

$ kubectl get repository -n demo postgres-sample-postgres-1
NAME                         INTEGRITY   SIZE        SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-1   true        1.324 KiB   1                73s                      6m7s

If we navigate to stash-backup/demo/postgres/sample-postgres-1 directory of our GCS bucket, we are going to see that the snapshot has been stored there.

Backup Second Sample PostgreSQL

Now, lets backup our second PostgreSQL sample using the same BackupBlueprint we have used to backup the first PostgreSQL sample.

Add Annotations:

Add the auto backup specific annotation to AppBinding sample-postgres-2.

$ kubectl annotate appbinding sample-postgres-2 -n demo --overwrite \
  stash.appscode.com/backup-blueprint=postgres-backup-blueprint

Verify Repository:

Verify that the Repository has been created successfully by the following command:

$ kubectl get repository -n demo
NAME                         INTEGRITY   SIZE        SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-1   true        1.324 KiB   1                2m3s                     6m57s
postgres-sample-postgres-2                                                                     15s

Here, repository postgres-sample-postgres-2 has been created for the second PostgreSQL sample.

If we view the YAML of this Repository, we will see that the variables ${TARGET_NAMESPACE}, ${TARGET_APP_RESOURCE} and ${TARGET_NAME} have been replaced by demo, postgres and sample-postgres-2 respectively.

$ kubectl get repository -n demo postgres-sample-postgres-2 -o yaml

apiVersion: stash.appscode.com/v1beta1
kind: Repository
metadata:
  creationTimestamp: "2019-08-01T14:37:22Z"
  finalizers:
  - stash
  generation: 1
  name: postgres-sample-postgres-2
  namespace: demo
  resourceVersion: "56103"
  selfLink: /apis/stash.appscode.com/v1beta1/namespaces/demo/repositories/postgres-sample-postgres-2
  uid: df58523c-b469-11e9-a6a0-080027aded7e
spec:
  backend:
    gcs:
      bucket: appscode-qa
      prefix: stash-backup/demo/postgres/sample-postgres-2
    storageSecretName: gcs-secret

Verify BackupConfiguration:

Verify that the BackupConfiguration CRD has been created by the following command:

$ kubectl get backupconfiguration -n demo
NAME                         TASK                   SCHEDULE      PAUSED   AGE
postgres-sample-postgres-1   postgres-backup-11.2   */5 * * * *            7m52s
postgres-sample-postgres-2   postgres-backup-10.6   */5 * * * *            70s

Again, notice the TASK field. This time, ${TARGET_APP_VERSION} has been replaced with 10.6 which is the database version of our second sample.

Wait for BackupSession:

Now, wait for the next backup schedule. Run the following command to watch BackupSession CRD:

$ watch -n 1 kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-2
Every 1.0s: kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=postgres-sample-postgres-2  workstation: Thu Aug  1 20:55:40 2019

NAME                                    INVOKER-TYPE          INVOKER-NAME                 PHASE       AGE
postgres-sample-postgres-2-1564671303   BackupConfiguration   postgres-sample-postgres-2   Succeeded   37s

Verify Backup:

Run the following command to check if a snapshot has been sent to the backend:

$ kubectl get repository -n demo postgres-sample-postgres-2
NAME                         INTEGRITY   SIZE        SNAPSHOT-COUNT   LAST-SUCCESSFUL-BACKUP   AGE
postgres-sample-postgres-2   true        1.324 KiB   1                52s                      19m

If we navigate to stash-backup/demo/postgres/sample-postgres-2 directory of our GCS bucket, we are going to see that the snapshot has been stored there.

Cleanup

To cleanup the Kubernetes resources created by this tutorial, run:

kubectl delete -n demo pg/sample-postgres-1
kubectl delete -n demo pg/sample-postgres-2

kubectl delete -n demo repository/postgres-sample-postgres-1
kubectl delete -n demo repository/postgres-sample-postgres-2

kubectl delete -n demo backupblueprint/postgres-backup-blueprint

Final thoughts

You've now gotten a deep dive into setting up a Kubernetes-native disaster recovery and backup solution with Stash. You can find a lot of really helpful information on their documentation site here. I hope you gained some educational knowledge from this post and will stay tuned for future tutorials!

Kubernetes is a powerful orchestration system, however, it can be really hard to configure its deployment process. Specific apps can help you manage multiple independent resources like pods, services, deployments, and replica sets. Yet, each must be described in the YAML manifest file.

It’s not a problem for a single trivial app, but during production, it’s best to simplify this process: search, use, and share already implemented configurations, deploy these configurations, create configuration templates, and deploy them without effort. In other words, we need an extended version of a package manager like APT for Ubuntu or PIP for Python to work with the Kubernetes cluster. Luckily, we have Helm as a package manager.

What is Helm?

Helm is an open-source package manager for Kubernetes that allows developers and operators to package, configure, and deploy applications and services onto Kubernetes clusters easily. It was inspired by Homebrew for macOS and now is a part of the Cloud Native Computing Foundation.

In this article, we will explore Helm 3.x which is the newest version at the time of writing this article.

Searches on Helm Hub for PostgreSQL from dozens of different repositories

Helm can install software and dependencies, upgrade software, configure software deployments, fetch packages from repositories, alongside managing repositories.

Some key features of Helm include:

• Role-based access controls (RBAC)
• Golang templates which allows you to work with configuration as text
• Lua scripts to process configuration as an object
• Deployment versions control system
  

Templates allow you to configure your deployments by changing few variable values without changing the template directly. Helm packages are called charts, and they consist of a few YAML configuration files and templates that are rendered into Kubernetes manifest files.

The basic package (chart) structure:

• chart.yaml - a YAML file containing information about the chart
• LICENSE (optional) - a plain text file containing the license for the chart
• README.md (optional) - a human-readable README file
• values.yaml - the default configuration values for this chart
• values.schema.json (optional) - a JSON Schema for imposing a structure on the values.yaml file
• charts/ - defines chart dependencies (recommended to use the dependencies section in chart.yaml)
• crds/ - Custom Resource Definitions
• templates/ - directory of templates that when combined with values, will generate valid Kubernetes manifest files
  

Templates give you a wide range of capabilities. You can use variables from context, apply different functions (such as ‘quote’, sha256sum), use cycles and conditional cases, and import other files (also other templates or partials).

What are Helm’s abilities?

1. As you operate Helm though a Command Line Interface (CLI), the helm search command allows you to search for a package by keywords from the repositories.
2. You can inspect chart.yaml, values.yaml, and README.md for a certain package. along with creating your own chart with the helm create <chart-name> command. This command will generate a folder with a specified name in which you can find the mentioned structure.
3. Helm can install both folder or .tgz archives. To create a .tgz from your package folder, use the helm package <path to folder> command. This will create a <package_name> package in your working directory, using the name and version from the metadata defined in the chart.yaml file.
4. Helm has built-in support for installing packages from an HTTP server. Helm reads a repository index hosted on the server, which describes what chart packages are available and where they are located. This is how the default stable repository works.
5. You can also create a repository from your machine with helm serve. This eventually lets you create your own corporate repository or contribute to the official stable one.
6. You can also call the helm dependencies update <package name> command which verifies that the required charts, as expressed in chart.yaml, are present in charts/ and are in an acceptable version. It will additionally pull down the latest charts that satisfy the dependencies, and clean up the old dependencies.
7. Apart from Chart and Repository another significant concept you should know is Release which is an instance of a chart running in a Kubernetes cluster. One chart can often be installed many times into the same cluster. And each time it is installed, a new Release is created. So, you can have multiple PostgreSQL in the same cluster, in which each Release will have its own release name. You can think of this like 'multiple Docker containers from one image'.
   

How does it work?

Source: developer.ibm.com

Helm client is used for installing, updating and creating charts, as well as compiling and sending them to a Kubernetes API in an acceptable form. The previous version had a client-server architecture, using a program run on a cluster with Kubernetes, called Tiller. This software was responsible for deployment’s lifetime. But this approach led to some security issues which is one of the reasons why all functions are now handled by the client.

Installing Helm 3 is noticeably easier than the previous version since only the client needs to be installed. It is available for Windows, macOS, and Linux. You can install the program from binary releases, Homebrew, or through a configured installation script.

Let’s try an example

1. Let's start with installing Helm.

bash master $ curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100  6794  100  6794    0     0  25961      0 --:--:-- --:--:-- --:--:-- 25931Error: could not find tillerHelm v3.1.2 is available. Changing from version .Downloading https://get.helm.sh/helm-v3.1.2-linux-amd64.tar.gzPreparing to install helm into /usr/local/binhelm installed into /usr/local/bin/helm

2. Check if everything is installed properly.

master $ helm version --short
V3.1.2+gd878d4d

3. By default, Helm doesn’t have a connection to any of the repositories. Let’s add connection to the most common stable one. (You can check all the available repositories with helm repo list).

master $ helm repo add stable 

https://kubernetes-charts.storage.googleapis.com/
"stable" has been added to your repositories

4. After adding the repository, we should let Helm get updated. The current local state of Helm is kept in your environment in the home location.

master $ helm repo update

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈

The Helm command defaults to discovering the host already set in ~/.kube/config. There is a way to change or override the host, but that's beyond the scope of this scenario.

master $ helm env

HELM_BIN="helm"
HELM_DEBUG="false"
HELM_KUBECONTEXT=""
HELM_NAMESPACE="default"
HELM_PLUGINS="/root/.local/share/helm/plugins"
HELM_REGISTRY_CONFIG="/root/.config/helm/registry.json"
HELM_REPOSITORY_CACHE="/root/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/root/.config/helm/repositories.yaml"

5. Let's search for a WordPress in the Helm Hub

master $ helm search hub wordpress

URL                                                     CHART VERSION   APP VERSION     DESCRIPTION https://hub.helm.sh/charts/presslabs/wordpress-...      v0.8.4          v0.8.4          Presslabs WordPress Operator Helm Chart
https://hub.helm.sh/charts/presslabs/wordpress-...      v0.8.3          v0.8.3          A Helm chart for deploying a WordPress site on ...
https://hub.helm.sh/charts/bitnami/wordpress            9.0.3           5.3.2           Web publishing platform for building blogs and ...

And also search in our repositories (we have only stable for now).

master $ helm search repo wordpress

NAME                    CHART VERSION   APP VERSION     DESCRIPTION
stable/wordpress        9.0.2           5.3.2           DEPRECATED Web publishing platform for building...

6. As mentioned earlier, you can inspect a Chart. For example, let’s take info from chart.yaml for the Wordpress chart.
You can also check helm show readme stable/wordpress and helm show values stable/wordpress.

master $ helm show chart stable/wordpress

apiVersion: v1
appVersion: 5.3.2
dependencies:
- condition: mariadb.enabled
  name: mariadb
  repository: https://kubernetes-charts.storage.googleapis.com/
  tags:
  - wordpress-database
  version: 7.x.xdeprecated: truedescription: DEPRECATED Web publishing platform for building blogs and websites.
home: http://www.wordpress.com/
icon: https://bitnami.com/assets/stacks/wordpress/img/wordpress-stack-220x234.png
keywords:- wordpress- cms
- blog
- http- web- application
- php
name: wordpress
sources:
- https://github.com/bitnami/bitnami-docker-wordpress
version: 9.0.2

7. Let’s create a namespace for WordPress and install a test chart.

master $ kubectl create namespace wordpress

namespace/wordpress created

master $ helm install test-wordpress stable/wordpress --namespace wordpress

The output of this command appears messy just because it’s so big.

You can also set variables, such as:

helm install test-wordpress \
  --set wordpressUsername=admin \
  --set wordpressPassword=password \
  --set mariadb.mariadbRootPassword=secretpassword \
    stable/wordpress

8. For now, let’s ensure that everything is deployed correctly:

As you can see, everything has been deployed properly.

Conclusion

Helm is a popular open-source package manager that offers users a more flexible way to manage Kubernetes cluster. You can either create your own, or use public packages from your own or external repositories. Each package is quite flexible and, in most cases, all you need is define the right constants from which the template will be compiled to suit your needs. To create your own chart, you can use the power of Go templates and/or Lua scripts. Each update will create a history unit to which you can rollback anytime you want. With Helm, you have all the power of Kubernetes. And, in the end, Helm allows you to work with role-based access, so you can manage your cluster in a team.

This brings us to the end of this brief article explaining the basics and features of Helm. We hope you enjoyed it and were able to make use of it.

Amazon Elastic Container Service for Kubernetes or EKS provides a Managed Kubernetes Service. Amazon does the undifferentiated heavy lifting, such as provisioning the cluster, performing upgrades and patching. Although it is compatible with existing plugins and tooling, EKS is not a proprietary AWS fork of Kubernetes in any way. This means you can easily migrate any standard Kubernetes application to EKS without any changes to your code base. You'll connect to your EKS cluster with kubectl in the same way you would have done in a self-hosted Kubernetes.

At this stage, EKS is very loosely integrated with other AWS services. This is definitely expected to change over time though, as EKS adoption increases. That, said, Kubernetes is much more popular than either Elastic Beanstalk or ECS.

Managed Control Plane

EKS provides a Managed Control Plane, which includes Kubernetes master nodes, API server and the etcd persistence layer. As part of the highly-available control plane, you get 3 masters, 3 etcd and 3 worker nodes, where AWS provisions automatic backup snapshotting of etcd nodes alongside automated scaling.

With EKS, AWS is responsible for maintaining master nodes for you by provisioning these nodes in multiple high-availability zones to maintain redundancy. So, as your workload increases, AWS will add master nodes for you. If you were running your own Kubernetes cluster, you'd have to scale it up whenever you added a worker node.  

VPC Networking

EKS runs a network topology that integrates tightly with a Virtual Private Network. EKS uses a Container Network Interface plugin that integrates the standard Kubernetes overlay network with VPC networking. This plugin allows you to treat your EKS deployment as just another part of your existing AWS infrastructure. Things like network access, control list, routing tables and subnets are all available in the Kubernetes applications running in EKS.

Each pod gets an IP address on an Elastic Network Interface, where these addresses belong to the block of the subnet where the worker node is deployed. In the diagram above, you can see the IP addresses assigned to the Virtual Ethernet Adapter on each pod. These pod IP addresses are fully rideable within the VPC, and they comply with all the policies and access controls at the network level. So, things like security groups and ACL remain in effect. On each EC2 instance or worker node, Kubernetes runs a daemon set that hosts the CNI plugin. This plugin is a thin layer that communicates with the network local control point. This network local control plane maintains a pool of available IP addresses. So, when the kubelet on a node schedules a pod, it asks the CNI plugin to allocate an IP address. At this point, the CNI plugin allocates an IP, grabs secondary IP address and associates it with the pod. It then hands that configuration back to the kubelet.

EKS-Optimized AMI

This thing is based on Amazon Linux too. It comes pre-configured to work with EKS out-of-the-box. It has all the required services pre-installed including Docker, the Kubelet and AWS IAM Authenticator. When you are provisioning your EKS worker nodes with the AWS supplied cloud formation template, it launches your worker nodes with some EC2 user data script which bootstraps the nodes with configuration allowing them to join your EKS cluster automatically.  

Conclusion

Amazon EKS on AWS provides a great opportunity to create a self-hosted managed Kubernetes cluster. It is also compatible with open-source Kubernetes, and can be safely migrated to any other Kubernetes instance at any time. Worth to mention, that for users who use solutions for centralized management of Kubernetes clusters, it makes sense to go with EKS instead of any other option such as ECS, since EKS exposes the same API as an open-source Kubernetes.

In this article we are going to consider the two most common methods for Autoscaling in EKS cluster:

• Horizontal Pod Autoscaler (HPA)
• Cluster Autoscaler (CA)

The Horizontal Pod Autoscaler or HPA is a Kubernetes component that automatically scales your service based on metrics such as CPU utilization or others, as defined through the Kubernetes metric server. The HPA scales the pods in either a deployment or replica set, and is implemented as a Kubernetes API resource and a controller. The Controller Manager queries the resource utilization against the metrics specified in each horizontal pod autoscaler definition. It obtains the metrics from either the resource metrics API for per pod metrics or the custom metrics API for any other metrics.

To see this in action, we are going to configure HPA and then apply some load to our system to see it in action.

To start with, let us start with installing Helm as a package manager for Kubernetes.

curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > helm.sh
 chmod +x helm.sh
 ./helm.sh

Now, we are going to set up the server base portion of Helm called Tiller. This requires a service account:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system
 

The above defines a Tiller service account to which we have assigned the cluster admin role. Now let's go ahead and apply the configuration:

kubectl apply -f tiller.yml

Run helm init using the Tiller service account we have just created:

helm init --service-account tiller

With this we have installed Tiller onto the cluster, which gives access to manage those resources within it.

With Helm installed, we can now deploy the metric server. Metric servers are cluster wide aggregators of resource usage data where metrics are collected by kubelet on each worker node, and are used to dictate the scaling behavior of deployments.

So let's go ahead and install that now:

helm install stable/metrics-server --name metrics-server --version 2.0.4 --namespace metrics

Once all checks have passed, we are ready to scale the application.

For the purpose of this article, we will deploy a special build of Apache and PHP designed to generate CPU utilization:

kubectl run php-apache --image=k8s.gcr.io/hpa-example --requests=cpu=200m --expose --port=80

Now, let us autoscale our deployment:

kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10

The above specifies that the HPA will increase or decrease the number of replicas to maintain an average CPU utilization across all pods by 50%. Since each pod requests 200 millicores (as specified in the previous command), the average CPU utilization of 100 millicores is maintained.

Let's check the status:

kubectl get hpa

Review Targets column, if it says unknown/50% then it means that the current CPU consumption is 0%, as we are not currently sending any request to the server. This will take a couple of minutes to show the correct value, so let us grab a cup of coffee and come back when we have got some data here.

Rerun the last command and confirm that Targets column is now 0%/50%. Now, let's generate some load in order to trigger scaling by running the following :

kubectl run -i --tty load-generator --image=busybox /bin/sh

Inside this container, we are going to send an infinite number of requests to our service. If we flip back over to the other terminal, we can watch the autoscaler in action:

kubectl get hpa -w

We can watch the HPA scaler pod up from 1 to our configured maximum of 10, until the average CPU utilization is below our target of 50%. It will take about 10 minutes to run and you could see we are now having 10 replicas. If we flip back to the other terminal to terminate the load test, and flip back to the scaler terminal, we can see the HPA reduce the replica count back to the minimum.

Cluster Autoscaler

The Cluster Autoscaler is the default Kubernetes component that can scale either pods or nodes in a cluster. It automatically increases the size of an autoscaling group, so that pods can continue to get placed successfully. It also tries to remove unused worker nodes from the autoscaling group (the ones with no pods running).

The following AWS CLI command will create an Auto scaling group with minimum of one and maximum count of ten:

eksctl create nodegroup --cluster <CLUSTER_NAME> --node-zones <REGION_CODE> --name <REGION_CODE> --asg-access --nodes-min 1 --nodes 5 --nodes-max 10 --managed

Now, we need to apply an inline IAM policy to our worker nodes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:DescribeLaunchConfigurations",
                "autoscaling:DescribeTags",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:TerminateInstanceInAutoScalingGroup",
                "ec2:DescribeLaunchTemplateVersions"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

This basically allows the EC2 worker nodes posting the cluster auto scaler the ability to manipulate auto scaling. Copy it and add to your EC2 IAM role.

Next, download the following file:

wget https://raw.githubusercontent.com/kubernetes/autoscaler/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

And update the following line with your cluster name:

            - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/<YOUR CLUSTER NAME>

Finally, we can deploy our Autoscaler:

kubectl apply -f cluster-autoscaler-autodiscover.yaml

Of course we should wait for the pods to finish creating. Once done, we can scale our cluster out. We will consider a simple nginx application with the following yaml file:

apiVersion: extensions/v1beta2
kind: Deployment
metadata:
  name: nginx-scale
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        resources: 
          limits:
            cpu: 500m
            memory: 512Mi
          requests:
            cpu: 500m
            memory: 512Mi

Let's go ahead and deploy the application:

kubectl apply -f nginx.yaml

And check the deployment:

kubectl get deployment/nginx-scale

Now, let's scale a replica up to 10:

kubectl scale --replicas=10 deployment/nginx-scale

We can see our some pods in the pending state, which is the trigger that the cluster auto scaler uses to scale out our fleet of EC2 instances.

kubectl get pods -o wide --watch

Conclusion

In this article, we considered both types of EKS cluster autoscaling. We learnt how the Cluster Autoscaler initiates scale-in and scale-out operations each time it detects under-utilized instances or pending pods. Horizontal Pod Autoscaler and Cluster Autoscaler are essential features of Kubernetes when it comes to scaling a microservice application. Hope you found this article useful but there is more to come. Till then, happy scaling!

Intro

Organizations are increasingly looking to containers and distributed applications to provide the agility and scalability needed to satisfy their clients. While doing so, modern enterprises also need the ability to benchmark their application and be aware of certain metrics in relation to their infrastructure.

In this post, I am introducing you to a cloud-native bench-marking tool known as Kubestone. This tool is meant to assist your development teams with getting performance metrics from your Kubernetes clusters.

How does Kubestone work?

At it's core, Kubestone is implemented as a Kubernetes Operator in Go language with the help of Kubebuilder. You can find more info on the Operator Framework via this blog post.
Kubestone leverages Open Source benchmarks to measure Core Kubernetes and Application performance. As benchmarks are executed in Kubernetes, they must be containerized to work on the cluster. A certified set of benchmark containers is provided via xridge's DockerHub space. Here is a list of currently supported benchmarks:

Let's try installing Kubestone and running a benchmark ourselves and see how it works.

Installing Kubestone

Requirements

• Kubernetes v1.13 (or newer)
• Kustomize v3.1.0
• Cluster admin privileges

Deploy Kubestone to kubestone-system namespace with the following command:

$ kustomize build github.com/xridge/kubestone/config/default | kubectl create -f -

Once deployed, Kubestone will listen for Custom Resources created with the kubestone.xridge.io group.

Benchmarking

Benchmarks can be executed via Kubestone by creating Custom Resources in your cluster.

Namespace

It is recommended to create a dedicated namespace for benchmarking.

$ kubectl create namespace kubestone

After the namespace is created, you can use it to post a benchmark request to the cluster.

The resulting benchmark executions will reside in this namespace.

Custom Resource rendering

We will be using kustomize to render the Custom Resource from the github repository.

Kustomize takes a base yaml, and patches with an overlay file to render the final yaml file, which describes the benchmark.

$ kustomize build github.com/xridge/kubestone/config/samples/fio/overlays/pvc

The Custom Resource (rendered yaml) looks as follows:

apiVersion: perf.kubestone.xridge.io/v1alpha1
kind: Fio
metadata:
  name: fio-sample
spec:
  cmdLineArgs: --name=randwrite --iodepth=1 --rw=randwrite --bs=4m --size=256M
  image:
    name: xridge/fio:3.13
  volume:
    persistentVolumeClaimSpec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 1Gi
    volumeSource:
      persistentVolumeClaim:
        claimName: GENERATED

When we create this resource in Kubernetes, the operator interprets it and creates the associated benchmark. The fields of the Custom Resource controls how the benchmark will be executed:

• metadata.name: Identifies the Custom Resource. Later, this can be used to query or delete the benchmark in the cluster.
• cmdLineArgs: Arguments passed to the benchmark. In this case we are providing the arguments to Fio (a filesystem benchmark). It instructs the benchmark to execute a random write test with 4Mb of block size with an overall transfer size of 256 MB.
• image.name: Describes the Docker Image of the benchmark. In case of Fio, we are using xridge's fio Docker Image, which is built from this repository.
• volume.persistentVolumeClaimSpec: Given that Fio is a disk benchmark, we can set a PersistentVolumeClaim for the benchmark to be executed. The above setup instructs Kubernetes to take 1GB of space from the default StorageClass and use it for the benchmark.

Running the benchmark

Now, as we understand the definition of the benchmark, we can try to execute it.

Note: Make sure you installed the kubestone operator and have it running before executing this step.

$ kustomize build github.com/xridge/kubestone/config/samples/fio/overlays/pvc | kubectl create --namespace kubestone -f -

Since we pipe the output of the kustomize build command into kubectl create, it will create the object in our Kubernetes cluster.

The resulting object can be queried using the object's type (fio) and it's name (fio-sample):

$ kubectl describe --namespace kubestone fio fio-sample
Name:         fio-sample
Namespace:    kubestone
Labels:       <none>
Annotations:  <none>
API Version:  perf.kubestone.xridge.io/v1alpha1
Kind:         Fio
Metadata:
  Creation Timestamp:  2019-09-14T11:31:02Z
  Generation:          1
  Resource Version:    31488293
  Self Link:           /apis/perf.kubestone.xridge.io/v1alpha1/namespaces/kubestone/fios/fio-sample
  UID:                 21cdbe92-d6e3-11e9-ba70-4439c4920abc
Spec:
  Cmd Line Args:  --name=randwrite --iodepth=1 --rw=randwrite --bs=4m --size=256M
  Image:
    Name:  xridge/fio:3.13
  Volume:
    Persistent Volume Claim Spec:
      Access Modes:
        ReadWriteOnce
      Resources:
        Requests:
          Storage:  1Gi
    Volume Source:
      Persistent Volume Claim:
        Claim Name:  GENERATED
Status:
  Completed:  true
  Running:    false
Events:
  Type    Reason           Age   From       Message
  ----    ------           ----  ----       -------
  Normal  Created  11s   kubestone  Created /api/v1/namespaces/kubestone/configmaps/fio-sample
  Normal  Created  11s   kubestone  Created /api/v1/namespaces/kubestone/persistentvolumeclaims/fio-sample
  Normal  Created  11s   kubestone  Created /apis/batch/v1/namespaces/kubestone/jobs/fio-sample

As the Events section shows, Kubestone has created a ConfigMap, a PersistentVolumeClaim and aJob for the provided Custom Resource. The Status field tells us that the benchmark has completed.

Inspecting the benchmark

The created objects related to the benchmark can be listed using kubectl command:

$ kubectl get pods,jobs,configmaps,pvc --namespace kubestone
NAME                   READY   STATUS      RESTARTS   AGE
pod/fio-sample-bqqmm   0/1     Completed   0          54s

NAME                   COMPLETIONS   DURATION   AGE
job.batch/fio-sample   1/1           15s        54s

NAME                   DATA   AGE
configmap/fio-sample   0      54s

NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
persistentvolumeclaim/fio-sample   Bound    pvc-b3898236-c698-11e9-8071-4439c4920abc   1Gi        RWO            rook-ceph-block   54s

As shown above, Fio controller has created a PersistentVolumeClaim and a ConfigMap which is used by the Fio Job during benchmark execution. The Fio Job has an associated Pod which contains our test execution. The results of the run can be shown with the kubectl logs command:

$ kubectl logs --namespace kubestone fio-sample-bqqmm
randwrite: (g=0): rw=randwrite, bs=(R) 4096KiB-4096KiB, (W) 4096KiB-4096KiB, (T) 4096KiB-4096KiB, ioengine=psync, iodepth=1
fio-3.13
Starting 1 process
randwrite: Laying out IO file (1 file / 256MiB)

randwrite: (groupid=0, jobs=1): err= 0: pid=47: Sat Aug 24 17:58:10 2019
  write: IOPS=470, BW=1882MiB/s (1974MB/s)(256MiB/136msec); 0 zone resets
    clat (usec): min=1887, max=2595, avg=2042.76, stdev=136.56
     lat (usec): min=1953, max=2688, avg=2107.35, stdev=142.94
    clat percentiles (usec):
     |  1.00th=[ 1893],  5.00th=[ 1926], 10.00th=[ 1926], 20.00th=[ 1958],
     | 30.00th=[ 1991], 40.00th=[ 2008], 50.00th=[ 2024], 60.00th=[ 2040],
     | 70.00th=[ 2057], 80.00th=[ 2073], 90.00th=[ 2114], 95.00th=[ 2409],
     | 99.00th=[ 2606], 99.50th=[ 2606], 99.90th=[ 2606], 99.95th=[ 2606],
     | 99.99th=[ 2606]
  lat (msec)   : 2=34.38%, 4=65.62%
  cpu          : usr=2.22%, sys=97.78%, ctx=1, majf=0, minf=9
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,64,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=1882MiB/s (1974MB/s), 1882MiB/s-1882MiB/s (1974MB/s-1974MB/s), io=256MiB (268MB), run=136-136msec

Disk stats (read/write):
  rbd7: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

Listing benchmarks

We have learned that Kubestone uses Custom Resources to define benchmarks. We can list the installed custom resources using the kubectl get crds command:

$ kubectl get crds | grep kubestone
drills.perf.kubestone.xridge.io         2019-09-08T05:51:26Z
fios.perf.kubestone.xridge.io           2019-09-08T05:51:26Z
iopings.perf.kubestone.xridge.io        2019-09-08T05:51:26Z
iperf3s.perf.kubestone.xridge.io        2019-09-08T05:51:26Z
pgbenches.perf.kubestone.xridge.io      2019-09-08T05:51:26Z
sysbenches.perf.kubestone.xridge.io     2019-09-08T05:51:26Z

Using the CRD names above, we can list the executed benchmarks in the system.

Kubernetes provides a convenience feature regarding CRDs: one can use the shortened name of the CRD, which is the singular part of the fully qualified CRD name. In our case, fios.perf.kubestone.xridge.io can be shortened to fio. Hence, we can list the executed fio benchmark using the following command:

$ kubectl get --namespace kubestone fios.perf.kubestone.xridge.io
NAME         RUNNING   COMPLETED
fio-sample   false     true

Cleaning up

After a successful benchmark run the resulting objects are stored in the Kubernetes cluster. Given that Kubernetes can hold a limited number of pods in the system, it is advised that the user cleans up the benchmark runs time to time. This can be achieved by deleting the Custom Resource, which initiated the benchmark:

$ kubectl delete --namespace kubestone fio fio-sample

Since the Custom Resource has ownership on the created resources, the underlying pods, jobs, configmaps, pvcs, etc. are also removed by this operation.

Next steps

Now you are familiar with the key concepts of Kubestone, it is time to explore and benchmark. You can play around with Fio Benchmark via it's cmdLineArgs, Persistent Volume and Scheduling related settings. You can find more information about that in Fio's benchmark page. Hopefully you gained some valuable knowledge from this post!

Intro

There are many challenges that engineering teams face when attempting to incorporate a multi-cloud approach into their infrastructure goals. Kubernetes does a good job of addressing some of these issues, but managing the communication of clusters that span multiple cloud providers in multiple regions can become a daunting task for teams. Often this requires complex VPNs and special firewall rules to multi-cloud cluster communication.

In this post, I will be introducing you to Skupper, an open source project for enabling secure communication across Kubernetes cluster. Skupper allows your application to span multiple cloud providers, data centers, and regions. Let's see it in action!

Getting Started

This tutorial will demonstrate how to distribute the Istio Bookinfo Application microservices across multiple public and private clusters. The services require no coding changes to work in the distributed application environment. With Skupper, the application behaves as if all the services are running in the same cluster.

In this tutorial, you will deploy the productpage and ratings services on a remote, public cluster in namespace aws-eu-west and the details and reviews services in a local, on-premises cluster in namespace laptop.

Overview

Figure 1 - Bookinfo service deployment

The image above shows how the services will be deployed.

• Each cluster runs two of the application services.
• An ingress route to the productpage service provides internet user access to the application.

If all services were installed on the public cluster, then the application would work as originally designed. However, since two of the services are on the laptop cluster, the application fails. productpage can not send requests to details or to reviews.

This demo will show how Skupper can solve the connectivity problem presented by this arrangement of service deployments.

Figure 2 - Bookinfo service deployment with Skupper

Skupper is a distributed system with installations running in one or more clusters or namespaces. Connected Skupper installations share information about what services each installation exposes. Each Skupper installation learns which services are exposed on every other installation. Skupper then runs proxy service endpoints in each namespace to properly route requests to or from every exposed service.

• In the public namespace, the details and reviews proxies intercept requests for their services and forward them to the Skupper network.
• In the private namespace, the details and reviews proxies receive requests from the Skupper network and send them to the related service.
• In the private namespace, the ratings proxy intercepts requests for its service and forwards them to the Skupper network.
• In the public namespace, the ratings proxy receives requests from the Skupper network and sends them to the related service.

Prerequisites

To run this tutorial you will need:

• The kubectl command-line tool, version 1.15 or later (installation guide)
• The skupper command-line tool, the latest version (installation guide)
• Two Kubernetes namespaces, from any providers you choose, on any clusters you choose
• The yaml files from https://github.com/skupperproject/skupper-examples-bookinfo.git
• Two logged-in console terminals, one for each cluster or namespace

Step 1: Deploy the Bookinfo application

This step creates a service and a deployment for each of the four Bookinfo microservices.

Namespace aws-eu-west:

$ kubectl apply -f public-cloud.yaml
service/productpage created
deployment.extensions/productpage-v1 created
service/ratings created
deployment.extensions/ratings-v1 created

Namespace laptop:

$ kubectl apply -f private-cloud.yaml 
service/details created
deployment.extensions/details-v1 created
service/reviews created
deployment.extensions/reviews-v3 created

Step 2: Expose the public productpage service

Namespace aws-eu-west:

kubectl expose deployment/productpage-v1 --port 9080 --type LoadBalancer

The Bookinfo application is accessed from the public internet through this ingress port to the productpage service.

Step 3: Observe that the application does not work

The web address for the Bookinfo application can be discovered from namespace aws-eu-west:

$ echo $(kubectl get service/productpage -o jsonpath='http://{.status.loadBalancer.ingress[0].hostname}:9080')

Open the address in a web browser. Productpage responds but the page will show errors as services in namespace laptop are not reachable.

We can fix that now.

Step 4: Set up Skupper

This step initializes the Skupper environment on each cluster.

Namespace laptop:

skupper init

Namespace aws-eu-west:

skupper init

Now the Skupper infrastructure is running. Use skupper status in each console terminal to see that Skupper is available.

$ skupper status
Namespace '<ns>' is ready.  It is connected to 0 other namespaces.

As you move through the steps that follow, you can use skupper status at any time to check your progress.

Step 5: Connect your Skupper installations

Now you need to connect your namespaces with a Skupper connection. This is a two step process.

The skupper connection-token <file> command directs Skupper to generate a secret token file with certificates that grant permission to other Skupper instances to connect to this Skupper's network.

Note: Protect this file as you would do for any file that holds login credentials.

• The skupper connect <file> command directs Skupper to connect to another Skupper's network. This step completes the Skupper connection.

Note that in this arrangement the Skupper instances join to form peer networks. Typically the Skupper opening the network port will be on the public cluster. A cluster running on laptop may not even have an address that is reachable from the internet. After the connection is made, the Skupper network members are peers and it does not matter which Skupper opened the network port and which connected to it.

The console terminals in this demo are run by the same user on the same host. This makes the token file in the ${HOME} directory available to both terminals. If your terminals are on different machines then you may need to use scp or a similar tool to transfer the token file to the system hosting the laptop terminal.

Generate a Skupper network connection token

Namespace aws-eu-west:

skupper connection-token ${HOME}/PVT-to-PUB-connection-token.yaml

Open a Skupper connection

Namespace laptop:

skupper connect ${HOME}/PVT-to-PUB-connection-token.yaml

Check the connection

Namespace aws-eu-west:

$ skupper status
Skupper enabled for "aws-eu-west". It is connected to 1 other sites.

Namespace laptop:

$ skupper status
Skupper enabled for "laptop". It is connected to 1 other sites.

Step 6: Virtualize the services you want shared

You now have a Skupper network capable of multi-cluster communication but no services are associated with it. This step uses the kubectl annotate command to notify Skupper that a service is to be included in the Skupper network.

Skupper uses the annotation as the indication that a service must be virtualized. The service that receives the annotation is the physical target for network requests and the proxies that Skupper deploys in other namespaces are the virtual targets for network requests. The Skupper infrastructure then routes requests between the virtual services and the target service.

Namespace aws-eu-west:

$ kubectl annotate service ratings skupper.io/proxy=http
service/ratings annotated

Namespace laptop:

$ kubectl annotate service details skupper.io/proxy=http
service/details annotated

$ kubectl annotate service reviews skupper.io/proxy=http
service/reviews annotated

Skupper is now making the annotated services available to every namespace in the Skupper network. The Bookinfo application will work as the productpage service on the public cluster has access to the details and reviews services on the private cluster and as the reviews service on the private cluster has access to the ratings service on the public cluster.

Step 7: Observe that the application works

The web address for the Bookinfo app can be discovered from namespace aws-eu-west:

$ echo $(kubectl get service/productpage -o jsonpath='http://{.status.loadBalancer.ingress[0].hostname}:9080')

Open the address in a web browser. The application should now work with no errors.

Clean up

Skupper and the Bookinfo services may be removed from the clusters.

Namespace aws-eu-west:

skupper delete
kubectl delete -f public-cloud.yaml

Namespace laptop:

skupper delete
kubectl delete -f private-cloud.yaml 

Final Thoughts

Enabling a multi-cloud approach has a lot of benefits and is getting easier, thanks to tools like Skupper. If you have time, try some of  Skupper's other examples on its Github Repo. I hope you learned something from this post. Stay tuned for more!

The Ghost blogging platform offers a lean and minimalist experience. And that's why we love it. But unfortunately sometimes, it can be too lean for our requirements.

Web performance has become more important and relevant than ever, especially since Google started including it as a parameter in its SEO rankings. We make sure to optimize our websites as much as possible, offering the best possible user experience. This article will walk you through the steps you can take to optimize a Ghost Blog's performance while keeping it lean and resourceful.

When we started working on the appfleet blog we began with a few simple things:

Ghost responsive images

The featured image in a blog have lots of parameters, which is a good thing. For example, you can set multiple sizes in package.json and have Ghost automatically resize them for a responsive experience for users on mobile devices or smaller screens.

"config": {
		"posts_per_page": 10,
		"image_sizes": {
			"xxs": {
				"width": 30
			},
			"xs": {
				"width": 100
			},
			"s": {
				"width": 300
			},
			"m": {
				"width": 600
			},
			"l": {
				"width": 900
			},
			"xl": {
				"width": 1200
			}
                 }
}

And then, all you have to do is update the theme's code

<img class="feature-image"
    srcset="{{img_url feature_image size="s"}} 300w,
            {{img_url feature_image size="m"}} 600w,
            {{img_url feature_image size="l"}} 900w,
            {{img_url feature_image size="xl"}} 1200w"
    sizes="800px"
    src="{{img_url feature_image size="l"}}"
    alt="{{title}}"
/>

Common HTML tags for performance

Next we take a few simple steps to optimize Asset Download Time. That includes adding preconnect and preload headers in default.hbs:

<link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin="anonymous">
<link rel="preconnect" href="https://cdn.jsdelivr.net/" crossorigin="anonymous">
<link rel="preconnect" href="https://widget.appfleet.com/" crossorigin="anonymous">

<link rel="preload" as="style" href="https://fonts.googleapis.com/css?family=Red+Hat+Display:400,500,700&display=swap" />
<link rel="preload" as="style" href="https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free@5.13.0/css/all.min.css" />

As we load many files from jsDelivr to improve our performance, we instruct the browser to establish a connection with the domain as soon as possible. Same goes for Google Fonts and the sidebar widget that was custom coded.

Most often than not, users coming from Google or some other source to a specific blog post will navigate to the homepage to check what else we have written. For the same reason, on blog posts we also added prefetch and prerender tags for the main blog page.

That way the browser will asynchronously download and cache it, making the next most probable action of the user almost instant:

<link rel="prefetch" href="https://appfleet.com/blog">
<link rel="prerender" href="https://appfleet.com/blog">

Now these optimizations definitely helped but we still had a big problem. Our posts often have many screenshots and images in them, eventually impacting the page load time.

To solve this problem we took two steps. Lazy load the images and use a CDN. The issue is that Ghost doesn't allow to modify or filter the contents of the post. All you can do is output the HTML.

The easiest solution to this is to use a dynamic content CDN like Cloudflare. A CDN will proxy the whole site, won't cache the HTML, but cache all static content like images. They also have an option to lazy load all images by injecting their own Javascript.

But we didn't want to use Cloudflare in this case. And didn't feel like injecting third-party JS to lazy load the images either. So what did we do?

Nginx to the rescue!

Our blog is hosted on a DigitalOcean droplet created using its marketplace apps. It's basically an Ubuntu VM that comes pre-installed with Node.js, NPM, Nginx and Ghost.

Note that even if you don't use DigitalOcean, you are still recommended to use Nginx in-front of the Node.js app of Ghost.

This eventually makes the solution pretty simple. We use Nginx to rewrite the HTML, along with enabling a CDN and lazy-loading images at the same time, without any extra JS.

For CDN, you may also use the free CDN offered by Google to all AMP projects. Not many people are aware that you can use it as a regular CDN without actually implementing AMP.

All you have to do is use this URL in front of your images:

https://appfleet-com.cdn.ampproject.org/i/s/appfleet.com/

Replace the domains with your own and change your <img> tags, and you are done. All images are now served through Google's CDN.

The best part is that the images are not only served but optimized as well. Additionally, it will even serve a WebP version of the image when possible, further improving the performance of your site.

As for lazy loading, you may use the native functionality of modern browsers that looks like this <img loading="lazy". By adding loading="lazy" to all images, you instruct the browsers to automatically lazy load them once they become visible by the user.

And now the code itself to achieve this:

server {
    listen 80;

    server_name NAME;

    location ^~ /blog/ {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host       "appfleet.com";
        proxy_set_header        X-Forwarded-Proto https;
        proxy_pass http://127.0.0.1:2368;
        proxy_redirect off;
		
        #disable compression 
        proxy_set_header Accept-Encoding "";
        #rewrite the html
        sub_filter_once off;
        sub_filter_types text/html;
 		sub_filter '<img src="https://appfleet.com' '<img loading="lazy" src="https://appfleet-com.cdn.ampproject.org/i/s/appfleet.com';
    }

}

First we disable compression between node.js and nginx. Otherwise nginx can't modify the HTML if it comes in binary form.

Next we use the sub_filter parameter to rewrite the HTML. Ghost is using absolute paths in images, so we add the beginning as well. And in 1 line enabled both the CDN and lazyloading.

Reload the config and you are good to go. Check our blog to see this in real time.

Intro

If you've spent days (or even weeks?) trying to spin up a Kubernetes cluster for learning purposes or to test your application, then your worries are over. Spawned from a Kubernetes Special Interest Group, KIND is a tool that provisions a Kubernetes cluster running IN Docker.

From the docs:

kind is a tool for running local Kubernetes clusters using Docker container "nodes".
kind is primarily designed for testing Kubernetes 1.11+, initially targeting the conformance tests.

Installing KIND

As it is built using go, you will need to make sure you have the latest version of golang installed on your machine.

According to the k8s docs, golang -v 1.11.5 is preferred. To install kind, run these commands (it takes a while):

go get -u sigs.k8s.io/kind
kind create cluster

Then confirm kind cluster is available:

kind get clusters

Setting up kubectl

Also, install the latest kubernetes-cli using Homebrew or Chocolatey.
The latest Docker has Kubernetes feature but it may come with older kubectl . Check its version by running this command:

kubectl version

Make sure it shows GitVersion: "v1.14.1" or above.
If you find you are running kubectlfrom Docker, try brew link or reorder path environment variable.

Once kubectl and kind are ready, open bash console and run these commands:

export KUBECONFIG=”$(kind get kubeconfig-path)”
kubectl cluster-info

If kind is properly set up, some information will be shown.
Now you are ready to proceed. Yay!

Deploying first application

What should we deploy on the cluster? We are going to attempt deploying Cassandra since the docs have a pretty decent walk-through on it.

First of all, download cassandra-service.yaml and cassandra-statefulset.yaml for later. Then create kustomization.yaml by running two cat commands.
Once those yaml files are prepared, layout them as following:

k8s-wp/
  kustomization.yaml
  mysql-deployment.yaml
  wordpress-deployment.yaml

Then apply them to your cluster:

cd k8s-wp
kubectl apply -k ./

Validating (optional)

Get the Cassandra Service.

kubectl get svc cassandra

The response is:

NAME        TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
cassandra   ClusterIP   None         <none>        9042/TCP   45s

Note that Service creation might have failed if anything else is returned. Read Debug Services for common issues.

Finishing up

That's really all you need to know to get started with KIND, I hope this makes your life a little easier and lets you play with Kubernetes a little bit more :)