Kubernetes Getting Started

This content has moved and will no longer be updated. Please go to https://docs.vmware.com/en/CloudHealth-Secure-State/ for the latest version. Please see the latest What's new for the more details about the move.

Last updated on January 18, 2022

Introduction

Cloud Health Secure State can monitor your Kubernetes clusters as a separate resource similar to currently supported cloud providers. By onboarding your Kubernetes clusters into Secure State you can take advantage of real-time monitoring for misconfigurations and security violations corresponding to CIS Kubernetes Benchmark standards, and build a more complete image of your organization's security posture in the cloud.

This feature is available for managed Kubernetes services on AWS, Azure, and GCP platforms. Support for self-managed Kubernetes services is currently in beta.


Getting started

Secure State is configured to automatically detect Kubernetes clusters that are deployed in any of the following services:

  • Azure Kubernetes Service (AKS)
  • Elastic Kubernetes Service (EKS)
  • Google Kubernetes Service (GKE)

Any clusters you've deployed in a cloud account are detected shortly after that account is onboarded with Secure State. Any clusters you deployed in a cloud account that is already onboarded are similarly detected.


Onboarding Kubernetes clusters with the Secure State UI

Prerequisites

Although Secure State can detect your Kubernetes clusters, you must still take action to fully onboard them. To do this, you need the following dependencies:

  • Access to an environment with kubectl installed.
  • The relevant provider CLI.
  • Kubernetes clusters on version 1.17 or later.

The latter is required to set kubectl's current context to your cloud-based clusters. You can also onboard clusters from your provider's cloud shell if preferred, though you may need to install kubectl depending on your provider.

Review detected clusters

From the Secure State dashboard, navigate to Settings > Kubernetes clusters to see a list of clusters already detected by Secure State. Any cluster that has been detected but not yet onboarded displays the Discovered status.

Discovered cluster

Try viewing this page in different contexts, if you have access to them. The list dynamically updates to show only clusters associated with cloud accounts scoped to the currently selected context.

Attach a cluster

To attach a cluster, you must run two kubectl commands to install a collector in your Kubernetes environment that sends configuration data back to Secure State, which can then trigger findings based on active monitoring rules.

  1. To begin, click the menu icon to the left of the cluster and select Attach cluster.

  2. Review your cluster information in the pop-up dialog, then click Next. This takes your cluster from Discovered to Pending status.

  3. On the next step, note the value for COLLECTOR_CLIENT_SECRET and copy it to a secure location. The secret isn't displayed again after this screen, and you'll need to detach the cluster and go through these steps again if you lose it.

    Collector client secret

  4. Copy this command from the UI (include your secret where indicated) and run it in your local or cloud shell:

    kubectl create namespace chss-k8s && kubectl create secret generic collector-client-secret --from-literal=COLLECTOR_CLIENT_SECRET='<your_secret>' -n chss-k8s

  5. Download the Kubernetes configuration YAML file supplied by Secure State. You can click the download icon next to the "View Configuration details (YAML)" link to save it to your local environment. If you're using a provider cloud shell, upload this file through your shell interface before taking the next step.

    Note: Every YAML configuration file is unique to the specific cluster and collector you're trying to attach. Do not re-use YAML configuration files from previous attempts, even if it's from the same cluster.

  6. In your CLI, navigate to the directory the YAML file is saved to and run this command:

    kubectl apply -f <filename.yaml>

  7. Refresh your screen and you should now see your cluster in the Ready status. It may take a few minutes to update.

    Online cluster

Update a cluster's collector

The Secure State team regularly updates the collector for Kubernetes clusters with new features and bug fixes. You can see which clusters need an update by going to the Kubernetes clusters page and reviewing the list of clusters in Ready status. Clusters with outdated collectors have an info icon in the Collector version field that shows you the latest available version. You can also see the total count of collectors that need to be updated at the top of the list.

To update a collector, follow these steps:

  1. Click the menu icon to the left of your cluster and select Update collector version.

  2. Download the Kubernetes configuration YAML file supplied by Secure State. You can click the download icon next to the "View Configuration details (YAML)" link to save it to your local environment. If you're using a provider cloud shell, upload this file through your shell interface before taking the next step.

  3. In your CLI, navigate to the directory the YAML file is saved to and run this command:

    kubectl apply -f <filename.yaml>

  4. Refresh your screen and you should now see your cluster with the latest version in the Collector version field. It may take a few minutes to update.

As a best practice, you should follow this process at least once a quarter.

Detach a cluster

Clusters in Pending or Ready status can be detached after the collector running in the cluster is removed and Collector Status is Offline. When you detach a cluster, the Kubernetes collector is uninstalled from your cluster, returning it to Discovered status. Secure State stops tracking any findings originating from the detached cluster, similar to disabling a rule.

  1. Run this command in your local or cloud shell to remove the collector.

    kubectl delete namespace chss-k8s

  2. From the Secure State dashboard, navigate to Settings > Kubernetes clusters.

  3. Locate your cluster and verify the Collector Status is Offline. You may need to wait a few minutes and refresh your screen before the status changes.

  4. Click the menu icon to the left of your cluster and select Detach cluster.

  5. Enter the name of your cluster where prompted and select Detach.

You can re-attach the cluster again by following the same process as before.

Add a self-managed cluster

Self-managed Kubernetes clusters are hosted on virtual machines for services like AWS EC2 or Azure Compute instead of AKS, EKS, or GKE. You can add self-managed clusters to Secure State from the Settings > Kubernetes clusters page.

  1. Click the Attach Self-Managed Cluster button.

  2. Choose a name for your cluster. You can enter any name that isn't currently in use by another cluster.

  3. Select the cloud account the cluster is associated with, then click Next.

  4. The steps to install the collector are identical to the directions for attaching a cluster.


Onboarding Kubernetes clusters with the Secure State API

You can onboard Kubernetes clusters through Secure State's public API if it is your preference. All API commands utilize the collector-apis endpoint in the Cloud Accounts Service API.

Prerequisites

Onboarding a Kubernetes cluster through Secure State's public API has the same prerequisites as the UI onboarding procedure: ensure you have kubectl kubectl installed and appropriately configured to your cluster.

It's also helpful to use an API development tool like Postman that allows you to easily download responses for certain steps. Since there are many different ways the API calls could be made, this guide presents them as cURL commands to have a common baseline.

Review detected clusters

Use this command to retrieve the cluster you want to onboard:

curl POST \
https://api.securestate.vmware.com/v1/collectors/query \
-H 'Authorization: Bearer {access_token}' \
-d ' {
  "filters": {
    "op": "EQ",
    "values": [
    {
        "op": "EQ",
        "key": "name",
        "values": ["<cluster name>"]
      }
    ]
  }
}'

Replace cluster name with the name of your cluster in the body. If there are multiple clusters you need to onboard, you can send the request with an empty body to view all clusters Secure State has detected.

A successful response for a cluster that hasn't yet been onboarded should look like this:

{
    "id": "{collector ID}",
    "name": "{cluster name}",
    "type": "Kubernetes",
    "cspOrgId": "{CSP Org ID}",
    "provider": "{provider}",
    "cloudAccountId": "{cloud account ID}",
    "status": "Discovered",
    "location": "{region code}",
    "credentials": null,
    "properties": {
        "kubernetesVersion": "{k8s version}",
        "managed": true
    },
    "createdTime": "{timestamp}",
    "lastUpdatedTime": "{timestamp}"
}

Get cluster credentials

The next API call retrieves the credentials you need to install a rules collector on your cluster to read its data. Copy all the information from your previous response that you see in this request body:

curl POST \
https://api.securestate.vmware.com/v1/collectors \
-H 'Authorization: Bearer {access_token}' \
-d '{
    "id": "{collector ID}",
    "name": "{cluster name}",
    "type": "Kubernetes",
    "cspOrgId": "{CSP Org ID}",
    "provider": "{provider}",
    "cloudAccountId": "{cloud account ID}",
    "status": "Discovered",
    "location": "{region code}",
    "properties": {
        "kubernetesVersion": "{k8s version}",
        "managed": true
    }
}'

You should see a successful response:

{
   "id":"{collector ID}",
   "name":"{cluster name}",
   "type":"Kubernetes",
   "cspOrgId":"{CSP Org ID}",
   "provider":"{provider}",
   "cloudAccountId":"{cloud account ID}",
   "status":"Pending",
   "location":"{region code}",
   "credentials":{
      "properties":{
         "clientId":"{client ID}",
         "clientSecret":"{client secret}"
      },
      "credentialsType":"cspCredentialPair",
      "creationTime":"{timestamp}"
   },
   "properties":{
      "kubernetesVersion":"{k8s version}",
      "managed":true
   },
   "createdTime":"{timestamp}",
   "lastUpdatedTime":"{timestamp}"
}

You should see your cluster's status changed to Pending. Copy the clientSecret value and keep it in a secure location; it is only displayed once.

Note: The id value changes whenever the cluster's status updates (From Discovered to Pending, or Pending to Ready). Always use the most recent collector ID when making API calls.

Create configuration file

Use this command to retrieve the content used to create a configuration file:

curl GET \
https://api.securestate.vmware.com/v1/collectors/{collector ID}/installer \
-H 'Authorization: Bearer {access_token}'

Save or copy the response as a YAML file, named to your preference.

Note: Every YAML configuration file is unique to the specific cluster and collector you're trying to attach. Do not re-use YAML configuration files from previous attempts, even if it's from the same cluster.

Install collector

Add the client secret and YAML file name to the following kubectl commands them:

kubectl create namespace chss-k8s && kubectl create secret generic collector-client-secret --from-literal=COLLECTOR_CLIENT_SECRET='{client secret}' -n chss-k8s

kubectl apply -f {filename.yaml}

Your cluster should now go from the Pending to Ready status. Verify through the API or UI.

Detach a cluster

To detach a cluster, run the following command:

curl DELETE \
https://api.securestate.vmware.com/v1/collectors/{collector ID} \
-H 'Authorization: Bearer {access_token}'

You should see the cluster status returned to Discovered. Verify though the app or UI.

Although this process uninstalls the collector, the chss-k8s namespace remains in place and should be removed before attempting to re-attach the cluster. Run this command to delete the namespace:

kubectl delete namespace chss-k8s


Review Kubernetes findings

Once your clusters are onboarded, you can monitor them for security issues like any other resource. Findings from your onboarded clusters are associated with the cloud accounts the clusters are deployed on.

To view findings associated with your clusters:

  1. Navigate to Findings > Findings (all) from the Secure State Dashboard
  2. Click on the filter icon next to All findings, near the top of the list.
  3. From the side panel, select Provider, then Kubernetes.
  4. Click on Apply.

You should now see a filtered list of any findings detected on your clusters. You can also select cloud accounts or Kubernetes-specific services to further refine results.

Kubernetes findings

You can also review Findings by rule and Findings by resource with the same filter applied to see different breakdowns of the results. See the Findings User Guide for more details on researching and filtering findings in general.

Findings types for Kubernetes

There are over 150 Kubernetes-specific rules that can generate findings on your clusters. Secure State offers two broad types of findings to align your Kubernetes security posture to best practices:

  • Findings that are aligned to Kubernetes CIS benchmark controls and hardening guidance adopted by industry practitioners.
  • Findings that are recommended by Secure State based on best practices promoted by cloud providers.

Findings of the former type are provider agnostic and can be generated on any Kubernetes cluster you deploy, while the latter are specific to the managed Kubernetes service you're using. You can easily pick these findings out by looking for the managed service (EKS, GKE, or AKS) in the name of the associated rule.


Setting up reports

You can set up reports for your Kubernetes clusters to further isolate and highlight their security findings apart from your other cloud resources.

  1. From the Secure State dashboard, click Report.
  2. Click Add New.
  3. Enter a name for your report, the project to select cloud accounts from, and the frequency you'd like the report to run at. Click Next when done.
  4. On the criteria page, click Add Another and select Provider from the options available. Set the value for the Provider field to Kubernetes.
  5. Select the cloud accounts you want to include in the report, then click Next.
  6. On the delivery page, choose the recipients for the report and click Save.

You've now configured regular reports for your Kubernetes clusters.