Using CheatSheets To Apply Best Practices

PKS Free CheatSheet

PKS Free CheatSheet

1.2 PKS Key Components

Name Summary
Computing virtualization Link: VMware vSphere Documentation
SDN networking NCP: CNI for Vmware NSX-T
VM/Cluster lifecycle management Link: BOSH CHEATSHEET, Github: Bosh
Node healing Link: BOSH CHEATSHEET
Container optimized OS CloudFoundry Stemcell, bosh vsphere ubuntu stemcell
Container runtime dockerd
Container Image Compliance Scan Harbor Clair
Docker image registry VMware Harbor
Packages k8s cluster orchestrator CFCR/Kubo, cfcr-etcd-release, GitHub: kubo-release, kubo-deployment
Component Packaging CheatSheet: Cloudfoundry Tile & OpsManager
CSI for persistent volume GitHub: hatchway
Reference Link: OpenShift Key Components, Link: PKS Key Components
Reference PKS CheatSheet, Bosh CheatSheet, Tile CheatSheet, UAA CheatSheet

1.3 VMware Product Integrations

1.4 PKS cli

Name Command
Check cli version pks --version
List all pks clusters pks clusters
Create cluster pks create-cluster <cluster-name> -e <subdomain>.pks.local -p "plan 1" -n 1
Delete cluster pks delete-cluster <cluster-name>
Check cluster status pks cluster cluster1
Get cluster kubectl credential pks get-credentials <cluster_name>
Set kubectl context kubectl config use-context <cluster_name>
List all available plans pks plans
pks login pks login -a <> -u <username> -p <passwd> -k
pks login pks login -k --username <username> --password <password> --api <>
Default roles in pks UAA pks.clusters.manage, pks.clusters.admin, Link: Manage Users in UAA

1.5 PKS Troubleshooting

Name Summary
Log files in pks vms /var/vcap/sys/log
How to run pks cli commands
How to run bosh cli commands, like bosh ssh
How to run kubectl command
How PKS supports k8s master HA GitHub: kubo-release, GitHub: cfcr-etcd-release
Workflow of how PKS creates a k8s cluster Link: Create a Kubernetes Cluster
How airgap integration tests are enforced For each node, load specific iptable rules. airgap-iptable.rules
Reference Link: PKS Troubleshoot

1.6 Deployment with NSX-T + NAT


1.7 Deployment with NSX-T + No-NAT + vswitch


1.8 Deployment with NSX-T + No-NAT + logical switch


1.9 PKS Assumptions

Name Summary
Multiple instances of K8S One PKS Multiple K8S clusters. Use k8s clusters or k8s namespace for multi-tenancy
No mixed versions Versions must match between master and node hosts, excluding upgrade
Support customization mainly at PKS level Not at k8s clusters or k8s namespace level
No hybrid cloud providers Support multi-clouds, but only can’t mix
Don’t run user workload in k8s master VMs Avoid messing up k8s control plane
Node roles Master nodes, worker nodes, and etcd nodes.

1.10 What PKS Adds to Kubernetes

Name Summary
Secure multi-tenant ingress NSX-T
Secure container registry VMware Harbor
Rolling upgrades to cluster infrastructure IaaS: bosh VM upgrade
Cluster provisioning and scaling IaaS: VM lifecycle management
Monitoring and recovery of cluster VMs and processes IaaS: VM lifecycle management
Embedded, hardened operating system Linux release for OS hardening
Log sink K8S Namespace multi-tenancy

1.11 PKS Challenges & Future Opportunities

Name Summary
Faster for typical use cases Create k8s clusters, resize k8s cluster, create pods, etc
Tile & OpsManager is not agile It slows down everything. The development, testing and deployment.
Extend PKS API layer Easy to add more functionalities for PKS admins
UX of PKS CLI The usage of pks cli could be more intuitive
Improve PKS control panel HA Online rolling upgrade for opsmanager, uaa, pks api, etc
Better storage support of PV HA for PV, and support more CSI providers
Cleanup for stale resources When operations have failed, need to do the cleanup in a safe way
More built-in security supports PKS supports most common security enhancements, but it doesn’t provides them

1.12 PKS Strengths

Name Summary
Kubernetes Federation Multiple clusters on-demand. Not only one kubernetes cluster for your infra
End-to-end integration Monitoring and logging works out of box
VM LCM: auto healing VM health check and auto-replacement
Less vendor lock-in Vanilla Kubernetes; Any infra; Any OS
Networking with NSX-T Advanced CNI
Image registry & security Image sign, audit, replication; vulnerabilities scan

1.13 Deployment Diagram

Name Summary
Bosh director vm VM manager
Ops manager vm Package manager
PKS API server vm See below
Build-in process in k8s master vm See below
Build-in process in k8s worker vm See below

1.14 PKS footprint: in control panel

  • Get process list in pks 1.2.0: ssh to the pks api vm, then sudo monit summary
Name Memory (RES)
pks-api 1 GB
uaa 500 MB
mysqld 500 MB
pks-nsx-t-osb-proxy 25 MB
telemetry 25 MB
bosh-agent 17 MB
bosh-dns 16 MB
on-demand-service-broker 16 MB
event-emitter 10 MB
galera-healthcheck 7 MB
bosh-dns-healthcheck 6 MB
cf-mysql-cluster-health-logger 6 MB
gra-log-purger-executable 2 MB

1.15 PKS footprint: in k8s master vms

  • Get process list in pks 1.2.0: ssh to k8s master vm, then sudo monit summary
Name Summary
etcd 120 MB
kube-apiserver 520 MB
kube-controller-manager 100 MB
kube-scheduler 35 MB
blackbox syslog 530 MB
fluentd 100 MB
ncp 70 MB
bosh-dns 19 MB
bosh-agent 15 MB
bosh-dns-nameserverconfig 5 MB
bosh-dns-health 10 MB

1.16 PKS footprint: in k8s worker vms

  • Get process list in pks 1.2.0: ssh to the k8s worker vm, then sudo monit summary
Name Summary
kube-proxy 30 MB
kubelet 100 MB
docker 70 MB
fluentd 180 MB
cadvisor 85 MB
blackbox syslog 60 MB
metrics-server 36 MB
ovs-vswitchd open vSwitch 35 MB
bosh-dns 20 MB
bosh-agent 18 MB
bosh-dns-health 7 MB
bosh-dns-namesever 5 MB
ovsdb-server vSwitch database 5 MB
nsx-node-agent 3 MB
nsx_kube_proxy 3 MB

1.17 PKS CLI Online Help

[ec2-user@ip-172-31-33-176 ~]$ pks --help

The Pivotal Container Service (PKS) CLI is used to create, manage, and delete Kubernetes clusters.
To deploy workloads to a Kubernetes cluster created using the PKS CLI, use the Kubernetes CLI, kubectl.

Version: 1.1.1-build.8

  pks [command]

Available Commands:
  cluster         View the details of the cluster
  clusters        Show all clusters created with PKS
  create-cluster  Creates a kubernetes cluster, requires cluster name, an external host name, and plan
  delete-cluster  Deletes a kubernetes cluster, requires cluster name
  get-credentials Allows you to connect to a cluster and use kubectl
  help            Help about any command
  login           Log in to PKS
  logout          Log out of PKS
  plans           View the preconfigured plans available
  resize          Increases the number of worker nodes for a cluster

  -h, --help      help for pks
      --version   version for pks

Use "pks [command] --help" for more information about a command.

1.18 More Resources

Leave a Reply

Your email address will not be published. Required fields are marked *