OpenShift Notes

From Bitbull Wiki
Jump to navigation Jump to search

1 General

This is stuff I don't want to remember and don't want to forget! So I keep it here! BTW: I'm working with OKD current version (4.7)

2 Security

2.1 SCC Review

Get a list with pods and their assigned scc
oc get pod --all-namespaces | awk '{print $2 " -n "$1}' | grep -v NAME | while read p ; do echo "======= pod $p" ; oc describe pod $p | grep scc: ; done
only show the interesting containers
oc get pod --all-namespaces | awk '{print $1 ","$2}' | grep -v NAME | while read p 
do 
  namespace=$(echo $p | cut -d, -f1)
  pod=$(echo $p | cut -d, -f2)
  oc describe pod $pod -n $namespace | grep scc: | grep -v ": restricted" | while read perm 
  do
    echo "$namespace -> $pod -> $perm"
  done
done | column -t

2.1.1 Example results on fresh AWS OCP 4.7

File: pods with privileges on fresh ocp Modified: 2021-04-02 

openshift-apiserver                               ->  apiserver-5c44597f66-26vp4                               ->  openshift.io/scc:  node-exporter      
openshift-apiserver                               ->  apiserver-5c44597f66-6pw9q                               ->  openshift.io/scc:  node-exporter      
openshift-apiserver                               ->  apiserver-5c44597f66-8kgpf                               ->  openshift.io/scc:  node-exporter      
openshift-authentication-operator                 ->  authentication-operator-78b4fdf6c4-bpwgh                 ->  openshift.io/scc:  anyuid             
openshift-authentication                          ->  oauth-openshift-7f84dd5dd9-b9z4m                         ->  openshift.io/scc:  anyuid             
openshift-authentication                          ->  oauth-openshift-7f84dd5dd9-xzsns                         ->  openshift.io/scc:  anyuid             
openshift-cluster-node-tuning-operator            ->  cluster-node-tuning-operator-844cff6b46-hfhr5            ->  openshift.io/scc:  anyuid             
openshift-cluster-node-tuning-operator            ->  tuned-9dzsb                                              ->  Annotations:       openshift.io/scc:  privileged
openshift-cluster-node-tuning-operator            ->  tuned-l92pl                                              ->  Annotations:       openshift.io/scc:  privileged
openshift-cluster-node-tuning-operator            ->  tuned-pl7j4                                              ->  Annotations:       openshift.io/scc:  privileged
openshift-cluster-node-tuning-operator            ->  tuned-vgkjw                                              ->  Annotations:       openshift.io/scc:  privileged
openshift-cluster-node-tuning-operator            ->  tuned-vq4z9                                              ->  Annotations:       openshift.io/scc:  privileged
openshift-cluster-node-tuning-operator            ->  tuned-whtrw                                              ->  Annotations:       openshift.io/scc:  privileged
openshift-cluster-storage-operator                ->  cluster-storage-operator-d454bcdf5-r4z4q                 ->  openshift.io/scc:  anyuid             
openshift-cluster-storage-operator                ->  csi-snapshot-controller-operator-66d94cbd79-phn82        ->  openshift.io/scc:  anyuid             
openshift-config-operator                         ->  openshift-config-operator-84d8fd8945-khs8m               ->  openshift.io/scc:  anyuid             
openshift-controller-manager-operator             ->  openshift-controller-manager-operator-5595786bf5-qpnk8   ->  openshift.io/scc:  anyuid             
openshift-image-registry                          ->  node-ca-55mg8                                            ->  Annotations:       openshift.io/scc:  privileged
openshift-image-registry                          ->  node-ca-gw8pf                                            ->  Annotations:       openshift.io/scc:  privileged
openshift-image-registry                          ->  node-ca-kkxbc                                            ->  Annotations:       openshift.io/scc:  privileged
openshift-image-registry                          ->  node-ca-m52th                                            ->  Annotations:       openshift.io/scc:  privileged
openshift-image-registry                          ->  node-ca-t8rfk                                            ->  Annotations:       openshift.io/scc:  privileged
openshift-image-registry                          ->  node-ca-vfp7r                                            ->  Annotations:       openshift.io/scc:  privileged
openshift-ingress                                 ->  router-default-564744dbcf-skbwh                          ->  Annotations:       openshift.io/scc:  hostnetwork
openshift-ingress                                 ->  router-default-564744dbcf-vhk6c                          ->  Annotations:       openshift.io/scc:  hostnetwork
openshift-kube-storage-version-migrator-operator  ->  kube-storage-version-migrator-operator-565897bfd9-xbtql  ->  openshift.io/scc:  anyuid             
openshift-kube-storage-version-migrator           ->  migrator-5475dbfddc-tx6d8                                ->  openshift.io/scc:  anyuid             
openshift-machine-api                             ->  cluster-baremetal-operator-7b584c7dfc-d647k              ->  openshift.io/scc:  anyuid             
openshift-marketplace                             ->  community-operators-5cp2b                                ->  openshift.io/scc:  anyuid             
openshift-monitoring                              ->  alertmanager-main-0                                      ->  openshift.io/scc:  nonroot            
openshift-monitoring                              ->  alertmanager-main-1                                      ->  openshift.io/scc:  nonroot            
openshift-monitoring                              ->  alertmanager-main-2                                      ->  openshift.io/scc:  nonroot            
openshift-monitoring                              ->  node-exporter-5fbp5                                      ->  Annotations:       openshift.io/scc:  node-exporter
openshift-monitoring                              ->  node-exporter-lt47z                                      ->  Annotations:       openshift.io/scc:  node-exporter
openshift-monitoring                              ->  node-exporter-vgt4r                                      ->  Annotations:       openshift.io/scc:  node-exporter
openshift-monitoring                              ->  node-exporter-wc65q                                      ->  Annotations:       openshift.io/scc:  node-exporter
openshift-monitoring                              ->  node-exporter-wsz7m                                      ->  Annotations:       openshift.io/scc:  node-exporter
openshift-monitoring                              ->  node-exporter-zp6pt                                      ->  Annotations:       openshift.io/scc:  node-exporter
openshift-monitoring                              ->  prometheus-k8s-0                                         ->  openshift.io/scc:  nonroot            
openshift-monitoring                              ->  prometheus-k8s-1                                         ->  openshift.io/scc:  nonroot            
openshift-oauth-apiserver                         ->  apiserver-6894d6684-cq4d2                                ->  openshift.io/scc:  node-exporter      
openshift-oauth-apiserver                         ->  apiserver-6894d6684-jxpvt                                ->  openshift.io/scc:  node-exporter      
openshift-oauth-apiserver                         ->  apiserver-6894d6684-wj9d5                                ->  openshift.io/scc:  node-exporter      
openshift-operator-lifecycle-manager              ->  catalog-operator-5d56d75ccf-zr87t                        ->  openshift.io/scc:  anyuid             
openshift-operator-lifecycle-manager              ->  olm-operator-5f7849c5c4-cm8dw                            ->  openshift.io/scc:  anyuid             
openshift-operator-lifecycle-manager              ->  packageserver-749dd7985b-672p8                           ->  openshift.io/scc:  anyuid             
openshift-operator-lifecycle-manager              ->  packageserver-749dd7985b-trc95                           ->  openshift.io/scc:  anyuid



2.2 Check all the SCCs

of course you should also be interested on the scc configuration. a modification of the default SCCs could lead you into big trouble.

get all the scc configurations
oc get scc | cut -d' ' -f1 | grep -v NAME | while read scc; do oc describe scc $scc > "$scc"; done

2.2.1 Example results on fresh AWS OCP 4.7

File: anyuid Modified: 2021-04-02 

Name:						anyuid
Priority:					10
Access:						
  Users:					<none>
  Groups:					system:cluster-admins
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			MKNOD
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				false
  Allow Host Ports:				false
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: RunAsAny		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: RunAsAny			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: hostaccess Modified: 2021-04-02 

Name:						hostaccess
Priority:					<none>
Access:						
  Users:					<none>
  Groups:					<none>
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			KILL,MKNOD,SETUID,SETGID
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,hostPath,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				true
  Allow Host Ports:				true
  Allow Host PID:				true
  Allow Host IPC:				true
  Read Only Root Filesystem:			false
  Run As User Strategy: MustRunAsRange		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: MustRunAs			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: hostmount-anyuid Modified: 2021-04-02 

Name:						hostmount-anyuid
Priority:					<none>
Access:						
  Users:					system:serviceaccount:openshift-infra:pv-recycler-controller
  Groups:					<none>
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			MKNOD
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,hostPath,nfs,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				false
  Allow Host Ports:				false
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: RunAsAny		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: RunAsAny			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: hostnetwork Modified: 2021-04-02 

Name:						hostnetwork
Priority:					<none>
Access:						
  Users:					<none>
  Groups:					<none>
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			KILL,MKNOD,SETUID,SETGID
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				true
  Allow Host Ports:				true
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: MustRunAsRange		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: MustRunAs			
    Ranges:					<none>
  Supplemental Groups Strategy: MustRunAs	
    Ranges:					<none>

File: machine-api-termination-handler Modified: 2021-04-02 

Name:						machine-api-termination-handler
Priority:					<none>
Access:						
  Users:					system:serviceaccount:openshift-machine-api:machine-api-termination-handler
  Groups:					<none>
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			KILL,MKNOD,SETUID,SETGID
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				downwardAPI,hostPath
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				true
  Allow Host Ports:				false
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: RunAsAny		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: MustRunAs			
    Ranges:					<none>
  Supplemental Groups Strategy: MustRunAs	
    Ranges:					<none>

File: node-exporter Modified: 2021-04-02 

Name:						node-exporter
Priority:					<none>
Access:						
  Users:					<none>
  Groups:					<none>
Settings:					
  Allow Privileged:				true
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			<none>
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				*
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				true
  Allow Host Ports:				true
  Allow Host PID:				true
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: RunAsAny		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: RunAsAny		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: RunAsAny			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: nonroot Modified: 2021-04-02 

Name:						nonroot
Priority:					<none>
Access:						
  Users:					<none>
  Groups:					<none>
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			KILL,MKNOD,SETUID,SETGID
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				false
  Allow Host Ports:				false
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: MustRunAsNonRoot	
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: RunAsAny			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: privileged Modified: 2021-04-02 

Name:						privileged
Priority:					<none>
Access:						
  Users:					system:admin,system:serviceaccount:openshift-infra:build-controller
  Groups:					system:cluster-admins,system:nodes,system:masters
Settings:					
  Allow Privileged:				true
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			<none>
  Allowed Capabilities:				*
  Allowed Seccomp Profiles:			*
  Allowed Volume Types:				*
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			*
  Forbidden Sysctls:				<none>
  Allow Host Network:				true
  Allow Host Ports:				true
  Allow Host PID:				true
  Allow Host IPC:				true
  Read Only Root Filesystem:			false
  Run As User Strategy: RunAsAny		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: RunAsAny		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: RunAsAny			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: restricted Modified: 2021-04-02 

Name:						restricted
Priority:					<none>
Access:						
  Users:					<none>
  Groups:					system:authenticated
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			KILL,MKNOD,SETUID,SETGID
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				false
  Allow Host Ports:				false
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: MustRunAsRange		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: MustRunAs			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>

File: anyuid Modified: 2021-04-02 

Name:						anyuid
Priority:					10
Access:						
  Users:					<none>
  Groups:					system:cluster-admins
Settings:					
  Allow Privileged:				false
  Allow Privilege Escalation:			true
  Default Add Capabilities:			<none>
  Required Drop Capabilities:			MKNOD
  Allowed Capabilities:				<none>
  Allowed Seccomp Profiles:			<none>
  Allowed Volume Types:				configMap,downwardAPI,emptyDir,persistentVolumeClaim,projected,secret
  Allowed Flexvolumes:				<all>
  Allowed Unsafe Sysctls:			<none>
  Forbidden Sysctls:				<none>
  Allow Host Network:				false
  Allow Host Ports:				false
  Allow Host PID:				false
  Allow Host IPC:				false
  Read Only Root Filesystem:			false
  Run As User Strategy: RunAsAny		
    UID:					<none>
    UID Range Min:				<none>
    UID Range Max:				<none>
  SELinux Context Strategy: MustRunAs		
    User:					<none>
    Role:					<none>
    Type:					<none>
    Level:					<none>
  FSGroup Strategy: RunAsAny			
    Ranges:					<none>
  Supplemental Groups Strategy: RunAsAny	
    Ranges:					<none>
Retrieved from "http://www.bitbull.ch/wiki/index.php?title=OpenShift_Notes&oldid=5295"