Install AWX on K3S

Since version 18, AWX, the comunity edition of ansible tower gets deployed by a kubernetes operator. This makes it easier to install and maintain the installation, but not all of us are familiar with kubernetes and operators. So I share a short step by step guide on how to setup ansible awx in a "semi professional" way on a single k3s kubernetes node.

1 VM Setup

1.1 VM requirements

Just setup a CentOS8 minimal VM with the following requirements

• OS: centos8 minimal

• CPU: 2
• MEM: 8GB (6 GB may work as well)
• DISK: 40G (7GB used on a fresh setup)

1.2 Prepare OS

dnf -y upgrade
dnf -y install setroubleshoot-server curl lsof wget

sed -i  '/swap/d' /etc/fstab
swapoff -a

firewall-cmd --permanent --zone=public --add-service=https
firewall-cmd --zone=public --add-masquerade --permanent
firewall-cmd --reload
reboot

2 Setup K3S

export INSTALL_K3S_EXEC="--etcd-snapshot-schedule-cron='0 */12 * * *' --etcd-snapshot-retention=14"
curl -sfL https://get.k3s.io | sh -

cat /etc/systemd/system/k3s.service
systemctl status k3s

kubectl get nodes
# all pods in running state? fine!
kubectl get pods --all-namespaces

3 Deploy AWX

3.1 Deploy the AWX operator

kubectl config set-context --current --namespace=default
kubectl apply -f https://raw.githubusercontent.com/ansible/awx-operator/devel/deploy/awx-operator.yaml

3.1.1 wait until image got pulled

kubectl get events -w --all-namespaces

3.1.2 check for running operator pod

kubectl get pods

3.1.3 optionally check the operators logs

kubectl logs -f deployment.apps/awx-operator

3.2 Deploy AWX Application

- first we create a namespace for our setup

kubectl create namespace awx
kubectl config set-context --current --namespace=awx

• vim myawx.yml

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  tower_ingress_type: Ingress

• kubectl apply -f myawx.yml

3.2.1 wait until images got pulled

kubectl get events -w --all-namespaces

kubectl logs -f deployment.apps/awx-operator -n default

• Wait for operator to finish the deployment of awx

PLAY RECAP *********************************************************************
localhost: ok=42 changed=0 unreachable=0 failed=0 skipped=30 rescued=0 ignored=0

3.2.2 Verify the deployment staus

two pods must be in running state

kubectl get pods

3.2.3 Verify awx service

notice the service port 80 for awx

kubectl get svc

NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
awx-postgres   ClusterIP   None            <none>        5432/TCP       35m
awx-service    NodePort    10.3.12.26      <none>        80:31982/TCP   35m

4 Deploy the ingress entry point by traefik

• vi ingress.yml

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: awx-service
            port:
              number: 80

kubectl apply -f ingress.yml

5 Fetch the secret and test the login

kubectl get secret awx-admin-password -o jsonpath='{.data.password}' | base64 --decode

firefox https://fqdn

• user: admin

6 Links

• https://rancher.com/docs/k3s/latest/en/quick-start/
• https://rancher.com/docs/k3s/latest/en/backup-restore/
• https://github.com/ansible/awx-operator

7 Debug Notes

7.1 Open Node Port for direct access

PORT=$(kubectl describe svc awx-service | grep NodePort: | awk '{print $3}' | tr 'A-Z' 'a-z')
echo PORT=$PORT
firewall-cmd --zone=public --add-port=$PORT

7.2 Disable SELinux

setenforce 0
> /var/log/audit/audit.log 
# do some bad things
sealert -a /var/log/audit/audit.log

7.3 Traefik Config

- https://levelup.gitconnected.com/a-guide-to-k3s-ingress-using-traefik-with-nodeport-6eb29add0b4b

kubectl -n kube-system edit cm traefik

7.4 Jump into container for debugging

# get pods
kubectl get pods
# get containers inside of pods
kubectl describe <pod-name>

kubectl exec --stdin --tty <pod-name> -c <container-name> -- /bin/bash