CentOS8 CheatSheet fuer System Administratoren

1 System

1.1 Hardware

• Show hardware details

lshw

• show bios hardware details

dmidecode

1.2 Disk/Filesystem

• Show block uage of mounted disks

df -hP

• Show inode uage of mounted disks

df -hiP

• show how blockdevices are configured in system

lsblk

• get 10 biggest files in current dir

du -sm * .[^\.]* | sort -n | tail

• use ncdu to find large files

dnf -y install epel-release
dnf config-manager --enable epel
dnf -y install ncdu
ncdu -x /  #replace / with needed mountpoint

1.3 Memory

• Show memory statistics

free -th

• Top provides memory information as well

top

1.4 CPU

• show system load (1m, 5m, 15m)

w

• detailed realtime statictics of procs, memory, swap, io, system, cpu

vmstat 1

1.5 Processes

• tui process manager

top

• show processes with tree

ps -f fax

• detailed process information

ps  -faxo ruser,ppid,pid,rss,vsz,pcpu,tty,args

1.6 Network

• show ip config

ip a

• show wan ip

curl ifconfig.me

• confgure network settings

nmtui

• activate changed network settings

nmcli connection reload
nmcli device reapply <nic name>
# or just reboot the host

1.6.1 SUDO: User als admin erstellen

useradd admin
passwd admin
id admin
usermod -a -G wheel admin
id admin
su - admin
# jetzt bist du als admin eingeloggt
sudo service chronyd restart
# das geht nur als root user (sudo rechte erteilt)

2 Services

• list all services

systemctl list-units --type service

• List failed services on boot

systemctl --failed

• show all running services

systemctl --all --state=running

• show detailed service status

systemctl status <svc-name>

• most common service handling

 systemctl [start|stop|restart|status|enable|disable|mask] <svc-name>

• check if service is enabled to autostart after reboot

systemctl is-enabled <svc-name>

3 SELinux

• show SELinux status

getenforce

• disable SELinux (not persistent)

setenforce 0

• change SELinux mode (reboot needed)

vi /etc/sysconfig/selinux

• show selinux alerts and recomendations

sealert -a /var/log/audit/audit.log

4 Firewalling

• show firewalling status

systemctl list-units --runtime | egrep 'firewall|tables'

• show active iptables rules

iptables-save

• show loaded nftables rules

nft list ruleset

4.1 disable firewalling for testing

• check which kind of firewall is running on the system

systemctl list-units --runtime | egrep 'firewall|tables'

• temporary disable ALL firewall rules (will reaper after reboot)

systemctl stop firewalld nftables iptables

• test now if it works

• restart all services that where running before testing (or just reboot)

systemctl start <svc-name>

5 Scheduling

5.1 cron (user/system)

• cron overview

tail -n +1 /etc/crontab /etc/cron.*/* /var/spool/cron/*

• edit crontab of user

crontab -e -u root

• show crontab of user

crontab -l -u bob

6 at (one time)

• create job

at 22:00 30.12.13

• list jobs

atq

• remove job

atrm <id>

7 systemd (user/system)

• list timers

systemctl list-units --type timer

• show configuration of timer

systemctl show <name>

• get timer status

systemctl status <name>

8 Events

8.1 Show last boots

journalctl --list-boots
last reboot

8.2 Journal

• Show log since last boot

journalctl -b

• Kernel messages (like dmesg)

journalctl -k

• Show latest log and wait for changes

journalctl -f

• Reverse output (newest first)

journalctl -r

• Show only errors and worse

journalctl -b -p err

• Filter on time (example)

journalctl --since=2014-06-00 --until="2014-06-07 12:00:00"

• Since yesterday

journalctl --since=yesterday

• Show only log of SERVICE

journalctl -u SERVICE

• Match executable, e.g. dhclient

journalctl /usr/sbin/dhclient

• Match device node, e.g. /dev/sda

journalctl /dev/sda

8.3 Log Files

• tail all currently open text files

lsof -F | sed '/^n\//!d;s/^n//;s/ .*//' | sort -u | xargs file | grep ' text$' | cut -d: -f1 | xargs tail -fn0

• tail all files under /var/log

tail -f -n0 /var/log/* /var/log/*/*

9 Software

install package and deps

dnf install httpd

remove package and deps

dnf remove php-mysql

update entire system and deps

dnf upgrade

update one package without question

dnf -y update httpd

clean package cache

dnf clean all

search for name in package-name and description

dnf search gstream

search for name in package-name

dnf list '*http*'

search for package which provides this comand

dnf provides '*fortune'

list all package groups

dnf grouplist

install package group

dnf groupinstall XFCE

remove package group

dnf remove "PostgreSQL Database"

downgrade package

dnf --showduplicates list samba-common
dnf downgrade package1 package2

10 Debugging

10.1 repeat comand

watch df -hP

10.2 network bandwith monitoring

dnf -y install epel-release
dnf -y install iftop
iftop

10.3 Dstat performance monitoring

dnf -y install pcp-system-tools
dstat -af
dstat -cdngy
dstat -f -M time,cpu,net,disk,sys,swap,page,load,proc,topcpu --output $(date '+%Y.%m.%d-%H.%M')-dstat.csv
dstat -cdngymsp --lock --tcp --output $(date '+%Y.%m.%d-%H.%M')-dstat.csv

10.4 Enter debug shell (reset root password)

• Reboot the machine
• When grub kernel selection is showing up, press e

• At the end of the line with beginning linux remove the "rhgb quiet" (on the end)
• At the end of the line with beginning linux add init=/bin/bash
• At the line beginning linux replace the ro with rw

• Press CTRL + x to boot into this settings
• Now system boots into single shell process instead of full linux env
• Now you can reset the password or debug the system
• hints:

su - # load full bash config

• change keyboard layout

loadkeys sg #swiss german
loadkeys us #forbidden :-)

• network config files

ls /etc/sysconfig/network-scripts/ifcfg-*

• selinux config (disable/enable)

vi /etc/sysconfig/selinux

• reset all selinux tags on next boot (relabel)

touch /.autorelabel

• force filesystem check on next boot

touch /.forcefsck

• force manual reboot

sync
reboot -f

10.5 strace examples (process tracing)

strace -ff -e trace=write -e write=1,2 -p SOME_PID
strace -e open ls
strace -e trace=open,read ls /home
strace -o output.txt ls 
strace -f -p 1725 -o firefox_trace.txt # f: follow process
strace -t -e open ls /home #timestamp
strace -c ls /home # statistics
strace -f -t -e trace=file systemctl restart SuSEfirewall2 2>&1 | grep open | cut -d'"' -f2

10.6 SOS Report (Log Bundle)

This collects a log bundle which can be used for later debugging and analyzing.
Start this BEFORE you try to repair or change something, as well do not forget to snapshot/backup System/Application before debugging.

dnf -y install sos
sosreport
cp -av /var/tmp/sosreport* /root/