Performance Optimization Guide BETA

Advanced techniques to maximize performance, increase efficiency, and deliver superior service quality on your infrastructure.

System-Level Optimization

Start with fundamental system optimizations that benefit all workloads.

# Network performance
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq

# Memory management
vm.swappiness = 10
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5
vm.overcommit_memory = 1
kernel.numa_balancing = 1

# File system
fs.file-max = 2097152
fs.aio-max-nr = 1048576

# Security and performance
kernel.pid_max = 4194304
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1

# Set performance governor for all CPUs
for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do
    echo performance > $cpu
done

# Disable CPU frequency scaling
systemctl disable ondemand

# Intel-specific optimizations
# Disable C-states for lower latency
echo 1 > /sys/module/intel_idle/parameters/max_cstate

# AMD-specific optimizations
# Enable CPPC for better performance
echo active > /sys/devices/system/cpu/cpufreq/policy*/energy_performance_preference

CPU Performance Optimization

Maximize CPU performance for compute-intensive workloads.

# Check NUMA topology
numactl --hardware

# Pin instances to NUMA nodes
slyd-provider config set instance.numa_aware true

# Set NUMA memory policy
numactl --cpunodebind=0 --membind=0 slyd-provider start

# Monitor NUMA statistics
numastat -c

# Reserve CPUs for instances (edit /etc/default/grub)
GRUB_CMDLINE_LINUX="isolcpus=4-31 nohz_full=4-31 rcu_nocbs=4-31"

# Update grub
sudo update-grub

# Configure SLYD to use isolated CPUs
slyd-provider config set cpu.isolated_cores "4-31"
slyd-provider config set cpu.system_cores "0-3"

Memory Optimization

Optimize memory allocation and performance for better instance density.

# Enable THP for better performance
echo always > /sys/kernel/mm/transparent_hugepage/enabled
echo always > /sys/kernel/mm/transparent_hugepage/defrag

# For KVM optimization
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

# Configure hugepages for VMs
echo 1024 > /proc/sys/vm/nr_hugepages
mount -t hugetlbfs hugetlbfs /dev/hugepages

# Enable KSM (Kernel Samepage Merging)
echo 1 > /sys/kernel/mm/ksm/run
echo 1000 > /sys/kernel/mm/ksm/sleep_millisecs

# Configure SLYD memory overcommit
slyd-provider config set memory.overcommit_ratio 1.2
slyd-provider config set memory.balloon_enabled true

Storage Performance Tuning

Optimize storage subsystem for maximum IOPS and throughput.

echo none > /sys/block/nvme0n1/queue/scheduler

echo noop > /sys/block/sda/queue/scheduler

echo deadline > /sys/block/sdb/queue/scheduler

# Mount options for performance
/dev/sda1 /storage ext4 noatime,nodiratime,discard,data=writeback,nobarrier 0 0

# Tune existing filesystem
tune2fs -o journal_data_writeback /dev/sda1
tune2fs -O ^has_journal /dev/sda1  # Disable journal for extreme performance

# XFS optimization for large files
mkfs.xfs -f -d agcount=32 -l size=512m /dev/sdb1

Network Performance Tuning

Optimize network stack for low latency and high throughput.

#!/bin/bash
INTERFACE="eth0"

# Increase ring buffers
ethtool -G $INTERFACE rx 4096 tx 4096

# Enable offloading features
ethtool -K $INTERFACE rx on tx on sg on tso on gso on gro on lro on

# Set interrupt coalescing
ethtool -C $INTERFACE adaptive-rx on adaptive-tx on rx-usecs 10

# CPU affinity for interrupts
systemctl stop irqbalance
echo 2 > /proc/irq/24/smp_affinity  # Bind NIC IRQ to CPU 2

# Enable jumbo frames
ip link set dev $INTERFACE mtu 9000

# TCP optimization
sysctl -w net.ipv4.tcp_fastopen=3
sysctl -w net.ipv4.tcp_mtu_probing=1
sysctl -w net.ipv4.tcp_low_latency=1

# Connection tracking
sysctl -w net.netfilter.nf_conntrack_max=1048576
sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=3600

# Enable BBR congestion control
modprobe tcp_bbr
sysctl -w net.ipv4.tcp_congestion_control=bbr

# Increase socket buffers
sysctl -w net.core.rmem_max=134217728
sysctl -w net.core.wmem_max=134217728

GPU Optimization

Maximize GPU performance for compute and AI workloads.

# Set GPU to persistence mode
nvidia-smi -pm 1

# Set performance mode
nvidia-smi -ac 1215,1410  # Memory,Graphics clocks for V100

# Disable ECC for better performance (optional)
nvidia-smi -e 0

# Set power limit to maximum
nvidia-smi -pl 300  # Watts, adjust per GPU model

# Configure GPU for compute mode
nvidia-smi -c EXCLUSIVE_PROCESS

# Monitor GPU performance
nvidia-smi dmon -s pucvmet

# Enable peer-to-peer access
nvidia-smi topo -m

# Set NUMA affinity for GPUs
nvidia-smi topo --setaffinity 0:0,1:1  # GPU:CPU

# Configure NVLink (if available)
nvidia-smi nvlink -s
nvidia-smi nvlink -g 0

Container Performance

Optimize container runtime for better instance performance.

# Storage pool optimization
lxc storage create fast zfs source=/dev/nvme0n1
lxc storage set fast zfs.use_refquota true
lxc storage set fast volume.zfs.remove_snapshots true

# CPU limits configuration
lxc profile set default limits.cpu.priority 5
lxc profile set default limits.cpu.allowance 100%

# Memory limits
lxc profile set default limits.memory.enforce soft
lxc profile set default limits.memory.swap false

# Network optimization
lxc network set lxdbr0 ipv6.dhcp.stateful true
lxc network set lxdbr0 bridge.mtu 9000

Performance Monitoring

Continuous monitoring is essential for maintaining optimal performance.

# CPU performance
mpstat -P ALL 1
perf top
turbostat --interval 1

# Memory performance
vmstat 1
slabtop
pcm-memory.x

# Storage performance
iostat -x 1
iotop -o
blktrace -d /dev/nvme0n1

# Network performance
sar -n DEV 1
ss -s
tcpstat -i eth0 -o "%R %T"

Performance Automation

Automate performance optimization with scripts and scheduled tasks.

#!/bin/bash
# SLYD Performance Auto-Tuning Script

# Function to tune based on load
tune_performance() {
    local cpu_usage=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d'%' -f1)
    
    if (( $(echo "$cpu_usage > 80" | bc -l) )); then
        # High load optimizations
        echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
        echo 0 > /proc/sys/kernel/numa_balancing
    elif (( $(echo "$cpu_usage < 30" | bc -l) )); then
        # Low load optimizations
        echo powersave | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
        echo 1 > /proc/sys/kernel/numa_balancing
    fi
}

# Run tuning every 5 minutes
while true; do
    tune_performance
    sleep 300
done

Performance Troubleshooting

Common performance issues and their solutions.

Optimization Best Practices

Follow these guidelines for sustained high performance.