saw a great example of how crappy VM management is implemented in linux. had a mysql server as an idle slave of the main mysql database running for days. was making backups of it, so it had all the data in RAM. switched to it from another slave and suddenly it started crapping out.
take a look at this top snapshot:
---
top - 21:33:00 up 32 days, 10:56, 1 user, load average: 27.02, 17.08, 7.93
Tasks: 104 total, 4 running, 99 sleeping, 1 stopped, 0 zombie
Cpu(s): 6.9%us, 92.9%sy, 0.0%ni, 0.0%id, 0.1%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 32941100k total, 32785596k used, 155504k free, 1164k buffers
Swap: 2096472k total, 812k used, 2095660k free, 74972k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1834 mysql 15 0 34.1g 31g 5084 S 580 98.9 920:17.30 mysqld
870 root 11 -5 0 0 0 R 72 0.0 11:56.62 kjournald
565 root 20 -5 0 0 0 R 68 0.0 17:00.23 kswapd0
696 root 15 0 0 0 0 S 1 0.0 0:37.72 pdflush
1 root 15 0 10324 464 352 S 0 0.0 0:56.23 init
---
all the RAM is used by mysql, there is no swap being used, yet kswapd0 is effectively wasting as much as it possibly can. what is it doing? who knows, all I know is that there is no disk i/o. what can a kernel swap process do when there is no swap used and there is no disk i/o? kjournald is wasting some resources too. why? who knows. what a brainless VM management. :(
