在系统维护期间我们可能会经常遇到CPU 过高的情况,如果长时间不处理,可能最终会导致宕机或者服务挂掉。那什么原因会导致这种情况,这就需要我们一步一步去分析了。
一、通过 top 命令查看占用CPU较高的进程。可以看出29433进程占用CPU较高。
top - 13:55:32 up 59 days, 19:18, 2 users, load average: 0.00, 0.04, 0.09
Tasks: 161 total, 1 running, 160 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.7 us, 0.3 sy, 0.0 ni, 96.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 16267564 total, 271608 free, 9033932 used, 6962024 buff/cache
KiB Swap: 8388604 total, 8364736 free, 23868 used. 5913400 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29433 wuuser 20 0 7805852 1.132g 19216 S 0.7 7.3 6:07.13 java
29355 wuuser 20 0 6909304 1.093g 17248 S 0.3 7.0 2:29.96 java
29558 wuuser 20 0 7763476 974788 15860 S 0.3 6.0 1:54.47 java
29945 wuuser 20 0 7793536 1.202g 21344 S 0.3 7.7 4:31.02 java
二、通过 top -H -p pid 命令查看进程内各个线程占用的 CPU 百分比 。通过这个命令
我们可以发现线程29436占用时长多 。
top -H -p 29433
top - 13:58:57 up 59 days, 19:22, 2 users, load average: 0.04, 0.07, 0.10
Threads: 132 total, 0 running, 132 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.0 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 16267564 total, 271024 free, 9034476 used, 6962064 buff/cache
KiB Swap: 8388604 total, 8364736 free, 23868 used. 5913044 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
29433 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:00.00 java
29436 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:08.91 java
29437 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:00.34 java
29438 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:00.35 java
29439 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:00.34 java
29440 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:00.34 java
29441 wuuser 20 0 7805852 1.132g 19216 S 0.0 7.3 0:00.34 java
三、使用 printf "%x\n" 线程号将异常线程号转化为16进制
printf "%x\n" 29436
72fc
四、使用 jstack 进程号| grep 线程号 -A90 命令来打印JAVA 堆栈信息,定位出现异常的代码,定位错误原因。
jstack 29433|grep 72fc -A90
"main" prio=10 tid=0x00007f16fc00e000 nid=0x72fc in Object.wait() [0x00007f17056d9000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000778044670> (a java.lang.Class for com.wu.mortgage.service.OrderServiceStartup)
at java.lang.Object.wait(Object.java:503)
at com.wu.mortgage.service.OrderServiceStartup.main(OrderServiceStartup.java:47)
- locked <0x0000000778044670> (a java.lang.Class for com.wu.mortgage.service.OrderServiceStartup)
"VM Thread" prio=10 tid=0x00007f16fc14a000 nid=0x7309 runnable
"Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f16fc01f800 nid=0x72fd runnable
"Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f16fc021800 nid=0x72fe runnable
"Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007f16fc023000 nid=0x72ff runnable
"Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007f16fc025000 nid=0x7300 runnable
"Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x00007f16fc027000 nid=0x7301 runnable
"Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x00007f16fc029000 nid=0x7302 runnable
"Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x00007f16fc02b000 nid=0x7303 runnable
"Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x00007f16fc02d000 nid=0x7304 runnable
"Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f16fc106800 nid=0x7307 runnable
"Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x00007f16fc102000 nid=0x7305 runnable
"Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x00007f16fc104000 nid=0x7306 runnable
"VM Periodic Task Thread" prio=10 tid=0x00007f16fcbed800 nid=0x7314 waiting on condition
JNI global references: 522