Debugging Flow For A Process

I've been stucked with a job which took a lot longer than what it is suppose to take to complete. A friend of mine shared his gems of tricks on how he troubleshoot it, and with a little bit of tweak to my taste, I've came up with a flow(well, almost all of it still from him :p) which I'm logging it here so that I won't forget it.


#1 Find the offending process id (_pid_)

ps -aux | grep job_name

#2 Look at the entire hierarchy of the pid and look at where it stops

pstree -pulna _pid_

#3 Look at the trace of the running program.

strace -t -s 22222 -p _pid_

#4 Look at the read/write IO activities 

cat /proc/_pid_/io

Useful Links

http://www.linux-tutorial.info/modules.php?name=MContent&pageid=84
http://chadfowler.com/blog/2014/01/26/the-magic-of-strace/