Fuzzing
Fuzzing involves sending malformed strings into application (and web) input and watching for unexpected crashes. There are lots of interesting tutorials as to how to do this. Many times after finding this malformed input, you would then use assembly to figure out how to deliver shellcode (or how to deliver the exploit).
Fuzzing does have practical usage in software development, but it is also a tool used by hackers to find vulnerabilities in applications.
Many fuzzing tools: ComRaider (activeX), see /pentest/fuzzers