Usual Suspects
1. Environment (Hardware and Software)
2. Code Issues (Bugs)
3. Data Issues (Corrupt or invalid data)
4. Software version compatibility issues. (Known issues).
Software Compatibility Matrix
Ideally we should create a Software Compatibility Matrix and write a script that will either pass or fail to enforce it. For instance, I was working with a custom testing framework written using Ruby, ChromeDriver to test the system from end to end using the browser. I had to spend almost an hour to find the cause of the problem. The workaround took just 5 minutes. It turned out that the latest version of Chrome broke the ChromeDriver. One hour could have been easily saved by creating a Software Compatibility Matrix that listed the versions of Mac OS X, Ruby, RVM, Chrome etc. This would state which versions are known to work with others without causing any problems. Any OS update that causes indirect update of any of the dependent software would be immediately caught by the script that enforces the software compatibility matrix for the custom testing framework.
When Rails is released, the compatible versions are usually specified in the release notes. This must be explicitly expressed in your project's software compatibility matrix. The usefulness will become obvious during minor/major upgrades to the gems and Rails. I had run into an issue where ActionMailer had locked the version of the depedent gem Mail to specific version that had a bug, it caused the rake script to crash everytime it tried to send an order confirmation email. The version of mail gem that had the bug fix could not be used because of the ActionMailer locking the version of the dependent gem. The only workaround was to set the SSL flag to true in the Rails configuration file.
