I've been using WinMTR every day after logging off and I've noticed a pattern, at least where I'm at. For reference, my ISP is Comcat (boo! hiss!).
There is massive packet loss when the signal is handed off from pnap.net to Level3.net - on the order of 15 to 20 percent.
Then there is more massive packet loss when the signal is handed off from Level3.net to Comcast.net. Again, on the order of 15 to 20 percent.
In addition, the transit across Level3.net averages 3x the ping response time PER SERVER of any of the other networks.
I don't know if Level3.net is the only feed pnap.net has, but it seems as if that network is the primary source of the bottleneck, at least where I'm sitting. Are other people noticing a similar pattern?