It's not just you. I see a lot of packet loss throughout the day and especially in the evening - even to networks out of Cogent. Packet loss is awesome for VoIP by the way :)
A test from a few minutes ago to a server I rent which is usually fine. Loss was at the Alter.net (verizon owned) hop.
Ping statistics for 108.62.209.x:
Packets: Sent = 667, Received = 649, Lost = 18 (2% loss),
Approximate round trip times in milli-seconds:
Minimum = 19ms, Maximum = 73ms, Average = 43ms