Not necessarily. They may have used their upgrade to buy subsidized, and decided to sell instead of using it. And the 3G can be jailbroken, so there are definitely such iPhones for sale.
Also, there are some people who are buying (as new AT&T customers), waiting and paying the first month's bill, and then canceling the contract, which means they can pay the ETF and keep the phone.
There are a lot of ways for people to have 3Gs for sale, but it's going to cost more than just walking into an AT&T or Apple Store.