In Tokyo at least (with my iPod Touch), I managed to find at least one patchy unlocked network every other block or so (most notably a widespread one called 'BB User").
Lots of coffee shops and fastfood joints will have free or paid for wifi networks & there seemed to be three blanket paid for ones (maybe one was called 'fon'?) that were even available deep in the metro. When I return in September I'll look into signing up to one - especially with the prospect of wifi VOIP apps to phone home for free.