I had no problem coming from a small school that few know of (granted, my programs aren't as competitive to enter as some). Got into seven top accounting and tax programs, one with nearly full funding (would have gone if I wouldn't have had to spend an extra semester there doing some prerequisites they required that no other schools did). My girlfriend goes to the same school I do and was accepted out of undergrad into a couple of psychology PhD programs (one with about a 2% acceptance rate, not sure about the other, but its well under 10%).
That said, we both have well above 3.9s, above average test scores, led clubs, volunteered, were hired as TA/tutors and spent time getting to know our professors (and she does research). Undergraduate is really what you make of it. If you take advantage of opportunities and get involved, whether at a small school or a larger well known one like Harvard, you'll likely succeed and not have a ton of problem getting into graduate school. If you don't put the effort in, even having Harvard on your diploma isn't going to help you much.
I also really think it depends on what you want to do. If you want to do neuroscience, the choice is pretty obvious, going to a small unknown school isn't going to give you the resources and opportunities to do much in undergrad and lack of resources will hurt your chances of doing work that will get you into graduate school. You have to go where the opportunites are in what you want to study; for some areas of study, you can do just as well at small liberal arts as you can at huge state or even Ivy, but for other areas, the resources just aren't going to be at small schools.
However, its certainly never going to hurt you to go to Harvard or another well known university if you get good grades, get involved, and work hard, especially if you're on a nice scholarship. Its not too likely that an admissions committee will look down on Harvard, et al and having a big name on your resume could certainly give you a little extra boost over a similar candidate for a lesser known school.
I will say, that in my experience, it is somewhat easier to get to know your professors at smaller schools. Every professor I've ever had knew my name and for the most part, they all still know it now; I had a sociology professor in the first semester of my freshmen year who still says "hi" to me using my first name when our paths cross. Most of my friends at UC Berkeley, UCLA, Irvine, CSUN, SDSU etc., can't say that. In fact, one of my friends was looking to connect with a professor he had taken one or two courses with in the past, he met with the professor five times, the guy didn't remember who he was once. Of course, if you get in as a TA/GA/DA/research assistant or you have a professor mentor you on a research project, it becomes significantly easier to connect with faculty. Personally, I think it takes more effort on the part of a student at a larger school, but that's not to say it is impossible.
Also, congratulations Fivetoadsloth, getting into JH is very impressive and quite an achievement!
