Here's one way to do it
In iTunes if you right click on the video and chose Get Info, then under the Options tab change the video type from iTunes U to Movie the video will show up under your movies on the ATV.
When you are done watching them just reverse the process and the videos will go back to the correct location (course) under iTunes U.
If you have a lot of videos for the course that may be a lot of work but it's the only way I know to do it.
Why Apple does not allow iTunes U videos to show up in the movie/video list by default I have no idea other than maybe they consider the courses to be more like iBooks.
Jon