I sort of had the same experience. I relied upon iPhoto, then Aperture, because a mess of applications let me choose photos from them even when they weren't open. But then I started using applications that did NOT access iPhoto or Aperture libraries, and had to use the Finder. Yuck.
No matter what organization with folders I tried I ran into problems. The basic deficiency of a Finder folder system is that you cannot have a file in more than one folder. And that's true for not only images.