Yes, you can use iCloud Drive the way you describe, but it takes a fair amount of work (Save image to Files then delete from Photos), and the Files app is not optimized for displaying photos - it's harder to find images, as you have little more than the file name and the upload date to work with. Further, I do a lot of image editing, and iCloud Drive has no photo editing tools (there are markup tools, but that's not what I want).
I'm a fan of iCloud Photos' Optimize Storage. Yes, it does use some storage space on the iPhone, but in my case, I have 103.2 GB of Photos in iCloud, and just 3.43 GB of Photos data usage on my iPhone (Settings > General > iPhone Storage > Photos). The way Optimize Storage works, if you don't view a photo on your iPhone, the full-size image will eventually be removed from the iPhone automatically, while leaving it in the cloud. What's left on the iPhone is the thumbnail-size image so that you can browse through your entire image catalog (each thumbnail is around 150 KB or smaller). If you tap on an image (to view full-size) and it is not in full-size on the iPhone, it is automatically downloaded to the phone. If you don't view it again for a while, it'll automatically be removed from the iPhone again.
From my perspective, I prefer to use the Photos app, as it's optimized for viewing, locating, and editing photos. But if you want zero photos storage on your iPhone and are willing to do all the manual labor necessary to achieve that goal, go right ahead and move photos into iCloud Drive.