Honestly, if you feel you paid too much, that feeling is going to get worse as time goes on.
When I got my first MacBook, about 3 months in, I got curious to see what was available on the PC side of things. I had believed the Apple hype that I had gotten what I had paid for and it was a good value. But as I went to newegg and started looking... I realized I had paid way too much money for my MacBook. At that time my MacBook was $1406. For that I got a 2GHz Core 2 Duo, 1GB of RAM, an 80GB HDD, an Intel GMA 950 GPU, and a 13.3" screen. For just over $800, a little more than half as much, I could have gotten a similar CPU, 17" screen, double the HDD space, twice the RAM, and a GeForce Go 7600 GT.
Conveniently, around that time OS X started to show its true colors. Meaning it wasn't as stable as Windows, plus the optical drive started to fail and Apple's build quality started to shine with the case starting to discolor and crack. I've regretted my purchase every day since then.
Especially after going through 2 replacements thanks to poor build quality and even worse repair work that left the first two systems in even worse shape than I sent them out in.
And now the hardware difference between a PC and a Mac is even greater.
For the same $1299 as that entry UniBody MacBook, you can get a PC with a 1680x1050 15.4" screen, 2.4GHz Core 2 Duo, 4GB of RAM, 320GB 7200 RPM HDD, 1GB GeForce 9800M GS, blu-ray, HDMI, VGA, card reader, etc.
If you want a 13.3" system, Dell will you the Studio XPS 13 which ships with a 2.4GHz Core 2 Duo, 4GB of DDR3 RAM, GeForce 9400M + GeForce 9500M 256MB running in Hybrid SLI, 320GB 7200RPM HDD, HDMI, card reader, etc.
For $999 you can get a 15.4" or 16" system with specs similar to the 15.4" system above, but with a 1GB GeForce 9650M GT. Thats essentially more GPU power with FOUR TIMES the video memory of the MacBook Pro costing twice as much, and more GPU power with double the video memory of the $2,499 MacBook Pro.
When it comes to other important things, like extended warranties, Apple's are the worst. You pay more than you would for one from a PC manufacturer and they don't even offer the same perks, like liquid damage, accidental damage, on-site support, or even 24/7 phone support.
As for build quality, despite the fact that people love to go on about the aluminum casing, the aluminum housing the screen and covering the bottom of the UniBody systems is soft and weak and dents and scratches extremely easily. The "UniBody" piece itself bends if hit. A plastic system will come out of the same hit without a mark on it.
Apple's plastic MacBooks are built even worse. They have known and admitted build quality issues that cause the casing to crack and break down just from regular use. A plastic PC won't have this problem.
As far as OS X being better than Windows? Well, there is quite a bit OS X can't do that Windows can, like have good game support as well as play modern video like blu-ray. OS X can't even play DVDs up to the same level as Windows, as OS X currently does not support full bitstream decoding of video.
OS X is also not any more stable than Windows. I know this from experience. Tiger and Leopard are the only OSes I've used this entire decade that have a knack for hard locking the entire system. Prior to owning a Mac, I hadn't seen an OS completely and fully lock up since the original Windows 98.
Honestly, you really should return the system if you're already feeling buyers remorse. It will only get worse with time. On top of that, if you go with a PC notebook, you'll get a much faster and much more capable system for the same price as that MacBook, and you'll get an operating system that can actually be called "modern" and will actually take advantage of the hardware in your system. You'll also have the option of protecting your system with optional warranties that will cover both failures as well as accidents.
