Skeumorphism is a term that refers to digital ornamentation modeled on real-word objects. Leather stitching may evoke a feeling a comfort with an unfamiliar user, but its value to the overall design is questionable (especially when it's conspicuous). Drop shadows are not ornamentation. They have a real utility and are essential to main principles of visual design. To say that drop shadows are skeumorphic is like saying buttons are skeumorphic.
The term has swelled and taken so many new meanings. It's stretched really thin. Soon people are going to complain the keyboard is skeumorphic because it's based on a typewriter, or the bubbles in Game Center are too skeumorphic, or the sounds, or the icons, or everything. Because when you boil down every design, it's based on something familiar; it has to be. Whether that something has a primarily real-world identity or not, it should make no difference. Everyone needs to drop the word skeumorphism from their amateur design vocabulary, or at least drop its negative connotation.