(I'm a programmer at heart, so I tend to like things more "structured" and "logical" versus "pretty"...)
If you work as a programmer, make sure that all your shell scripts handle paths with space characters correctly. Well, there are other problems, like "rm -r *" being a valid filename, for example...
You can even enter "/" characters in a file name in the Finder. (Fun exercise to a programmer with no experience in Mac programming to figure out what happens and why it works).
----------
Can you give an example of an app that has such a problem? I've never encountered one.
The only obvious thing that isn't allowed is the ":" character in the Finder (or the "/" character in a Posix file name). Other than that, any Unicode character is allowed. "." as the first character hides the file. And the Finder will tell you if renaming a file changes it's suffix, and changing the suffix may break things.
You _will_ have problems exchanging files with Windows when you use certain characters.