Both o/s's will read and write, OSX will see your windows partition and you can drag and drop files within OSX, but if you are booted into windows you have no access to your mac partition, afaik.
Fat32 is the only one that can be read/written in both operating systems, and yes it does have the 32gb partition limit and a 4gb file size limit.
What do you exactly need more than 32gb for? Is this on the same machine or on an external to go between two machines? If 32gb isn't big enough then you can start burning DVD's or wait until a flash drive comes out that's larger than that