tell him instead , to keep iCloud separate , do not share your iCloud account with anyone. but offer to share or ' family ' your separate iTunes accounts. that way you get each others apps, tv shows, movies, music and books.
you don't want to share others calendars and phone books and browser tabs on your iPhones. too much information. I'm sure your brother doesnt need to know what websites you visit
when he says he wants your data from your phone, he probably would be more interested in the apps you bought, and you can share those purchases without having to share the iCloud password