Don't let it leave your site. Also keep your iOS firmware up to date. A skilled hacker could use a vulnerability to get into your system. Examples of such a vulnerability include the jailbreakme jailbreaks.
Have a passcode set, and set it to wipe the phone after 10 failed login attempts. With a passcode set you can only access the phone from a computer that has previously accessed it, on any newer computer you need to give the passcode. A very skilled hacker can probably get in through DFU mode, but ordinary mortals cannot - and no available tool will bypass it, as far as I know.
Oh, and SSH is pretty safe, as long as you remember to set roots password and mobiles password. But remember, if you restore and rejailbreak the phone, they are both set back to "alpine", even if you read in from backup.
If you're on the same wifi network and you haven't changed the SSH password from alpine, then yes someone could easily SSH into your phone. They have to figure out what your current ip on that netwrok would be, but I'm sure it wouldn't take long to discover it. One of the first things I do after a jailbreak and installing OpenSSh is change the root and mobile passwords.
There are plenty of tut's on how to do it using mobile terminal or terminal on your mac. (I've never done it with a pc.)