Can't connect to server/TM folder

I'm running latest version of server on a mac mini with latest Mavericks. From my other mini (also running current mavericks), I have set full access and permissions via a group I am part of. I recently have been having trouble connecting to the server; Time Machine looks for the proper folder, and in the finder, when I select the server, I just get the linux spin in the corner of the finder winder and the "Connecting..." with the "Connect As..." and "Share Screen..." options. When I click the first, nothing changes; when I click the second, Screen Sharing opens but nothing else happens (no window with the server gui pops up).

What's odd is that I can connect to any folder or drive on the server using "Go-->Recent Folders", and iTunes home sharing works perfectly, so it's just a weird problem with Finder, Screen Sharing, and Time Machine. I've rebooted a few times and everything works fine for a while, then at some point I just get logged out (but obviously not really, since I still have access) and those problems present themselves...