AFP rendering Mac servers unusable

Discussion in 'Mac OS X Server, Xserve, and Networking' started by fragra, Jan 14, 2009.

  1. fragra macrumors newbie

    Joined:
    Jan 14, 2009
    #1
    We've had problems for months now, big problems with our MacOSX 10.5x Server in connection with server based user folders.

    Apple has been no help. Complaints are piling up worldwide and the threads are becoming longer and longer. Even the last update 10.5.6 doesnt address the issue, though apple said it would.

    Here's our problem briefly. On our current XServe systems, which are made available over the AFP protocol server-based user folders, the AFP service is completly using the CPU. On our system with 8 cores the CPU burden is up to 750%. Work by all logged on users is no longer possible because all applications no longer react and the computer hangs up eventually. The only thing left is to restart the server, but in most cases this does not help because the problem returns after a new start. The AFP service takes the whole CPU. For a company like ours with over 100 server-based workstations this is inacceptable.

    Administrators around the world are frustrated and clueless. Apple says nothing. There is no feedback, help from Apple Enterprise support is like a trying to poke-through fog. Many now believe that there is such a serious problem with the AFP proctocol that even Apple developers are helpless. Please this thread at Apple: http://discussions.apple.com/thread.jspa?messageID=8798468 The problem is evidently so bad that companies are experiences damages from down time. And there is also a new website, that perhaps you would want to publicize with a notification. www.go2xserve.com

    We administrators no longer know where to turn. Apple is not being helpful and provides no feedback or support of any kind. Thanks much for any help you can provide.

    Yours truly,
     
  2. mikedavis macrumors newbie

    Joined:
    Dec 13, 2008
    #2
    fragra,

    are you 10.5 servers updated from 10.4? or, are they fresh installs of 10.5?
     
  3. fragra thread starter macrumors newbie

    Joined:
    Jan 14, 2009
    #3
    in April 2008 we tried to update from 10.4 to 10.5.3 but unpredictably apple had a bug in the afp protocol, so the managed useres disconnected after a few minutes automatically.

    we installed totally new with 10.5.4, at the moment we are at 10.5.6. it looks like each version had the same problem!
     
  4. Sayer macrumors 6502a

    Sayer

    Joined:
    Jan 4, 2002
    Location:
    Austin, TX
    #4
    So you are running apps over AFP on over 100 desktops? Have you thought maybe you should restructure instead of trying to fit a square block through a round hole?

    Why not net-boot from a single pre-made image and use AFP for sharing files, not apps and I presume the data the apps are working with?

    Why does the app *have* be run from one server? Sounds like a poor app choice, or skimping on buying a proper client/server solution.
     
  5. foidulus macrumors 6502a

    Joined:
    Jan 15, 2007
    #5
    Because stuff like this worked fine in Tiger? It's harder and harder to be an Apple apologist after the non-stop bug-fest that is Leopard, however Apple refuses to admit their is a problem and as Leopard goes on, the bugs become more numerous in my opinion.

    We haven't switched any of our servers over to Leopard, and I am incredibly hesitant about doing so, but since Apple forces you to use Leopard with new hardware, we will have to make the plunge soon.
     
  6. corbywan macrumors regular

    Joined:
    Feb 4, 2008
    Location:
    Forest Grove, OR
    #6
    I must have missed it. The the OP say apps? I thought it was just home folders on the server for documents.
     
  7. fragra thread starter macrumors newbie

    Joined:
    Jan 14, 2009
    #7
    it is "just" home folders on the server for documents, no apps!
     
  8. SC68Cal macrumors 68000

    Joined:
    Feb 23, 2006
    #8
    I'm surprised that you were even able to get close to that kind of usage with AFP shares. Sadly, you're going to be waiting in vain for a fix from Apple. Your best bet is to switch over to SMB/CIFS which can and does handle those kinds of loads. Apple never tests anything that they do on large enterprise deployments because they actively avoid enterprise solutions and their company culture does not put a priority on addressing enterprise needs beyond putting up a nice PR web page about VT when they bought 200+ Xserves for a computing grid.

    So many admins got screwed over when they put out OS X Server 10.5 and everyone's Active Directory got screwed up. Did they ever test it? Apparently not! It took them until the point two update to get their stuff together and send everyone a fix. I know myself - after having a colleague wrestle with trying to keep his server and client deployments on the same release that Apple wants to sell everyone a shiny product, take the money, and run.

    From the way Apple conducts themselves - it's almost like they're laughing behind your back when you purchase an XServer.

    100 clients for simple file services is not a square peg in a round hole. That's Servers 101 buddy. Have you ever administered anything bigger than two computers attached with a crossover cable?
     
  9. Les Kern macrumors 68040

    Les Kern

    Joined:
    Apr 26, 2002
    Location:
    Alabama
    #9
    A while back I was having some terrific issues with laptops linking to my OD servers... response was sluggish, lots of beach balls. I have an OD master with replicas, about 20 Xserves 10 with home directories and 2200 clients. Eventually it was traced to my HP AP's and switches. Once that was identified things are a lot better. Sure, there are occasional OD issues like some of the 10.5/Legacy settings I have yet to solve, but during all that I have never, ever seen numbers as described here. And Apple technical support has been nothing but helpful along the way.
    So you say you're surprised at the AFP usage rates. Why? My servers are hammered daily with little problem. You say we will "wait in vain for Apple to fix this". Why's that? Do you have inside information you'd like to share? If not, this is nothing more than bashing. I have to assume you have first-hand knowledge?
    Being no fan-boy, I'd be the first to scream at Apple if my mission critical systems take a hit because of their gear, but I pick my battles and TRY make SURE it's nothing my limited knowledge has missed. And, since I have a brain that is sometimes the size of a pea, missed things I have, and screamed I have. But in the end it's corrected, with yours truly looking like a dolt.
    This being said, I will be investigating this issue more... perhaps there is something to it. Always best to keep an eye on everything.
    It's what we do.
     
  10. Les Kern macrumors 68040

    Les Kern

    Joined:
    Apr 26, 2002
    Location:
    Alabama
    #10
    Sometimes when the issue is that serious I like to schedule a visit by an Apple SE. They are pretty pricey, but make no mistake... they are to a person brilliant. Some of the projects I've had them assist me on have been incredibly complicated, and in every case they finish the job quickly and completely. It's obvious that your problems are affecting not only the productivity of your users, but it quite probably is destroying the goodwill we try so hard to keep healthy... and what they charge I would consider chump-change if there is resolution.
    Having an SE on site is a lot different that talking to some nebulous individual hundreds of miles away, concerned or not.
     
  11. fragra thread starter macrumors newbie

    Joined:
    Jan 14, 2009
    #11
    yeah but i´m not old and slow, it´s jut a mac! it´s bsd, we can do it.
    we are in a very close connection with an applecare technician but he is helpless...
    even the "new" appleshare version he gave us didn´t make it.

    There are a number of possible workarounds for this, whose effectiveness range, until now we have no solution.
    The top 10 are:
    1. Setting AFP wan threshold
    2. Turning off spotlight on server by marking AFP volume as private
    3. Turned off Time Machine even though no backup volumes where defined
    4. Set kern.maxfiles=200000 and kern.maxfilesperproc=5000 sysctl’s
    5. Turned DS_Store off on both clients and servers
    6. Turned off smb (windows samba) file server
    7. Disabled auto-disconnect in AFP after idle time.
    8. Removed spotlight indexing on all afp volumes and deleted .SpotLight-V100 directories on AFP volumes.
    9. Verified that Host Cache Flushing is disabled on external RAID array.
    10. Set the following default: defaults write com.apple.desktopservices DSDontWriteNetworkStores true. Set as preference for all groups.
    11. Disabled kerberos for AFP authentication.
    12. Changed the fibre topology to Point to Point for all 4 fiber connections to the Promise VTrak array.
    13. Stop spotlight indexing by using the command: touch /Volumes/Sharename/.metadata_never_index
    14. Renamed odpac.bundle in /System/Library/KerberosPlugins/KerberosAuthDataPlugins/ to odpac.bundle_DISABLED
    15. volume with our home dirs has to keep at least 10% free for performance reasons

    --> so we did everything above but no. 1 did it --> wan_quantum and wan_threshold

    We’ve deployed the modified plist this morning to all of our clients, the server (AFP service) is running normal at 10% - 40%. i think that this was part of the problem. but we have to keep an eye on it, i don’t think this is the final solution.

    funny idea:
    -> HowTo: Make Ubuntu A Perfect Mac File Server And Time Machine Volume [Update6]
    http://www.kremalicious.com/2008/06/ubuntu-as-mac-file-server-and-time-machine-volume/
    "1. Modify and install Netatalk (Open Source AFP implementation)" -< last updated version in 2005

    my company is a very complex setup with some very different compartements.
    we want to get it worked out and don´t want to play around with some 10 or 20 clients...
     
  12. Les Kern macrumors 68040

    Les Kern

    Joined:
    Apr 26, 2002
    Location:
    Alabama
    #12
    Talk about thorough... copy/paste. I'll be looking more at this....
    :)
    And I saw this article at 548.
    http://www.afp548.com/article.php?story=20060329213629494

    Thanks.
     

Share This Page