OSX 10.11.1 File Server SMB shares to Windows 7 users using AD VMWare machine

[OT-Mac]

Hi All,

We have updated our main file server from OSX 10.8.5 to 10.11.1. This was mainly done to ensure stability and to help our Windows 7 users who have recently had renewed issues with opening files and keeping them open on the older 10.8.5 server (as per previous problems with white pages on open PDF documents and random drop-outs of the server connection discussed on this forum about 2 years ago - see discussions on LANMAN regedits). A recent Windows 7 update has caused this issue to re-appear. Early testing on OSX 10.11.1 showed this was finally fixed. SMB3 connections to PCs seem to have fixed most issues. However we still have some remaining issues. Here is a bit of a run-down of our systems and processes to both help others who may experience the same problems and hopefully get some help fixing the last of our issues:

Our setup:

• MacPro (cheese-grater) running 10.11.1, does main file shares using OSX server app as well as Kerio Email Server.
  
• VMWare on MacPro runs an AD server (Windows 2008 Server) and some legacy apps.
• Client Machines are all iMacs, some running bootcamp Windows 7, and others running Mac OSX10.8.5 and parallels with Windows 7 for legacy or specialty apps.

What now works with OSX 10.11.1 file server (that didn't or only marginally worked with the 10.8.5 and bombed out completely with 10.9.x):

• PDFs on server now open properly on Windows 7 clients without "white screens" after short periods of timeout.
• Files are now correctly showing up as "in use" if opened on multiple machines at once.
• Windows Applications can now be installed directly off the server (without having to copy them to the users desktop first).
  
• SMB share stability seems to have improved and is faster.
• General server stability seems to have improved.
• The new VMWare for the AD server also seems to have improved stability (touch-wood!).

What we needed to do to get there:

The upgrade was pretty smooth (plain "update" no need to "reinstall"). We thought we had it all under control, with no issues reported for half a day. This changed after lunch when multiple people using Windows 7 reported access to drives dropping out constantly. We traced this down via PAM error logs to the SMB service crashing once a certain number of files had been opened. Thanks to some helpful advice adapted from here:

http://docs.basho.com/riak/latest/ops/tuning/open-files-limit/​

This disaster has been averted. Basically, OSX 10.11.1 seems to have an inbuilt limit of 256 open files! You can check your limit by typing this into the Terminal:

launchctl limit maxfiles​

We changed that to 65,000 by following the advice of inserting the limit.maxfiles.plist and limit.maxproc.plist and this has made the SMB service more reliable (see above document link for plist requirements).


What we still have issues with (any help appreciated):

While the SMB service now keeps running, there are times when for some unknown reason it "drops out" in that file browsing in Windows Explorer says the sharepoint is unavailable. Waiting a few seconds usually lets the PC users continue browsing. This problem is intermittent and occurs more when heavy file browsing is being done. When it does occur, multiple people will have the same issue. The only OSX log entries we see are:

smbd[47114]: query_directory error<0xc000000f>​

and

digest-service[47082]: digest-request: init return domain: DOMAINNAME server: SERVERNAME indomain was: <NULL>​

These logs appear all the time in small doses, but when the file browsing on Windows Machines bombs out, these log entries seem to flood the logs. Sometime we also get this message in the midst of the flood:

com.apple.xpc.launchd[1]: (com.apple.smbd[20695]) Service exited due to signal: Segmentation fault: 11​

This problem has not yet been fixed. If anyone has any fixing ideas we're happy to try them .

 

[OT-Mac]

PS. Other things that now work with OSX10.11.1 Server (that didn't under 10.8.5) are:

• AutoDesk Revit File access
  
• AutoCAD extraction tables

 

[chrfr]

digest-service[47082]: digest-request: init return domain: DOMAINNAME server: SERVERNAME indomain was: <NULL>

Is this the actual error message or are DOMAINNAME and SERVERNAME sanitized for this post?

Hmm, the forum just ate a longer response I'd written and don't have time to recreate right now.
But the summaries:
OS X doesn't have a 256 files open limit. My computer here has over 5000 open right now.

Is the server bound to AD or does it have its own credentials?

You say "SMB keeps running" but it is crashing when you see the seg. fault errors, so it doesn't actually keep running. OS X restarts the smbd process.

 

[OT-Mac]

Hi Chrfr,

That is correct. SERVERNAME and DOMAINNAME are the sanitised version and include our actual server and domain names. The NULL remains NULL in the logs.

I may not be expressing the issue correctly, but I can assure you that the resource limits on a standard unmodified OSX 10.11.1 server return the values:

maxfiles 256 unlimited.
​

These limits may not apply to normally opening documents on your computer but they did affect our SMB sessions and created lots of PAM logs before forcing us to restart the SMB service. When changing them to 65000 using the above link's plist creation instructions our issues with SMB bombing out completely in this manner were removed.

The Server is bound only to the AD server (the VM running on it). We used to run an AD/OD setup, but the OD previously had issues in this triangle (under the old OSX 10.8.5 server) and we didn't really need it, so we removed it.

In terms of SMB keeping on running now; it may well be crashing, but OSX restarts it very quickly. This is better than SMB crashing and NOT restarting as was the case with the previous max files limits. We know that there is still a problem with SMB and Windows 7 clients that is causing these intermittent crashes/restarts of SMB - hence why I've posted here to see if anyone else has come across this and/or may have resolved this.

The segmentation fault only occurs much less than the other two log messages. The intermittent failure to be able to browse the SMB shares using Windows 7 may not be related to the segmentation errors. However the other two messages definitely pop up more when there are share access issues. Unfortunately there doesn't appear to be much other log information to provide and/or to help troubleshooting.

 

[vid99]

smbd[47114]: query_directory error<0xc000000f>

Wondering if you ever found out anything about this one, we are seeing it also.

OSX 10.11.4 with 5.1 server has been better for the Windows 7 people in our office, but still kind of crappy. Dropouts related to the smdb error?

Also clients use a program called Bluebeam for PDFs and opening off the server and making changes causes corrupt PDFs.

ugh, so many smb issues since the 10.6 days, think its time to just go Linux or Windows File server...boo