Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Auto Proxy Config not fetching wpad.dat

G

GrafAllrad

macrumors newbie
Original poster
Aug 12, 2010
1
0
We are trying to integrate our Macs properly into our local network.

Internet Access is provided through a Microsoft TMG as a web proxy.

This proxy is configured to supply a proper wpad.dat file for Auto Proxy Discovery.

All Micro$oft clients in our network properly fetch the wpad.dat file, access from these clients shows up in the proxy server logs in a proper way, they all access http://wpad/wpad.dat. Everything works here

On the Macs there seems to be a slight problem:

When configuring a snow leopard 10.6.4 machine for Auto Proxy Discovery, the mac is accessing the proxy server and requests the wpad file, however it seems like mac os x adds %2500 to the end of the request, so the fully logged request is logged as http://wpad/wpad.dat%00

And this is the problem, the Proxy Server throws out an error code and is not delivering the wpad.dat file. Where does this %2500 come from ?

We tried to reproduce this error on another Mac machine - bingo - same problem.

:confused::mad::mad:

Is this a known problem ? Google brought up only 1 hit, same problem - no solution


Any ideas anybody ?

Thanks upfront, GrafAllrad
 
Make sure there aren't any "junk" characters in the field where the address of the PAC file is. You can test this by trying to select the end of the field, beyond the text. If stuff is there, delete it.
 
GrafAllrad said:
When configuring a snow leopard 10.6.4 machine for Auto Proxy Discovery, the mac is accessing the proxy server and requests the wpad file, however it seems like mac os x adds %2500 to the end of the request, so the fully logged request is logged as http://wpad/wpad.dat%00

And this is the problem, the Proxy Server throws out an error code and is not delivering the wpad.dat file. Where does this %2500 come from ?
Click to expand...

Spent a few hours today troubleshooting this with a colleague. Loooong explanation to follow, you might want to skip to the end if you're not interested in the technical jargon.

--------------------Explanation--------------------
The problem is due to an incompatibility between Microsoft DHCP and OSX' WPAD implementation. (BTW, OS X does implement WPAD properly, contrary to some claims by the majority of Google posts on the topic.)

The WPAD protocol first tries settings supplied in DHCP - option 252 - and then falls back to trying to resolve through DNS if that fails (i.e. the default of http://wpad/wpad.dat.)

Trouble is, when Microsoft's DHCP implementation (and possibly other implementations that interpret the RFC the same) supplies a string in a DHCP option, it null-terminates it. The DHCP RFC says that this "should" not be done (i.e. it's permitted but not recommended.) However, the RFC also says that clients that receive options "should" cater for the possibility of the string being null-terminated.

So what ends up happening is that neither Windows Server or OS X plays ball - Windows passes OS X a null terminated string and OSX interprets it literally, appending a double-escaped null character to the URL (%25 in a URL normalises to the "%" character, and %00 is the null character. So %2500 = %%00 which gets interpreted as null.

It looks like the problem is more widespread than just OS X (I found some references to other Linux DHCP client implementations with similar challenges.) The references I found indicate that the problem was fixed in later releases, but there's no telling when this will be fixed in OS X (for backwards compatibility reasons, I don't expect Microsoft to ever "fix" this in their DHCP server implementation. There's just too much that can break.)

--------------------Summary/workaround--------------------
So, to cut a very long story short, the workaround is to remove Option 252 from the DHCP server and rely solely on DNS resolution - i.e. the client must be able to resolve the "wpad" shortname. Also, unlike Windows cilents, OS X seems to cache the option from when it receives its IP address (rather than retrieving it again when the browser launches), so to get rid of the option you will need to do something like disconnecting/reconnecting the cable to refresh the DHCP options.

PS - Keep in mind that for Windows Server 2008, the DNS server will block queries for wpad for security reasons - they must be explicitly permitted as per http://support.microsoft.com/kb/2003485.

PPS - unlike most posts on the topic, it is NOT necessary to enter the wpad.dat URL in the PAC script dialogue in the proxy settings of OS X - using the normal "autodetect proxy" setting works just fine, provided the above is taken into consideration. All entering the URL in the PAC script field does, is cause OS X to ignore the DHCP Option 252 settings that it receives.

Hope that helps someone...
 
MacBook don't fetch Micro$oft DHCP option 252 wpad.dat

Thank to JannieH very detailled explanations, I got the crazy idea to create a file named wpad.dat%00 in my proxy server who deliver the file. So I have one for PC (wpad.dat) and one for Mac (wpad.dat%00).:)

And you know what ? it works !!! :cool: my MacBook get the file and use the proxy.
So I guess DHCP option is parsed by MacOs X. I don't know how to check but I know my Mac now browse Internet with only "Auto Proxy Discovery" option checked. I also know I don't have any DNS resolution on wpad shortname. And finally I know it was not working before I create this file.

If it could help someone else, let me know
 
Last edited:
langldid said:
Thank to JannieH very detailled explanations, I got the crazy idea to create a file named wpad.dat%00 in my proxy server who deliver the file. So I have one for PC (wpad.dat) and one for Mac (wpad.dat%00).:)
Click to expand...

Excellent bit of additional information. What web server/OS are you running on your HTTP server delivering the file? How did you create the file - is it merely a plaintext %00 that you added to the name?

If I recall correctly, this did not work on Windows IIS when I did troubleshooting; CodeRed and other worms/attacks several years ago made the IIS team completely paranoid about escaped or double-escaped characters, so everything is aggressively normalised the moment it enters the application stack, long before it reaches the filesystem layer.
 
MacBook don't fetch Micro$oft DHCP option 252 wpad.dat

O.S. : Windows 2003 standard WebServer : Apache 2.0.59 (Win32)

We added that few years ago for supporting ".dat" type into apache's httpd.conf : "AddType application/x-ns-proxy-autoconfig .dat"
We didn't change anything to that.
Here's all we did :
Copy "wpad.dat" as " wpad.dat%00" (Format is still basic plain text file).

Worked straight after this ! :)

On IIs6 (W2003) it also works but you have to define an associated "MIME Type" (In IIs Console, right clic on "local computer" -> Properties ->"MIME Types -> New) :
Extension : .dat%00
Mime type : text/plain

We didn't try on IIS 7 for now but it will come :)

Cheers!
 
Another solution...

I had noticed the %00 appended to MAC's request for DHCP.

I've added a question mark ? to the end of the wpad string in DHCP. So:

http://wpad/wpad.dat?

This seems to work quite nicely.

Now if someone can help me getting iphones ipads to use either DHCP and/or DNS to get wpad settings I would be very grateful indeed! (I don't want to write anything in to the iPhone's settings: I'm prepared to go as far as selecting "Auto" on the config screen but leave the space for the URL blank...)
 
Has anyone successfully pushed proxy settings with wpad.dat to an iOS device (iPad, iPhone) using DHCP option 252?

I've can see option 252 being sent in the sniffer trace from the DHCP server but I can't tell what the iPad is doing with the information. I don't see any hits to the web server hosting the wpad.dat file and the HTTP proxy setting on the iPad doesn't show the HTTP Proxy URL after renewing the lease. I've tried leaving it on the default 'off' and also selecting 'auto' and leaving it blank while renewing, neither work.

anybody figured this out?
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back