Fixing Mavericks's Outdated HTTPS

[Wowfunhappy]

Edit: Please move further discussion to new thread in the more appropriate Early Intel Macs forum. https://forums.macrumors.com/threads/fixing-https-issues-on-old-versions-of-os-x.2281326/

I run Mavericks on my primary computer, and that computer is where I spend most of my day. Its my favorite OS despite its age, and for the most part, the age isn't a problem.

But one place where age does cause problems is in HTTPS support. Because Mavericks doesn't support newer cipher suites, it's unable to connect to a lot of websites nowadays. This isn't a problem in Firefox, which has its own HTTPS implementation, but it creates issues in a host of other apps. Apple Mail is filled with broken images, Dashboard widgets like Delivery Status are unable to connect to various services, and RSS Readers like NetNewsWire can't be used with certain RSS feeds.

This sucks, so, I fixed it! The trick is to set up a proxy server which acts as a "Man in the Middle" for all of your computer's HTTPS traffic. I went with Squid, because it's super lightweight, and because I was able to compile a version that works on Mavericks! It took a very long time to configure, but now that I have it set up, it should be easy for anyone to run!

This is now a proper .pkg that should be easy to install. It includes a hack to force the Dictionary app to use the system proxy.

https://jonathanalland.com/downloads/wowfunhappy-https-proxy.dmg

 

[Wowfunhappy]

@vorob I know you're on Lion and this is the Mavericks section, but let's move the conversation over here so others can find it.

If you search for "Squid" in Activity Monitor, does anything come up?

 

[vorob]

In CPU tab? No it’s not there. Just two pictures, maybe they will help somehow.

 

[Wowfunhappy]

In CPU tab? No it’s not there. Just two pictures, maybe they will help somehow.

Okay, so Squid isn't running. Does anything happen if you run /Library/Squid/squid in the Terminal?

 

[vorob]

Did it, that’s what happened:


Need to clarify some thing. I’m new in macs, bought this ancient device for old gaming purposes, and study OS X for fun.

Ps, Growl is not part of your stuff? This thing appeared in settings and dunno from where.

 

[Wowfunhappy]

Did it, that’s what happened:

View attachment 945658
Need to clarify some thing. I’m new in macs, bought this ancient device for old gaming purposes, and study OS X for fun.

Ps, Growl is not part of your stuff? This thing appeared in settings and dunno from where.

Okay, yeah, so that means it's not going to work on Lion; it would have to be recompiled on a Lion machine, which I don't have. Sorry, I tried!

(Growl isn't from me, that's a notification system used by a lot of old apps, before Apple implemented their own notifications system.)

 

[vorob]

Okay, thx anyway. How can I revert all this things? No files were overwritten? I can simply delete everything? And how can I unload thing loaded on terminal step?

 

[Wowfunhappy]

Okay, thx anyway. How can I revert all this things? No files were overwritten? I can simply delete everything? And how can I unload thing loaded on terminal step?

Yep, just delete the directories you created, delete the certificate from keychain access, and disable the proxy. Once the plist is deleted, the launchd job will get automatically removed on next boot. The advantage of not having a fancy installer package is you know exactly where all the files are, because you put them there!

If I ever manage to get a Snow Leopard VM up and running I'll compile a new copy and let you know (generally, compiling on an older OS means it will work with anything newer).

 

[maverick28]

Hello,
I did everything as per instructions, but now my Mac can't connect to the Internet. Connections in Safari are broken, Weather and Web-clip widget don't load. I loaded your launchd instance with admin permissions sudo launchctl load /Your/Agent/, BTW.
Are you sure we need this proxy?
Also, wouldn't it make a difference, and for the better, if we trust all of the policies in the certificate?
[automerge]1598093433[/automerge]

 

[Wowfunhappy]

Are you sure we need this proxy?

Well, if you're not having https issues, you don't need it. I was noticing lots of broken images in Apple Mail, and lots of RSS feeds wouldn't load in NetNewsWire. And the Delivery Status Dashboard widget couldn't sync to Junecloud, and QuickTime had trouble playing some live https streams...

Same questions as I asked vorob, is Squid running in Activity Monitor? If not, what happens if you run in the Terminal:

/Library/Squid/squid

Also, wouldn't it make a difference, and for the better, if we trust all of the policies in the certificate?

It should have no effect whatsoever.

 

[maverick28]

Activity Monitor shows nothing. Terminal output was:

WARNING: Cannot write log file: /Library/Squid/logs/cache.log /Library/Squid/logs/cache.log: Permission denied messages will be sent to 'stderr'.

I use Vienna as an RSS reader, I see refresh failure warning badges next to every subscription entry in the list.

 

[Wowfunhappy]

WARNING: Cannot write log file: /Library/Squid/logs/cache.log
/Library/Squid/logs/cache.log: Permission denied
messages will be sent to 'stderr'.

Okay, that would do it!

Is there a "logs" folder in /Library/Squid? If it's already there, can you change the permissions so that "everyone" can "read & write" The latter really shouldn't be necessary, but, I don't know why else permission would be denied.

Afterwards, try the terminal command again.

 

[maverick28]

I previously had a lot of issues with Mail failing to connect to Yahoo servers, iTunes 11 rejecting my AppleID credentials combined with empty placeholders in the main iTunes Store window.

Here's what Mail's Connection Doctor shows me, the problems with iCloud SMTP has been a recurring issue:

 

[maverick28]

Ok, it finally was able to write a log. Here's what the log said:

2020/08/22 14:18:05| FATAL: failed to open /Library/Squid/squid.pid: (13) Permission denied exception location: File.cc(190) open

 

[Wowfunhappy]

Ok, so, it finally was able to write a log. Here's what the log said:

2020/08/22 14:18:05| FATAL: failed to open /Library/Squid/squid.pid: (13) Permission denied exception location: File.cc(190) open

Okay, I don't know what is going on with permissions.

Try changing permissions on the whole /Library/Squid folder. Set it so "Everyone" can "Read & Write", then click the ⚙ and "Apply to enclosed items".

 

[maverick28]

Changing permissions helped. Apparently, some process running on behalf of the system needed to have an access to Squid. Widgets load, Safari works. iTunes Store, Yahoo still rejects to sign me in.

 

[Wowfunhappy]

Yahoo still rejects to sign me in

This was happening before you set up the proxy, right? The proxy didn't break it, it just didn't fix anything?

If the proxy actually broke it, you can tell the proxy to not intercept Yahoo traffic. Open /Library/Squid/squid.conf, and find the line:

acl excluded_sites ssl::server_name .apple.com

And add to the end ".yahoo.com". So the line reads:

acl excluded_sites ssl::server_name .apple.com .yahoo.com

After that, you'd have to run /Library/Squid/squid -k reconfigure so Squid sees your changes.

It would be weird if you had to do this, though. The reason apple.com is excluded is because the iMessage app uses certificate pinning (ie, it ignores the Squid certificate you added to your Keychain). That shouldn't apply to anything in Mail...

 

[maverick28]

Yes, it was happening before I installed Squid. Is it safe to remove "apple.com" so that iMessage wouldn't break?
Not that I can't live without Yahoo, it's a crappy service, anyway. I just wanted to see how it would work with this mini-hack, out of curiosity.

 

[maverick28]

Oops, I misread your last message, my excuses. I need just to append it.

 

[maverick28]

Actually, I figured out from the conversation I don't need to change that .conf file at all because Yahoo is broken with me, anyway, so the proxy has nothing to do with that. It only succeeds in Mojave and iOS.

 

[Wowfunhappy]

As a quick example of something fixed by this proxy, try going to https://www.cloudflare.com/ in a webkit-based browser (like Safari). If you disable the https proxy in System Preferences, it won't load. If you enable the proxy, it works fine. (Cloudflare, of course, is a key backbone of the modern web.)

I'm expecting this to happen on more and more websites over time, as they drop support for older cipher suites. A lot of tech literature is actually recommending that websites drop old cipher suites, and by extension, some validators will dock a website's security score if they're present. Because screw older systems, I guess.

 

[maverick28]

Yeah, I tried with a couple of sites, it loaded successfully, however apple, apparently, have some tough security walls around their network and access to such subdomains of their domain network as "developer.apple.com" results in the familiar "Safari couldn't establish secure connection" (Safari 9.1.3). I remember signing into developer.apple.com with Safari 5, Firefox 10 and Chrome 49 (just for fun) as recently as in the late 2018 which turned out to be the deadline for the dropping everything that let older browsers get the access for 7 years before the change.
I also wasn't able to revive my iTunes Store experience with iTunes 11.4. iTunes 12 connects OK (and I can even use my iOS 12 device with it on Mavericks) but I don't like it. Some modifications are buried deeply in the code.

 

[Wowfunhappy]

I'm a little confused by your note about developer.apple.com, that loads for me with or without a proxy. All of Apple does, actually. But, remember, I explicitly exempted all apple.com traffic from the proxy. You could remove '.apple.com' from the config file, although it will break iMessage, and possibly other apps to.

(I happen to know from an experiment about a year ago that iTunes 7 does not use certificate pinning—which means you could use it with a proxy like this—and iTunes 12 does. So certificate pinning was introduced at some point between those two versions. This means you can do fun things in iTunes 7 like edit your request to the server to download old versions of apps—but I don't think it's going to help you connect.)

 

[goosnarrggh]

deleted

 

[maverick28]

deleted

I still could read your post in a Mail notification message. iTunes 10.6.3 has no issues logging in, but iTunes 11 has. However, Genius doesn't work in both. Genius and logging have been problematic since the fall of 2019 coincided with the public release of Catalina that was to include the new Music app. Apple Phone Support was of no help. Later, around January, the issues temporarily went only to return again. I think they did that deliberately. I'd not be surprised if they had their own plants monitoring some discussions here, because exactly a day after I posted to one of the threads about my observations namely that the login&Genius in iTunes 11 had itself miraculously restored, they resumed in old course. Fixing HTTPS compatibility with the method implemented by the OP restores the connectivity only partially, but it has no a profound effect on the underlining WebKit engine used by corresponding stock apps such as iTunes and MAS (and the latter, surprisingly still connects) that requires updates at a more fundamental level which is nearly impossible by anybody but Apple themselves.