Using Apache as a WebProxy for https - your opinion/thoughts needed

[vddrnnr]

Hi all,

After the post about the "Web Rendering Proxy" I got to think if there would be
an alternative to using this idea with Apache HTTP server.
I'm not an expert on Apache's configuration but I was able to configure it to receive https requests
that do not have the same requirements that regular websites have for security reasons and then
have apache with it's own OpenSSL libraries doing the communication with the real website.
I've been successful communicating with every site ( google, macrumors, youtube, duckduckgo, etc ) using the last version of leopard webkit as the browser without any issues.

As far as I can see leopard webkit is the "fastest" browser available and as far as rendering html5 pages
it seems to work perfectcly. With this it could be a "major" game changer in webbrowsing "regular" sites.

What do you think about this option?

Best regards,
voidRunner

 

[sparty411]

Hi all,

After the post about the "Web Rendering Proxy" I got to think if there would be
an alternative to using this idea with Apache HTTP server.
I'm not an expert on Apache's configuration but I was able to configure it to receive https requests
that do not have the same requirements that regular websites have for security reasons and then
have apache with it's own OpenSSL libraries doing the communication with the real website.
I've been successful communicating with every site ( google, macrumors, youtube, duckduckgo, etc ) using the last version of leopard webkit as the browser without any issues.

As far as I can see leopard webkit is the "fastest" browser available and as far as rendering html5 pages
it seems to work perfectcly. With this it could be a "major" game changer in webbrowsing "regular" sites.

What do you think about this option?

Best regards,
voidRunner

Perhaps you'll find this interesting. Ive posted it here before, but the post didn't get much attention.

https://github.com/atauenis/webone

 

[vddrnnr]

Hi @Sparty,

It is interesting thank you

The only thing my option may have some additional benefits is that you can
do it using for instance windows 2000 inside VP6/7 on your own machine instead of
another machine on the network and without any more "complicated" dependencies.

My idea would also be to only setup the most used websites not all.

Best regards,
voidRunner

 

[vddrnnr]

Hi guys,

So to give you an update on my endeavours.
Right now I've got Apache 2.4.2 with openssl running which allows me to connect
using TLSv1.2 to every site I want to setup on it.
I've been browsing with Firefox 60 Useragent with everything on with great performance
using the last version of leopard webkit.

Best regards,
voidRunner

 

[Ruonis]

Hi vddrnnr,
Is this method suitable for Safari (or others old browsers) on Snow Leopard?
I have problem to browse sites with TLS 1.2. I tested about 30 browsers which work on 10.6.8 (macbook 2010). 7 browsers can show wikipedia, youtube, facebook etc, but early Safari worked better during sessions with many windows and tabs (about 400-600). Today in that session about 40 % sites do not browse. I try to migrate to Palemoon, but it's good for small sessions. And I try to use Leopard webkit for Snow Leopard. Youtube works normally, but wikipedia shows ugly page about security and necessity of update.

Sorry, but I didn't find forum thread about SL, so I write here.

 

[AphoticD]

@vddrnnr I’m wondering if it would be possible to package up Apache pre-configured as a proxy server with most popular domains already setup, to be installed on any Tiger/Leopard Mac to run locally.

Similar to how MAMP is packaged to run standalone from the Applications folder - Come to think of it, MAMP could be used as the basis for this project as it has all dependencies taken care of (no need for Tigerbrew, Macports or Xcode) and can be just dropped into a system to install. What do you think?

 

[vddrnnr]

Hi Guys,

@Ruonis
Yes it would work as long as the version of Safari Used Is capable of "rendering" the sites.
Apache will take care of the https part.

@AphoticD
Yes that could also be an option.
I just used a VM with win2000 inside because it is easier to setup and test.

I've been fiddling with QEMU the last few days to try and see if it is more
suited for this than VPC 6/7. I think I'm finally getting somewhere.
I'll let you know.

Best regards,
voidRunner

 

[AphoticD]

I've been fiddling with QEMU the last few days to try and see if it is more
suited for this than VPC 6/7. I think I'm finally getting somewhere.
I'll let you know.

Can you share your config? I’d be interested in having a tinker.

 

[vddrnnr]

Hi AphoticD,

What do you need?
VM setup or http?

Best regards,
voidRunner

 

[AphoticD]

Hi AphoticD,

What do you need?
VM setup or http?

Best regards,
voidRunner

A little primer would be greatly appreciated. Even just some pointers for further study, depending on your availability.

I can see this solution would genuinely bridge the gap for the older OS X cats and System 7 through OS9.

It could theoretically also put CorePlayer back in the hot-seat for YouTube playback on G4/G5 systems.

 

[vddrnnr]

Hi AphoticD,

This may be a bit lengthy ;-)

As I mentioned on previous posts my objective is to overcome the TLSv1.2/1.3 problem that all older
OSX browsers have because of "outdated" security frameworks on OSX.
I had already thought of using some solution like this before but when I read the thread about the "Web
Rendering Proxy" I finally got the elements I needed.

- The Problem

For me OSX Leopard or Tiger will allways be the best OSs to run on MACS with PowerPC chips as they
were optimized for them. Linux does a great job but all the new developments are done with the new
architectures in mind and will not be optimized for PPC or lower specced machines.
So in OSX "current" browsers for PPC like TenFourFox or Arcticfox although doing their best they were not
ment to run on PPC hardware and are not optimized for it so in the end on the lower specced machines like
powerbooks and the older G3 macs will always be "slow"/"not fluid" and the faster ones like the G5s will
have to work harder to give you "good performance" ( which means more power consumption and heat
which is very bad especially for our "old ladies" ).
Older browsers based on webkit or the Opera Browser for example will always be faster no matter what you
throw at them and we even have a relativelly uptodate version of Webkit in the form of Leopard Webkit. So
as long as you have a browser thats is capable of rendering html5 standards properly and have an optimized
javascript engine it will "work".

- The options:

a) The "Web Rendering Proxy" solution

IMO has severall drawbacks/difficulties especially for us PPC mobile users:

1. It requires a second MAC running Intel hardware ( dificult to be mobile )
2. It dependes on the chrome engine for rendering the sites ( may fail to generate the correct output )

b) Using Apache as a "Reverse Proxy"

1. This solution requires for it to be portable to be running on your machine but you can also have it on a
central machine serving several other machines.
2. A solution that is easy to setup and update with newer builds independent of any browser
3. May allow "browser developers" to focus on the performance of rendering and javascript and not on the security of the
cummunication/connection
4. Can be implemented in several ways wherever you can setup an Apache HTTP server with OpenSSL like
a VM or local http server.
5. It's a "balancing act" between the power required for running the HTTP server and the power required
for the browser.
6. User agent optimizations. You can configure the UserAgent passed to the website when proxying. This is also
independent of the client browser so if your browser does not support UserAgents per site configurations this
will always work

- My setup right now

a) Centralized option

1. Windows 2003 VM on WMW*re with Apache 2.4.2 on an Intel Hack ( with this I can watch youtube inside
webkit at 70% or less cpu usage for a 360p video on a Powerbook DLSD 1.67ghz ).

b) Local

1. Windows 2000 Pro VM inside VP6/7 or QEMU with the same Apache version ( same install package ).
This setup is the trickiest because of the limitations to install newer software. In this option I'm getting
a bit higher CPU between 75 and 90% when page loaded fully and watching the same video.

Note: To get Apache 2.4.2 on windows 2000 I updated windows 2000 SP4 with the following packages

Windows2000-KB891861-v2-x86-ENU.EXE ( this is an unofficial update from MS to SP4 )
Windows2000-UURollup-v11-d20141130-x86-ENU.7z ( this is the last version I could find of this project to
update Windows 2000 to allow it to run more recent software )

In QEMU my configuration is

-> QEMU 0.15.1 and 1.0.1 that I got from

Building Qemu 0.15.1 on OS X 10.5.8 | Virtually Fun

You need to install the package for Macports for Leopard if you don't have glib2 already on your system.
For this you will also need XCode installed before.

https://distfiles.macports.org/MacPorts/MacPorts-2.3.4-10.5-Leopard.dmg

Right now the best config I have is using the 1.0.1 with the following command

qemu-system-i386 -L pc-bios -vga cirrus -m 256 -localtime -drive file=/Volumes/SHARED/QEMU/Windows2000VP7.qvm/Harddisk_1_3.raw,media=disk,cache=writethrough,cache=writeback,aio=native -boot c -net user,hostfwd=::8443-:8443 -net nic,model=pcnet -machine pc-0.15 -cpu pentium2

-> In the host

And then do

sudo su -
ssh -L 443:127.0.0.1:8443 <username>@127.0.0.1 ( iptables should be a better option for this )

In the host you also have to add every site you want to /etc/hosts pointing to 127.0.0.1

-> Inside the VM

In Apache my configuration ( httpd.conf ) is basically to enable the required modules for proxying and ssl

I have the following lines to enable them

LoadModule headers_module modules/mod_headers.so ( allows you to set the UserAgent header )
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so

You then have to follow the steps at the following URL to generate a self signed certificate for SSL

Apache+SSL Win32 HOWTO

And at the end of your httpd.conf you need something like this:

LISTEN 8443

NameVirtualHost *:8443

<VirtualHost *:8443>
ServerName www.youtube.com
SSLEngine On
SSLCertificateFile "C:/Program Files/Apache Software Foundation/Apache2.2/conf/server.cert"
SSLCertificateKeyFile "C:/Program Files/Apache Software Foundation/Apache2.2/conf/server.key"
SSLProxyEngine On
ProxyPass "/" "http://youtube:8082/"
ProxyPassReverse "/" "http://youtube:8082/"
</VirtualHost>

LISTEN 8082

NameVirtualHost *:8082

<VirtualHost *:8082>
ServerName youtube
SSLProxyEngine On
SSLProxyProtocol all -SSLV2 -SSLV3 -TLSv1 -TLSv1.1 +TLSv1.2
SSLProxyCheckPeerCN off
#LogLevel debug
ProxyPass "/" "https://www.youtube.com/"
ProxyPassReverse "/" "https://www.youtube.com/"
RequestHeader set User-Agent "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0"
</VirtualHost>

And in your hosts file you need

127.0.0.1 localhost
127.0.0.1 youtube

- Current/Future Tasks

-> Trying to optimize even more VM startup time
-> Try to lower VM CPU usage ( maybe stop additional services )
-> Find a more recent version of QEMU to test
-> Testing a Linux VM and see if it behaves better ( maybe a PPC linux one )
-> Finding a better way to add sites to httpd.conf
-> Different certificate generation / importing into browser to avoid the server certificate mismatch authorization

Best regards,
voidRunner
[automerge]1577580820[/automerge]
Hi AphoticD,

Can you tell me a bit about your CorePlayer idea?

Best regards,
voidRunner

 

[AphoticD]

This may be a bit lengthy ;-)

No, this is fantastic! Thank you for taking the time to write it up. I will make some time to setup and test.

You may get better performance out of a more recent Qemu build. I have attached the portfile for v2.4.0 on this old post: https://forums.macrumors.com/threads/the-panther-thread.2046542/post-24817633 (skipping past the first half of the post).

Re: CorePlayer; https:// is not supported, but if you can serve CorePlayer an http:// URL then it would be capable of streaming from YouTube using one of the methods devised by @Dronecatcher or @Lastic, lowering CPU overhead on playback and potentially playing higher def formats on older hardware.

 

[vddrnnr]

Hi Aphotic,

Yes the idea for CorePlayer is a good one.
But with that one we also need something else.
Coreplayer is Intel 32bit also so if we can
"patch" it in intel to accept the same method of activation as it does on PPC ( I've tried it and it
fails because the code that was pactched was the PPC inside the Universal binary, I have to take a look
at the post where it is explained how it was done and try the same steps when running on intel.
This will also be good for the early intel machines that get hammered by youtube's site while
loading/playing.

I'll try this for PPC as soon as I can get viewtube running inside iCab which as a similar "script" engine to
support greasemonkey like scripts.

Best regards,
voidRunner

 

[vddrnnr]

Hi AphoticD,

So I have ViewTube working inside iCab.
I've tested it with calling mplayer and it is working fine.
I'll try it with CorePlayer as soon as I can get the urls for the videos that youtube calls.
I think that for those to get them through http requests may need a solution a bit different
as the base url changes.
First I'll try and get a url from Dronecatchers script and set it up to test outside of the browser.
I'll let you know how it goes.

On a side note I rebuilt glib2 from Macports ( I was using the packaged one provided in the link
I posted before ) and the VM with QEMU 1.0.1 the host CPU now idles between 15 and 20% usage
before it was higher

Best regards,
voidRunner

 

[vddrnnr]

Hi AphoticD,

So I've tried from the command line and it works .
I was able to stream with CorePlayer through HTTP over the Apache Reverse Proxy.
In this case I'm connecting to port 8082 directly so only one virtualhost is needed.
To work from @Dronecatcher's script you have to change it to modify the url's so that they use
HTTP and connect to port 8082.

Best regards,
voidRunner

 

[AphoticD]

Awesome! This is great news for CorePlayer fans.

I am looking forward to seeing further progress

 

[vddrnnr]

Hi AphotidD,

Have built qemu 2.2.0 based on your port file.
Thanx it's much better.

I also now have Apache "dynamicly" reverse proxying for googlevideo urls.
Working with dnsmasq to get OSX to do the same for hosts that would complete what is necessary to use
@Dronecatcher's script for calling mplayer,CorePlayer,VLC....

Best regards,
voidRunner

 

[vddrnnr]

It's working

Best regards and a Happy new Year ;-)

 

[Ruonis]

Hi Guys,

@Ruonis
Yes it would work as long as the version of Safari Used Is capable of "rendering" the sites.
Apache will take care of the https part.

Best regards,
voidRunner

Thank you very much! It's interesting solution. I'll try to do it

 

[vddrnnr]

Hi all,

Just a teaser ;-)
Youtube native inside ICab browser relinked with leopard webkit 604.
Using Apache HTTP Server inside a Puppy Linux VM in VPC6.

Best regards,
voidRunner