macOS Divert sockets

Hi all,

I went through some of the early discussions on the topic of divert sockets. I also saw few sample code such as http://www.loudhush.ro/files/divert.m

My intention is to do something similar to what is described above, set a rule such that I get only TCP packets, that too only HTTP. Is it possible? The rule suggested is

00001 divert 8999 tcp from any 80 to any out

I assume the rule says divert any packet coming from my system to any website to be diverted to port number 8999 where my application will be listening

This rule should ensure that I receive only TCP packets. My main interest is HTTP payload. I want to log all the HTTP headers going out from my browser to internet, modify it if needed and reinject it back. Since I am dealing with only outbound traffic from my laptop to the internet, using "out" in the rule is appropriate. Please correct me if there is some misunderstanding here.

The question I have is what is the best way to handle the traffic going out? I mean, assume I restart a browser with multiple tabs. Once the browser comes up, at least 20 to 30 odd connections (HTTP) to the internet are going to take place. Obviously each of these sessions will have a unique source port number, so in my code, the recvfrom() will get all these 20 to 30 connections while running in a loop.

Is it smart to spawn off a thread for each individual connection to check for a HTTP header and modify and reinject to the same port and exit? Or a better design will be to maintain some hash table for all the connections and use the same thread to process the modify and write? I don't see how select() can help here.

Another aspect of this design is if I am not filtering on port 80, I will get all TCP packets (including SYN, SYN-ACK etc) for which I need to quickly reinject back without modification. I am only interested in TCP with HTTP payload. Can you suggest me the best approach?

Regards,
Varun

ElectricSheep said:

Since all you care about is logging, you may be more interested in using a tee rule instead of a divert. The tee will send a copy to a specified divert(4) socket, and you can perform whatever logic you wish without having to re-inject the packet back into the networking stack.

This is for a test run, In future I plan to modify it a bit more and be able to change the HTTP payload before re-injecting it back. I need divert socket for that.

Note that this will only work if you have a fairly simple ruleset, because once a packet matches the tee rule, it is accepted and any further rules are not applied.

Click to expand...

My concern is as follows:
- Is it guaranteed that HTTP's GET/POST/HEAD methods will always be one single packet so I don't have to bother about fragment handling when I'am modifying HTTP payload?

- Is it a scalable solution to have one thread reading these HTTP packets from the socket and storing them in a hash table, where the hash key is based in src port, destination address? Another thread would just read from hash table modify the HTTP header and reinject it back. What do you think?

Regards,
Varun