macOS Virtual "Internal Seral Port" on Mac?

[moutonnoir]

Hi!

I am trying to find information about creating and interacting with a 'virtual serial port' on both PC and Mac. I have found a lot of info on how to do this on the PC side, but nothing in Mac. On PC side there are various activeX controls, etc.

My need is this:

I have a off the shelf program I use that sends and receives commands via RS-232. It is cross platform and I use it on both Mac and PC. I want to be able to intercept these communications in another program running on the same system, modify or manipulate the data, then send it either back to the program, or to my external hardware on a real RS-232 port, or a USB=>serial on Mac.

I do not want to use a kludgy hardware 'solution' such as a null model cable.

Any suggestions? I have seen that I can possibly use the kernel to create a TTL serial 'socket?'.


Thanks!

 

[balamw]

I have a off the shelf program I use that sends and receives commands via RS-232. It is cross platform and I use it on both Mac and PC. I want to be able to intercept these communications in another program running on the same system, modify or manipulate the data, then send it either back to the program, or to my external hardware on a real RS-232 port, or a USB=>serial on Mac.

Let me get this straight you're looking for the kind of utility often called a "serial port monitor," "serial port sniffer", "serial port analyzer" or "serial spy", right? Without the real port and device on the other end of it you won't generally get the full conversation.

If so, check here, http://homepage.mac.com/chen/w7ay/Serial Tools/index.html for a starting point.

B

 

[moutonnoir]

Let me get this straight you're looking for the kind of utility often called a "serial port monitor," "serial port sniffer", "serial port analyzer" or "serial spy", right? Without the real port and device on the other end of it you won't generally get the full conversation.

If so, check here, http://homepage.mac.com/chen/w7ay/Serial Tools/index.html for a starting point.

B

Hey! Thanks for the reply.

Ok. So my final goal is a self contained cross-platform application that when running creates a virtual port that is available to an animation program via RS232. The program sends motion control commands.

I do not want to use a 3rd party solution - I want this to be part of my program. I tried serial tools, actually, to sniff\monitor the data I was sending to my hardware prototypes, but it did not work as a 'passthrough' - it needed physical hardware.

Here is the application in slightly more detail:

1) A Mac\PC program that does animation is running and has a hardware interface that uses RS232. On mac this requires a USB-serial cable.
2) My software sequences lights and other things and uses RS-232 to talk to my hardware prototype.
3)I want the animation program to instead talk to my application - so I can get the data I need from it, make decisions in my software, and then send the data on to the hardware.
4)Conversely, sometimes the hardware interface will tell my software to do something to the animation software.

I want it to be seamless as possible, as I will be using this extensively. Ideally when my application is running, the Mac will see a serial port in it's enumerated port list. My application will see the same list, and the user would select the hardware serial port.

Thanks for any advice!

 

[chown33]

Let me get this straight you're looking for the kind of utility often called a "serial port monitor," "serial port sniffer", "serial port analyzer" or "serial spy", right?

I thought he meant a Unix domain socket:
http://en.wikipedia.org/wiki/Unix_domain_socket

The thing is, a Unix domain socket may not be 100% compatible with an actual serial-port device inode, so things like ioctl() may not work identically.

I don't know of any virtual serial port drivers for Mac OS X. Someone with experience writing actual device drivers could probably whip one up, but it's not something to try without any experience. There may also be some Apple sample code for writing device drivers that could provide a starting point.

I'm also pretty sure the FTDI and Prolific PL2303 USB-to-serial driver source can be found on SourceForge. Those drivers appear as char-device inodes in /dev.

It might help if we knew what the "off the shelf program" was. There may be more than one way to accomplish the goal.

 

[moutonnoir]

I thought he meant a Unix domain socket:
http://en.wikipedia.org/wiki/Unix_domain_socket

The thing is, a Unix domain socket may not be 100% compatible with an actual serial-port device inode, so things like ioctl() may not work identically.

I don't know of any virtual serial port drivers for Mac OS X. Someone with experience writing actual device drivers could probably whip one up, but it's not something to try without any experience. There may also be some Apple sample code for writing device drivers that could provide a starting point.

I'm also pretty sure the FTDI and Prolific PL2303 USB-to-serial driver source can be found on SourceForge. Those drivers appear as char-device inodes in /dev.

It might help if we knew what the "off the shelf program" was. There may be more than one way to accomplish the goal.

The software is DragonStopMotion. http://www.dragonstopmotion.com/ It does not appear to offer an API beyond the RS232.. They have a scripting feature, but it does not pass any variables. The RS232 control interface is much fuller featured (though still pretty sad).

I want to use sequencing software Ive written and use for various things as a way to.. hiJack the frame position command from Dragon and move steppers\control lights. This is all, actually, more or less easy for me. The hangup is the serial port issue on mac. On PC there are many options to do this same task.

The interface is very simple, from an RS-232 point of view. Simple simple.

I was hoping I would not have to write a 'device driver' per se, but more of a wrapper. The thoughts about starting with the FTDI source is a good one, and I had also thought of it.

I am a bit concerned that the complete lack of a commercial version of my serial emulation goal is a bit foreboding. Maybe there is great difficulty in emulating a serial port on a mac?

The serial sniffer portion of serial tools would be perfect example if it had a virtual serial feature as well, or a 'intercept serial communications on X port' mode.

 

[moutonnoir]

Bummer. FTDI source code is not available..

 

[chown33]

The interface is very simple, from an RS-232 point of view. Simple simple.

But it probably still expects things like the control calls to set baud-rate, etc. to work. It's those that I expect would cause difficulties when trying to use a unix domain socket. I suppose you could try it and see what happens. It shouldn't be that difficult to find examples of unix domain sockets on Mac OS X.

I am a bit concerned that the complete lack of a commercial version of my serial emulation goal is a bit foreboding. Maybe there is great difficulty in emulating a serial port on a mac?

I wouldn't expect it to be any more difficult than any other device driver for Mac OS X. Which doesn't imply it's trivial, or is a suitable project for the inexperienced Mac programmer.

On platforms that don't have something like unix domain sockets, or that don't have a loopback network interface (man 4 lo), I can see why "virtual serial ports" would be a useful thing to have. If there were a long history replete with external devices that attached to Mac serial ports, then I could also see the utility. But in recent times, more external devices communicate with TCP/IP, and there isn't a long Mac history filled with external serial-port devices. A lot of the devices that were attached to serial-ports transitioned to ADB and then USB, or they went the TCP/IP route. Anything that needed an actual serial-port needed the actual serial-port, so a USB-to-serial adapter was the migration path.

The serial sniffer portion of serial tools would be perfect example if it had a virtual serial feature as well, or a 'intercept serial communications on X port' mode.

But then it would need to provide a device driver.

Honestly, I see no way around the "it needs a device driver" problem. If a unix domain socket is going to work, then it's because the ioctl or other calls made by the app doesn't care that those things don't work. If a unix domain socket is going to work, then the only way to discover that is to try it. If it's not going to work, then a device driver is the only path I can see.

 

[moutonnoir]

But then it would need to provide a device driver.

Honestly, I see no way around the "it needs a device driver" problem. If a unix domain socket is going to work, then it's because the ioctl or other calls made by the app doesn't care that those things don't work. If a unix domain socket is going to work, then the only way to discover that is to try it. If it's not going to work, then a device driver is the only path I can see.

Fabulous info. And the conclusion i have come to as well. My steps will now be:

1)Attempt to create an IOKit based Kext that reflects input from one port and out the other

2)If this works, figure out how to deploy it within the framework of my, really rather simple, application.

Do you have any hints? Another poster told me: " Be careful, though -- you are playing in kernel space."

I dont even know what that warning means. I am very very new to mac programming. I was\am still hoping to do most of this with Processing java language!

 

[chown33]

I was\am still hoping to do most of this with Processing java language!

I estimate zero chance of that happening. Java doesn't exist at the kernel or device-driver level.

 

[moutonnoir]

Well..

Apparently the warnings have to do with memory errors. Furthermore it looks like I need '2 machines' to do kext development. I wonder if I can ssh in with my PC, or if I have to try and setup a virtual machine running OSX on my PC?\

Either is OK.

This is really sad. This is, by far, the least fun part of the application I am writing. It is almost looking like a stumbling block.

 

[moutonnoir]

I estimate zero chance of that happening. Java doesn't exist at the kernel or device-driver level.

Oh. Yea. For the kernel stuff I will, of course, use the recommended Xcode techniques.

I am also not against writing 2 separate native applications. The tedious code (handling the animation engine\tweening) is already written and is easy to port to any given C\java variant. It uses some decent abstraction to keep the display and logic separate - it is also much more 'up my alley' in terms of the type of code I can just whip up. Worst case scenario would be to simpy develop the entire application in my hardware prototype. This is the easiest option for me, but it would add a very expensive LCD screen to the hardware. I want the desktop application to tell the hardware what it is going to be doing, so there is a rich user experience on the PC, to setup keyframes and tween settings.

I am not a trained programmer - and I have little OSX\unix experience. I am from industrial design background.

Have you written any kexts? know of a good starting point or guide? I am reading the apple documentation now.

 

[balamw]

So if I understand correctly your goal is not just a virtual serial port, but a virtual clone of your hardware prototype. i.e. it would not only need to act just like a serial port, but also be able to provide the responses that your hardware would. (Serial devices that don't ack in some way are bad juju, so I hope your device sends some chatter back.).

I just ran across this the other day: http://www.hhhh.org/wiml/proj/keyspan.html I'm a big fan of the Keyspan USB/serial converters and it turns out that they are inherently hackable. You could actually embed a simulation of your hardware in the USB/serial dongle and would not have to mess around with kexts at all.

Just a thought.

My only device driver experience was for an embedded Linux project and it was no fun, even though by the time we got done it worked well. Too bad the thing it was supposed to drive didn't work as well as its controller.

B

 

[moutonnoir]

Yes. but the command interface is super simple - It is extremely easy for me to write the parser\forwarder. It is only 6 commands, and the actions the desktop software need to worry about are very simple. The handshake is pretty simple - the off the shelf animation software does, unfortunately, minimal interactions - but that is fine if the interconnect is reliable - my hardware device could then have it's own handshake..

I also considered the serial dongle device you thought of! I dont like it.. I would prefer the less attractive but more functional\easier approach of just having 2 hardware ftdi serial ports on my device. That would be pretty hackish though...

 

[balamw]

Does this help?
http://www.daemonforums.org/showthread.php?t=634

socat and netfwd.

B

 

[moutonnoir]

Does this help?
http://www.daemonforums.org/showthread.php?t=634

socat and netfwd.

B

Brilliant - good find! Now I really have some reading to do.. Argh. Such is life.

It looks like socat is what I need, and netfwd may provide a sample of a 'switcher' to route the data as I need to.

I just wish there were a library 'fakeSerialPort.H' or something!

This approach seems far from a available 'recipe'..

 

[moutonnoir]

I see that socat, which looks like a potential winner, is a command-line terminal application.

Is the process for me to get this functionality to be more 'automatic' going to be to look at the socat code and attempt to rewrite it in xcode objective c?

is there a 'good' way to use a objective C or other Mac desktop app to run these command-line type commands? Shell scripting or something?

 

[balamw]

I see that socat, which looks like a potential winner, is a command-line terminal application.

Is the process for me to get this functionality to be more 'automatic' going to be to look at the socat code and attempt to rewrite it in xcode objective c?

is there a 'good' way to use a objective C or other Mac desktop app to run these command-line type commands? Shell scripting or something?

You can call command line utilities from within Objective-C 2.0, look up NSTask. In fact this is recommended if your task requires elevated privileges. (I would imagine that creating an item in /dev will require privileges).

Since Objective-C is just a strict superset of C you can generally just replace the UI of the program, but need to be sure that the license is compatible with the code you integrate it with. i.e. don't use GPL code in this way unless your app will be GPLed.

B