Sounds like you want to look into pseudoterminals (pty's).
This is a POSIX thing, more than a Mac-specific thing, and the definitive reference would be "Advanced Programming in the UNIX Environment", Addison-Wesley, 1992, ISBN 0-201-56317-7, by (the late) W. Richard Stevens.