New at C++, ideas for Jarvis?

Discussion in 'Mac Programming' started by Jakintosh™, May 17, 2010.

  1. Jakintosh™ macrumors member

    Jun 21, 2007
    So, after reading an article about a guy who made his own "Jarvis" (Tony Stark's personal AI Butler from the movie Iron Man) I thought that would be a neat project to test out my newly acquired programming skills. I just completed two college courses, Intro to C++ and Intro to OOP (using C++). We got through functions, classes, pointers, and lots of other basic things and I'm very confident in my skills, and am eager to learn more.

    Anyway, I sat down opened Xcode and created a new cmd line C++ file, set up main and created a new class, but didn't know what to make it.

    So I ask you, what should be (some basic) functions to start off making my personal butler? What do you think it should know how to do? I think I should stick to things local to my machine for now...

    Any and all ideas are welcome, and thanks for your help :)
  2. miles01110 macrumors Core


    Jul 24, 2006
    The Ivory Tower (I'm not coming down)
    In my opinion you should work on a project you actually care about (or that you're getting paid for ;-) ) instead of trying to emulate what someone else is doing. If you don't care about a project, when things start to get difficult you're much more likely to give up on it rather than deriving nerd pleasure from figuring out a clever solution.
  3. sammich macrumors 601


    Sep 26, 2006
    A paul_bettany() function.

    But seriously, if you wanted to pursue this as something to further you knowledge in C++, you would need to come up with ideas of your own. It's your own butler, what is the use of it if it only does things that other people have suggested.
  4. Jakintosh™ thread starter macrumors member

    Jun 21, 2007
    I actually DO care about this project, and I'm really excited about it, but I was just wondering if anyone had any ideas on a good place to start based on my limited skill set.
  5. sammich macrumors 601


    Sep 26, 2006
    So, this JarvisClone you wish to make, it's supposed to take plain english typed into the console and it will reply/do things for you?

    I'm afraid you're limited to an absolutely gigantic case structure, until you study things like AI. It will essentially become a shell of sorts.
  6. notjustjay macrumors 603


    Sep 19, 2003
    Canada, eh?
    Well, you could start by reading up on intelligent software agents. And, yes, do read up on the current state of AI techniques, including data mining. But mostly on agents and web services.

    Saying "I want to write a Jarvis clone" is a bit like saying "Mac OS X is a pretty cool operating system, I want to write my own just like it". Sure, but first you need to understand the fundamentals of operating system kernels, file structures, memory management, database storage, etc. It's not just writing C/C++ code, it's a whole raft of knowledge that you need to have first before you can start writing anything. Or imagine saying the F-22 is a pretty cool fighter aircraft, I want to design one too. Sure, but not without lessons in aerodynamics, metallurgy, thermodynamics, advanced physics and chemistry, understanding the current state of research, etc.

    It's a pretty huge undertaking, but every journey begins with a first step, right?

    You might look at existing software such as the Siri Virtual Personal Assistant for inspiration.

    Forget the english-to-commands interpretation for now, unless you want to dive right into natural language processing and lexical parsers (though this could be a starting point). Hey, ELIZA was written decades ago -- that's a start, right? :)

    Start small. Pick a simple task and write a program to go do it. For example, "Hey Jarvis, what's the weather like in San Antonio?" Write a program that asks you to type in a ZIP code, then goes out to the internet, gets the weather data, and prints it on screen. Doing this will give you some experience with client-server architecture, the TCP/IP stack, data parsing (probably XML), web services, understanding how to work with remote APIs, and other underlying technologies and techniques that will be essential to help you fulfill your ultimate goal.

    Once you're comfortable with that sort of thing, you can branch out beyond simply the weather. "Hey Jarvis, tell me about that Iron Man movie." Maybe your program could go out to or Wikipedia, enter some search terms on your behalf, pull in the data off the website, and display it. For bonus points, feed it through Mac OS X's text-to-speech generator. Suddenly your butler is a whole lot smarter...
  7. Jakintosh™ thread starter macrumors member

    Jun 21, 2007

    Thank you, that is more like the answer I was looking for :)

    looks like I have my work cut out for me :D
  8. ptaylor9 macrumors member

    Sep 16, 2009
    nice post notjustjay, good to see some useful ideas :eek:)
    although it effectively emulates what osx can already do, you said you wanted to do things locally too, so how about being able to ask it stuff about your local files?

    "where did I save file xxxx?"
    "how much space is left on drive xxxx?"
    "create a new document for me called xxxx"

    i know its really simple stuff, but it might give you some ideas about how to parse your input and work out what the user wants..

    I like the idea of a butler though. But the guys above are also right about being dedicated to it. I'm a windows application programmer by trade, but i'm currently learning objective-c and cocoa for nothing other than wanting to do it myself, so one of my upcming chalenges is finding a big project to stick with too. Its very different when you are not being paid for it!
  9. notjustjay macrumors 603


    Sep 19, 2003
    Canada, eh?
    Ultimately the goal is to have fun and learn something new. The warning I wrote a few posts ago was simply to make sure everyone is aware of the scale of a project like this. And it's something I actually have a little bit of knowledge about, because my master's thesis dealt with (to a small degree) data mining and machine learning techniques, and (believe it or not!) I wrote an ELIZA style chat program back when I was in grade 8.

    The way I see it, there two major problems to solve:

    1. Having a repertoire of things that Jarvis would be able to do
    2. Taking plain text input and interpreting what the user wants.

    The first is limited only by what services you feel like programming or what service API's are available on the web (e.g. weather, movie schedules, restaurant reservations). If you really wanted to get fancy, you could define a generic schema that would let you (or someone else, or even Jarvis itself) add new capabilities on the fly. In the movies, the AI's have almost unlimited capabilities and keep learning as they go. In real life, you'd have to program each specific capability.

    Sticking with my weather example, you know that you need two pieces of data to complete a query about weather: you need a place, and a time (what's the weather like in Hong Kong tomorrow?) Either value could be explicitly specified, or implied ("what's the weather like tonight?" implies the place is "here", and "what's the weather like in Florida?" implies the time is "now".)

    So the second part is extracting the data from a user input. You could leverage research that's already been done in natural language processing techniques, but just for fun let's start from scratch. One thing we could do is start stripping away "filler" keywords like "the", "in", "tell me", "what is", "please", and punctuation marks, since they're not strictly required and really add no useful data to the query. So the following queries:

    Jarvis, please tell me what the weather is like in Toronto this evening.
    What is the weather in Toronto tonight?
    The weather in Toronto this evening is...?
    toronto weather 9:00pm

    ... would all boil down to "weather", "Toronto", "this evening/tonight/9:00pm".

    The keyword "weather" tells you what data to process, then you filter the rest to determine the time and place. You might have a list of phrases that translate to specific times, for example "this evening", "tomorrow", "next Tuesday", etc. You could also add more keyword translations, for example "how hot is it" or "what is the temperature" are other ways of asking about "weather". You'd also want to translate, say, city names, country names, airport codes, ZIP codes, etc.

    By the way, as long as you're doing your learning and development on OS X, look into Apple's speech recognition services and AppleScript. There's a lot of functionality in there that you could apply right away. For example, OS X already knows how to answer spoken questions like "what time is it?" or "tell me a joke".
  10. mrbash macrumors 6502

    Aug 10, 2008
    I'm not sure a project heavily dependent on NLP would be the easiest to start with. There is a lot of theory you would probably need to know first.

    While there may be some off the shelf components that will provide you with a parse tree of a sentence, you would still need to know what to do with it.

    Perhaps you should first start by rigorously defining what you want to do. At present it is much too broad to be realistic. Once you have that definition, we can help you de-construct what you would need to do. And from that you can choose which things you are able to do.
  11. BadWolf13 macrumors 6502

    Dec 17, 2009
    I don't want to be a downer, but this is a HUGE undertaking you're talking about. In terms of making something like Jarvis, yes, that would involve AI, speech recognition, and syntax comprehension. These are three of the biggest and most complicated aspects of programming. To put it in perspective, a U of T researcher recently got a $200,000 grant to work on syntax comprehension.

    The thing is, depending on the complexity you want out of this program, it could either be something average, or the most complex and advanced program on the planet.
  12. Bill McEnaney macrumors 6502

    Apr 29, 2010
    Prolog is excellent for NLP because its inventors designed it for it. But as my friend Dr. Michael Covington, the director of the AI Lab at the University of Georgia, told me, before you do NLP, you need to learn his specialty, linguistics. I agree with him because I tried to do NLP when I didn't know enough about linguistics. I still don't know enough about it.

    Dr. Covington is a Prolog expert who also teaches his students to write NLP programs in Python.
  13. qtx43 macrumors 6502a

    Aug 4, 2007
    Everyone is talking about AI, my first reaction was you'll have to write device drivers, or get some premade interface for robotics programming, and then build the mechanical hand and arm and ...etc. Everybody sees something different. A huge project doesn't even begin to describe it. But that doesn't mean it can't be fun, but I'd pick just one small aspect that could be useful in and of itself, and focus on that. Or join some other project.
  14. notjustjay macrumors 603


    Sep 19, 2003
    Canada, eh?
    It's only AI once you get to the level shown in the movie where Tony Stark and Jarvis were having intelligent conversations with each other. There are plenty of other sci-fi examples of this including the LCARS computers in the Star Trek universe.

    Otherwise, all you really need to do is solve a small subset of the problem. Have you ever tried to understand someone speaking a foreign language that you were only partially familiar with? You can get by -- usually -- by picking up one or two important keywords and deriving the rest from the context.

    Again, using my weather example above, all you need to do is recognize the word "weather" -- or maybe some related keywords like "temperature" -- and be able to do enough NLP to pick out a place and a time, and you've got enough information to answer the question.

    A fairly simple processing engine could attempt to parse the input and decode basic concepts like times, places, names, phone numbers, dates, etc. You could make it context-aware by storing the data for a while so that future conversation could refer to previously-discussed concepts.

    Then you feed it some processing rules, like:

    Keyword "weather"
    Required data: location (default "here"), time (default "now")

    Keyword "movie", "ticket", "purchase"
    Required data: title, location (default "here"), time (default "now")

    Keyword "restaurant", "reservation"
    Required data: name, location, time (default "now"), number of people

    You just need enough to get by, depending on what exactly your program should do for you.
  15. Phazotron Guest

    Feb 10, 2005
    You talking about this guy?

    He claims he's going to make the code for his project open-source, so you might just be able to pick up where he left off.

    I'm considering doing something along these lines, but some other posters are correct - designing a sophisticated AI like Jarvis could/would take years of work.

    I have to repeat myself a few times just to get the built-in voice recognition in OS X to recognize my commands, so we've got a long way to go to get a Jarvis going.

    In the meantime, I'll probably start off with trying to get some voice-activated applescripts going.... must be a way to do that.
  16. CFreymarc Suspended

    Sep 4, 2009
    You need to become a professional and quit with the fan boy stuff. You need to write something that you can sell and make money that others will appreciate. Create your own intellectual property that someone will want to buy from you.

    This is not just fun and games anymore, your work and effort should make others product that goes into the community instead of something for your own amusement. That is way more satisfying than just making some fandom novelty.

    In other words, as I heard it said best, "Time to take off the condom, knock the girl up and be a real man."
  17. BadWolf13 macrumors 6502

    Dec 17, 2009
    This is the best place to start. Once you turn on speakable items, your mac is always monitoring incoming sounds for the key phrases. However, my experience is the same as Phazotron's. I have to say everything a few times before the mac will understand it. However, given that Applescripts are voice commandable(if that's even a word) if you put them in the speakable items folder, there's your start.
  18. cartman76 macrumors newbie

    Jun 25, 2010

Share This Page