macOS Where do I start (want to build stock-market app)?

It sounds like your going to build a spider or web crawler to grab stock info off of certain web sites.

Not difficult to do, I've built them in Java. You could use perl or any scripting language that has an API to access to the web.

One word of caution however, read the web sites terms that you plan on getting the data from. Most sites don't like web crawlers pulling their data (especially financial data). They will explicitly state it somewhere or another. I thought there might be a certain html tag (I'm not a web developer) that web crawlers should respect as well. Hopefully some web folks will jump in here and offer some advice from that perspective.

Once you have that working, its just a matter of getting it into a data store (database or custom file organization) and then writing your stock picking algorithms.

What language do you want to work in?

Xcode lets you develop in a bunch of different languages, but in a way it's sort of tailored for Objective-C/Cocoa development. If you want to learn to develop Mac software that's probably the way to go anyway.

Awhile back, I made a Java program that simply queried yahoo.com everyday the market was open and dumped the stock prices into a MySQL database.

http://quote.yahoo.com/d/quotes.csv?s=AAPL&f=l1&e=.csv //fetches price only
http://quote.yahoo.com/d/quotes.csv?s=AAPL&f=sl1d1t1c1ohgv&e=.csv //A bunch more information

's' is the stock symbol and 'f' defines the format of the .csv file.

hypomojo said:

Next, what's the best way to store that data in a "native" Mac manner? MySQL? Is data storage/manipulation native to Carbon/Cocoa or what else?

Click to expand...

As long as you only need Tiger support CoreData will do it, it stores as SQL, XML or Binary (i'd recommend XML at least while your changing stuff in your application) with no extra work.

EDIT: To explain myself better, for Mac only stuff you probably want to use Cocoa, Core Data is a subset which you should learn after learning Cocoa, with a book like Cocoa: Programming for OS X by Aaron Hillegass, and there is an easy 30 minute tutorial of core data, once you've mastered the basics on Apple.com,

hypomojo said:

Next, what's the best way to store that data in a "native" Mac manner? MySQL? Is data storage/manipulation native to Carbon/Cocoa or what else?

Click to expand...

The best way should be determined by your data and from what you have suggested, it should definitely go into a database. You will probably want to select groups and sections of data, and run you different predictions / analyses at different times and compare results between times / date ranges / stocks etc. One of the most important features will likely be making your data manageable and accessible - do not underestimate this and use a tool designed for the job. (This is statement is bias by my own experience - it just sound like you are working similar sort of stuff as I do).

If I am way off base here, I have obviously mis-understood your descriptions, so skip to someone else's post ;-)

Then, what's the best way to update that data and run the formulas to calculate say a 20, 50 and 200 day moving average for each stock symbol? Shell scripting to run a file diff and Perl to lop of what isn't needed is what I know would do it but it isn't very elegant.

Click to expand...

Download the data everyday through a cron job and load it into the database. wget, as you mentioned, and SQL will meet your needs and you can just trim data you don't need inside the database - firstly loading everything into a "load" table and then transferring relevant data into a "live" table.

In addition, this does mean that your app is going to need a good database connectivity layer. If you choose one database management system, this is less of a problem (MySQL will probably be fine). If you want to connect to multiple data sources, IMHO, you are really looking at Java. Nothing else has the flexibility to connect to different DB platforms on the Mac.

What are the formulae? You can do a lot of calculations through SQL and putting these in stored procedures may well be the best option. You may want to check out the database functionality in different platforms. Call the procedure and read the results from a table - you could even do the same thing using views.

All computing languages will do basic mathematics but if you want anything more sophisticated (in terms of calculations), you want something with easy access to decent maths / scientific libraries. If you don't want to go the SQL route, many of these libraries will automatically do basic calculations like moving averages, regression and histograms for you.

If you are anything like me, you are going to need to need some sort of visualisation system - drawing graphs and such like. Easily rendering a series of data visually will be a high priority and again you want to get this through a library. A picture paints a thousands words, as they say, and as a rule of thumb, you want something that will do this in only a handful of lines of code.

With respect to these requirements, I would suggest trying

Java - using JDBC and any database, a suitable package from http://math.nist.gov/javanumerics/ and, I suggest, JFreeChart

Python - using MySQL or Postgre, NumpPy / SciPy and matplot (look at http://pythonmac.org/packages/)

at least to PROTOTYPE your application's functionality. I don't know if Cocoa offers suitable libraries for what you want to do - there is third party MySQL library for it, and while it offers some good drawing functions, I don't know how far these go down the route of producing decent graphs for minimal effort (comparable the JFreeChart, for example - in fact I would be interested to know just how much is available if anyone cares to fill in the blanks). There is no end to the maths libraries accessible through a C interface but stuff like the CLAPACK interface offered in Apple's vecLib, while very fast to run, is slow to develop with unless you have substantial experience.

I have kind of a bold and random question, how do you make a website that would crawl other websites? Essentially I am asking how to make google, but clearly it does not have to be as sophisticated code as the google boys put together, but what language is their engine made in and is it even plausible that I could begin making a "cr*ppy" version to search for a set of information? Also from what it sounds like, there is a clear difference between making an application to run on m mac and writing code so as to make a search engine. Does Xcode only work for making an application? Or would it work be applicable to making what I am asking about? Also, what if I wanted to make an application to do something like this, then Xcode? Sorry for what I am sure are really stupid questions, but I am trying to learn and just wanted to know what were the right steps to take?