1. It's for online search
When you say crawl ("data crawled from the Internet", from Apple, which does not make sense at iOS 8/OS X Yosemite since they forward the search to Bing), it means Internet search. That's the definition of crawler (the software that crawl): indexer that continuously follow every links to recursively index all the linked sites.
2. Why are you surprised? Apple has already collect thousands of Open Search API (check your Safari Preference). Just by building a cache system in between can provide a sufficient amount of quality of search.
e.g. When you search Raymond Reddington, the server will crawl all topic/site-based search engine. Let's say the wikia for blacklist return a result, it will have a higher quality that what you got in Google because it's an entry of specific topic, not just a website mention the name.
And when spotlight to display the page, Apple can caching the result, record whether you click on it, and build a webpage index that people actually read. Google is building their own by PageRank. But the problem is: it can only how much the page is trustworthy, but not "whether the user will click on it", which is the only goal in search engine.
In fact, by recording the search query, Apple can actually build a list of words that you will actually say, which in term could potentially allowing them to build their own voice recognition system, or at least an accurate dictionary of words you will actually say.
And even if you don't believe all those stuff above (you really need to learn how it works to appreciate the value of those data), you seriously believe Apple adopt an Amazon 2005 protocol in 2014 just for the "goodness in their hearts"?
3. In fact, even if Apple doesn't want to do it to harm Google, they have should a workable prototype as the next generation of Spotlight.
I mean, the only way to get context based search result (e.g. based on location, time, calendar, files, topic you just search) is by building your own search engine, since the amount of data set is so big there's worth tenth of dollars per customers. Who have access to all these beside Google, MS and Apple?
The goal is so sweet, Apple should have worked on it to demo at next event.