macOS 27 Spotlight: a good AI RAG alternative?

[Populus]

This post is aimed at those who already have access to Siri AI on their macOS 27 beta and have completed the indexation. Ideally, if you have an M3 or better Mac with at least 12GB of RAM, better.

I need to know if the new semantic search of Spotlight powered by Siri AI is powerful enough to replace a local RAG system. I need to know if, for example, I have tons of texts, power point presentations or long PDFs, if the system has them semantically indexed and if I ask in natural language “look for a document where this process is explained” even if it is in the page 12 of a 30 pages document, or even harder, “look for the documents on this folder (or the entire disk) that can help me understand this topic / solve this problem”.

If any of you can perform similarly demanding tests please post your outcomes here.

Thank you.

 

[Populus]

Hey, @mactracker , I’ve seen that you have lately explored many of the new Golden Gate features quite deeply.

I wanted to ask you wether PDF, or other text analysis by Siri AI has been subject of your exploration. If so, could you report if there’s still a limit in the size of the document being “ingested” by Siri AI, or if it still reads only the first few pages of the document? Otherwise I will have to install the beta myself and test it…

Thank you!

 

[pdfu]

Siri AI will extract up to 1,048,576 characters from a document. The Private Cloud Compute input limit is 262,144 tokens, so the server will receive a trimmed version of the extracted text. In my tests, PCC receives roughly the first 200,000 characters.

 

[Populus]

Siri AI will extract up to 1,048,576 characters from a document. The Private Cloud Compute input limit is 262,144 tokens, so the server will receive a trimmed version of the extracted text. In my tests, PCC receives roughly the first 200,000 characters.

Thank you! This information is immensely valuable.

So... wait, you say it is capable of extracting a million characters from a document but only capable of processing the first 200,000... Then, there must be a way to circumvent this:

a) PCC has this limit, but what about local? Doesn't Siri AI have the option to process this things locally (at least on a last gen Mac with an M5 chip and 32GB of RAM? Or it is mandatory to send it to PCC?

b) If those 200,000 characters are a hard limit, then I guess we'll have to divide our longer documents in "chunks" of 200,000 characters... and analyze them separately.

Really hoping Apple keeps developing this local approach to AI.

 

[pdfu]

There’s a tiny local planner (Skimmer) that handles a super limited number of tasks on your device, including opening apps and telling you the time. They recently gave it the tools to do math, but it’s not enabled yet.

Everything else goes to PCC. File attachments automatically bypass the local model, so it doesn't even get a chance to reason over files:

Skimmer deferring … to the model planner: request has explicit attachments.

More powerful devices enable expressive voices and advanced dictation, nothing more.

 

[Populus]

Thank you Mactracker.

Siri AI is ******** than I thought then…