Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Google Appeals Antitrust Ruling, Says Apple Chose Its Search Engine 'Fair and Square'
- Thread starter MacRumors
- Start date
- Sort by reaction score
I often think of the same thing... Wouldn't it be nicer if the mega corporations like Google were dissolved into multiple separate companies? Google Search, Youtube, Gmail, Android... If they all had to compete on their own merits, they'd (perhaps) be more likely to put emphasis on quality and interoperability.
p96
macrumors regular
Thank you for sharing! With all due respect to the authors, unfortunately IMO this isn't a very good paper...
First of all, the queries used for testing which search engine is the best are the following (verbatim quote):
General Type: What is Database Navigation Type: Sql in DatabaseTransection Type: Download Sql database book free Their whole testing is based on these 3 very specific queries which surely aren't representative of general search engine quality.
Their methodology for ranking which search engine does best also seems off - they simply do a "percentage of accurate results" for each of the searches... But they don't really explain how it's determined what result is relevant and what isn't - therefore I'm assuming they just determined it themselves. They don't seem to have a "master list" of what results they were expecting to see, so they can check whether which results are surfaced by which of the engines, which is surely how you'd want to do this.
Many of the tables have weird numbers, bad formatting or values you wouldn't expect:
Why do they have different numbers of total results? Why does DDG have "more and more" results rather than a number? Why does DDG have "Youtube" listed for the video column? Why does ChatGPT have decimals instead of percentages listed?
Why is ChatGPT even considered a search engine? Since the authors show in the table above that there's actually only 1 total result (assuming this is the chat response) which they somehow deduce the percentage of relevancy for. So they're comparing 100 total "classic" search engine results to a single ChatGPT response... In a different table, they also map this "1 response" to "100 responses" which is also not a scientifically good way to do things.
Minor compared to the other issues, but I'm also confused by how they managed to get 10x slower response times from DDG (also didn't indicate whether this is in seconds or ms):
ChatGPT also somehow places second with the fastest responses. Highly doubt the accuracy of these numbers, and the testing methodology is not discussed in the paper.
Ultimately not discounting Google (or ChatGPT for that matter) as a search engine, but this article is not it and I also don't agree that such a thing as "objectively best search results" exists. Sure, if you type in "Steve Jobs" if a search engine doesn't come up with his Wiki page and only has results for other people named Steve then that's a bad search engine. But for queries like "best 2026 comedy movies" you get into subjective territory - who even is the authority here?
point of the paper is testing against general (informational), navigational, and transactional which are three distinct types of human intent. hence the three isolated queries.
the paper is speaking to an audience with a Data Science and Information Retrieval background. if you're asking how a result is relevant, then you would need to go study the field and learn about how text is vectorized and compute similarities.
the omission of the unit means the mathematically obvious unit is assumed.
the "change 11,13" and "more and more" is showing the inconsistencies with the layout at the per page links when using the service.
I'm realizing that most of your questions are basically the result of you skimming the paper. I recommend reading the entire paper thoroughly.
you're free to dismiss the paper but there is objective data in this paper to back up the claim that google search is better than the competitors. while JCBI isn't the most prestigious, it still has papers go through a double-blind peer review process.
Last edited:
The chart you linked shows results from a single study. It tested for specific domain precision finding, not for general information web searching. The domain was “agricultural”, the specific search item was limited to “durian”, a Southeast Asian fruit, for each of eight queries that were. As a result, the study has neither content nor external validity. Moreover, the researchers concluded that “the mean average precision value 0.51 given by Google is still low to be satisfied neither by the researchers nor the information seekers.”
Thanks for the link. I am ready to accept this much of your claim:
“There is no product with a better search function than Google Search when searching for the Southeast Asian fruit durian.”
- - - - - - - - - - - - - - > when you tap View Publication on your link:
Z.
I read the paper. In my academic area, a paper like this would have never made it to publication. But it does present solid evidence for
“There is no product with a better search function than Google Search when searching for the Southeast Asian fruit durian.”
😊
Marshall73
macrumors 68030
With the current administration NOTHING is illegal if you give Trump a ‘taste’.
if only you could put a price on success by merit. oh wait..
pray tell what 'better innovation'? the search is getting worse and worse, and if they work just a little harder, it will soon go the way of google-owned youtube's.
eh?
which is why Gemini is not at all playing catch-up, and is definately not free-riding on Google's data hoard.
the whole appeal reads like some sour grapes slop.
1. it's double blinded peer reviewed paper. if your academic area doesn't do this minimum requirement before publication, it says a lot about your academic area.
2. the paper in that discussion doesn't discuss south asian fruit. you clearly didn't read the paper 😂
3. paper provides solid evidence that search accuracy is quantifiable, something you argued wasn't quantifiable. remember, your quote was: "you can’t quantify “better search than …”. " 😂
1. My area does not just have double blind peer review as a requirement, it also obligates the authors to fix spelling, syntax, semantics and style errors before the paper is cleared for publication. Obviously, the journal you linked to neither requires such basic professional standards from the researchers nor the editorial board — not even in the abstract:
However, the question arises, does search engine perform the same when seeking …
[ … do search engines perform … ]
There were four search engines have been selected namely Google, Bing, Yahoo and DuckDuckGo for the experiment.
[ These four search engines were selected for the experiment: … ]
While for the domain specific, we chose to test information about the popular fruit in Southeast Asia that is durian.
[ Using “agricultural” for the specific domain, we … ]
Nevertheless, the mean average precision value 0.51 given by Google is still low to be satisfied neither by the researcher nor the information seekers.
[ … value of 0.51 obtained for Google is still too low to satisfy researchers and … ]
2. You claim the paper does not discuss Southeast Asian fruit, really? Did you read the article:
3. see next post
Quantifiable in terms of your claim that it is the best search engine, that is what I argued. And it is not as the paper you linked shows. The latter singles out one single specific domain. Aside from the fact that even on a limited scope none of the engines performed above 52% retrieval access accuracy, the findings cannot be extrapolated to precision findings for multiple domains and definitely not for general information search performance.
When you find a study proving the superiority of the Google search function with external validity, let me know. Until then, I am happy that Google works so well for you for every type of search.
Finally, I suggest we stop the back and forth at this point. You are not convincing me, neither am I convincing you. If Google were proven to be 100% effective in all search applications and DuckDuckGo only 80%, I would still go with the latter because it outperforms Google in privacy protection.
You replied to a thread discussing about a second paper that the other person asked for which did not discuss southeast asian fruit. Your rest of the argument goes up in flames, sadly.
Click through the thread history that you replied to and you'll see the second article mentioned here:
Google Appeals Antitrust Ruling, Says Apple Chose Its Search Engine 'Fair and Square'
sure https://jcbi.org/index.php/Main/article/view/967/675
forums.macrumors.com
Next time, if you're going to jump into a separate thread of discussion, I suggest you read through the entire response history so you won't make this same mistake again.
Last edited:
In one specific domain it was quantifiably better. And in three different user intents in another domain it was quantifiably better (in the second paper which you failed to click on).
Your claim that it must be better across other domains would be a subjective requirement for my claim, which you have yet to show anyways, but even if that is true, under that pretense, it proves that your suggestion that it's not quantifiable is literally false.
Yes, I agree I'm done with this back and forth with you. This concludes my argument with you. Have a great day.
Last edited:
True, I joined the discussion in progress. Thanks for the link to the second paper. In addition to showing yet another editorial oversight by this journal, it actually helped to extinguish the flames of my argument, happily.You replied to a thread discussing about a second paper that the other person asked for which did not discuss southeast asian fruit. Your rest of the argument goes up in flames, sadly.
Click through the thread history that you replied to and you'll see the second article mentioned here:
Google Appeals Antitrust Ruling, Says Apple Chose Its Search Engine 'Fair and Square'
sure https://jcbi.org/index.php/Main/article/view/967/675
This study also shows that checking things like accuracy and how many correct results are found is important to understand how is good a search engine. Each search engine uses algorithms that directly affect the relevance of results. This means that users should choose a search engine based on their specific information needs. In the future, research should include more diverse and complex queries such as multi-step research tasks, sensitive topics, or multimedia searches and examine user behavior through click patterns and feedback
This concludes my argument with you. Have a great day.