About to re-read Foundation before watching Foundation.
This got me to thinking about "PsychoHistory", the basic premise of Foundation that allowed history to be predicted.
Asimov initially thought that
Psychohistory was possible, then later in life he changed his mind.
I think that he was originally onto a good thing.
First, people everywhere are pretty much the same. I found this out when visiting a PNG village in the 1970s and noticing that village people gossiped about the same things that people in "advanced" western societies did -- who's doing what to whom, and where and when.
Second, genetic research has shown that people are about 50% Conservative and 50% Progressive. This is backed up by watching elections, where across many nations, they are generally fairly close, with about 48% voting one way and 52% voting the other. These proportions swing one way and the other over a decade, but that swing is usually within those bounds.
Third, up until the Google/Amazon* era, we didn't have the computing power, statistical tools and programming knowledge to make historical predictions, but
now we do.
So, why have election polls got it so wrong in recent years in Australia, Britain, US? Because they use sample sizes that are too small. My second point shows that with the population being so close in Progress/Conservative proportions, and with a small (only a few percent) making the difference one way or another, you have to have extremely large sample sizes, approaching 100% of the population, to get an accurate reading. The only way we have been doing that is to actually have an election.
Could we do better? Yes, by using the data and tools that Google/Amazon have amassed. They have very large sample sizes, over many years, and the tools to correlate that with predicted behaviour (they wouldn't do all of this expensive work unless it made money for them). This way we could better predict how elections would go. Do we want to do this? I am reminded of a short story where, due to data collection and analysis, elections were so predictable, that it boiled down to one person. That person was declared the "Elector" for that election. On election day, that person made their vote, that one vote was collated with the rest of the data, and the election declared.
So, could we use the these tools and data to predict history?
Should we?
I leave it as an exercise to the reader.
* I don't just mean Google and Amazon. I really mean BBDC, Big Business Data Collection. This includes the Tobacco companies, who have been analysing people and working out how to maintain and preferably increase their user base for over 70 years.