I understand what you're getting at, but there is an influence from the software. Your argument is assuming the necessary data is actually available (and in such cases, is correct).
It has to do with how the application loads the data to the RAM from disk. If it's not optimized to keep it fed in a manner to reduce/eliminate misses, it's going to slow the throughput down (i.e was the file loaded sequentially for all cores, or was it broken down in a manner that allows it to load data simultaneously, so no core is starved while it waits for the data it needs).
A poorly written compiler can cause issues with IO as well, and cause some degree of starvation. I see it essentially as a bottleneck caused by software rather than hardware.
Of course it is. But not all software can, and that which is possible, may have problems keeping the right data feed in time without the need to load data that should have been pre-loaded to prevent starvation as much as possible (not a situation where there's just a lack of RAM capacity for the task).
I'm looking at it from a commodity parts (fans in this instance) and dimensional POV (while keeping within PCB layout rules). That is, yes, you could go taller and narrower to accommodate another pair of DIMM slots, but what about air flow (i.e. available fans in terms of size and CFM)?
Taller and narrower heatsinks could force the use of a fan/s (potentially stacked) that're unable to push enough air flow due to dimensional limitations (width).
It's the "glued" aspect that I draw the definition though. That differentiates it from previous super computers (i.e. Crays of yore), as it was a single system. Multiple enclosures may have been needed, but it wasn't commodity systems via standardized interfaces, such as Infiniband, that modern clusters are created from.
I wasn't trying to indicate that workstations will obviate clusters. On the contrary, actually.
My point was, that current and not too distant workstations will improve the cost/performance ratio. The additional cores will allow users to do more work for less funds (i.e. 8 core SP chip vs. DP Quad core).
Now in the cases that even a DP system won't suffice, what I meant as "heavy lifting", a cluster/farm is an effective means to increase the performance and resulting output. And it's getting cheaper and easier to do on a smaller scale (i.e. dedicated or at least fewer users = fewer systems = lower cost). There's independent filmmakers for example, that have built render farms of miscellaneous PC's.
As GPGPU processing becomes more prevalent, that could increase the performance as well without the need to increase the unit count in the cluster to gain additional performance. And it should be more cost effective as well (i.e. SP boards with 3 or 4x graphics cards = better cost/performance ratio than a DP system).
Costs are getting lower, and making it more accessible to smaller companies, and even independents/consultants (video, engineering, scientific modeling,...). For example, instead of having to purchase time, it may now be possible to construct a dedicated cluster for the desired task. As it's dedicated, it could be made smaller (fewer systems = lower cost). Combine it with the increasing cost/performance ratios that are expected, it should become a viable solution to users previously unable to afford it.
It won't be affordable for everyone (dedicated in-house), but with reduced costs to construct a cluster, hosted time should also reduce, making that more accessible as well (thinking additional hosting companies will also rise, helping to keep costs in check through competition).
Scheduling has it's place, but it's not the only consideration I meant, as you've seen above.
I'm seeing the potential for clusters to become more commonly used as a result of multiple factors converging.
In some cases, this may be all that's needed, depending on interface and/or graphics requirements.
This is definitely the potential "fly in the ointment" aspect to clusters/farms.
Hopefully, software vendors that have products that're meant to be used this way, will license accordingly in order to generate sales in a manner that maximizes profits (don't make it so expensive that it causes potential purchasers to exclude the product from consideration over licensing costs).
But it remains to be seen, if this will actually happen.
