It is a Watch and not a person. It can't 'see' everything and every way you do something to formulate a 100% objective understanding of the activity and the calories burned. Take 2 examples.
Running: This is easy because every arm swing of the accelerometer equals 2 steps. HR is also fairly consistent during a cardio workout.
Cross Fit: Externally hard to use the accelerometer to understand what exercise or movement is used. Moments and exercises are constantly changing and very dynamic. HR is all over the charts and is hard to read because of constant tissue constriction. Also anaerobic exercise caloric burn is not directly tied to HR values.
Yeah I see your point. I guess I don't expect it to be perfect and 100% accurate. But I do expect that it is consistent. When Apple advertised this product to monitor your day-to-day activity to help count calories, the message to me was that you only had to wear the thing and it would consistently count them (however it does). The workout app seemed like a more accurate way to count calories for specific motions as well as give extras like timers, GPS, distances, etc. But I didn't think it would be so different with and without the workout app engaged. If Apple never advertised this as monitoring your day to day activity (i.e. brisk walk, gardening, playing with the kids) as a way to count calories, I wouldn't have this assumption or expectation. But they did.
Walking/Running: It seems like the AW should be able to sense this sort of motion with and without the workout app and give you credit. Heart rate should add to the reading to give an idea of intensity level.
Gardening/Playing with kids: More similar to Crossfit (from AW perspective) where there are no regular movements. So how is it supposed to know how many calories you're burning? Heart rate seems to be a great indicator for this type of thing.
I am sure I'm wrong about this, but I thought "Other" workout was supposed to be this generic workout that is similar to the generic monitoring that is always going on without the Workout app. If you did a cross-fit workout with Other and one with no Workout, I would hope your calorie count should be sort of similar. If they are not, how can Apple claim they monitor your day-to-day activities with any sort of consistency. I don't expect them to be exact though... when you select Other it puts the heart rate sensor into play and it knows you're intentionally doing some specific workout. But how different is close enough? I'd say if they were within 20% of each other I'd be happy.
Here are two examples:
I did a 28 minute elliptical workout with and without the Workout app (Elliptical).
No Workout app: 86 calories, 2 minutes on exercise ring
With Workout app: 301 calories, 20+ minutes on the exercise ring
If it was any good at all at measuring day-to-day activity I would have expected these numbers to be closer together.
I also just did an experiment sitting at my desk here. 20 minutes just sitting here and 20 minutes in an "Other" workout.
No Workout app: 15 calories, 1 minutes of exercise (I did get up to go to the bathroom)
With Workout app: 89 calories, 19 minutes of exercise
The workout app should have sensed I wasn't doing anything! Why is it giving me any workout credit for sitting at my desk?
I just don't think you can trust the calorie burn this watch tells you is all. It seems like it is underestimating your calorie burn when not engaging the workout app and over-estimating when the workout app is engaged.