…The problem is, if you make a task look hard, if you have to exert a lot of physical and emotional effort to accomplish it, you end up teaching your body that that task is hard. I see it all the time in the context of lifting weights: women and men just barely grinding out reps, grimacing in pain the whole time. Not to mention, consistent all-out effort often leads to all-out injury.
I really don’t think this is a major problem for 90% of the population. Most people are far from overdoing it, and get sore going gardening. The problem is doing nothing at all, or acting like walking your dog around the block is a worthwhile fitness program. Sure, “something is better than nothing”, but if “better than absolutely nothing” is your fitness plan, you might try amping it up slowly until leisure activities don’t count as a workout for you.