That's somewhat mitigated if the game has good, always-present map markers. For example, Ghost of Tsushima was an incredibly smooth and fun experience, despite being somewhat non-linear, since it clearly showed where locations and quests could be located, and guided players to them with the wind.Non-linearity in game design usually just results in getting lost and being frustrated, which isn't fun and is bad.
If there is no aid like that (for example, Breath of the Wild), it immediately becomes a boring, frustrating disaster that quickly leads to becoming lost and a loss of motivation. I should not have to check an online map to find the game's content, or have to place my own inaccurate map markers.