+1 to this. On some TVs the latency is awful (hundreds of milliseconds), game mode makes a huge difference on those. Makes a difference on any TV, but on most Samsung TVs, it's like 18ms with game mode off and 6ms with game mode on, which is an improvement, but 18ms is already pretty low and I don't notice any lag personally in the types of games I play.
Game Mode might be a separate setting (as it is on my TV) but on a lot of TVs it's under the picture mode setting (where you select Movie, Dynamic, User etc...), which is a bit confusing as you expect that to only affect the picture settings like brightness, contrast etc. Every TV made in the last 10+ years should have the setting somewhere though.
But also. The Switch Pro Controller is bluetooth, and bluetooth is known for having latency. I've never noticed any latency from it when I was using it on the Switch, but I did notice latency when I tried using it with my phone, especially when I had bluetooth headphones connected to my phone at the same time. Some people are more sensitive to it than others though and it also depends on what type of games you play.
The only way to get rid of that latency is to use the controller wired, since the Switch only supports bluetooth.