How much does the science matter when picking a specialty?

So ever since I decided to go into medicine, I was interested in neuroscience and wanted to be some sort of neuro doc. Now that I am a few months into medical school and I have been able to shadow and speak with more physicians, they seem to emphasize that, while an interest in the science and system functions is important, the day to day and the way that you get to treat patients as a clinician matters more. My question is, if I were to find something else that I am interested in the... How much does the science matter when picking a specialty?
Source: Student Doctor Network - Category: Universities & Medical Training Authors: Source Type: forums