Most upvoted comment
Why can I track a moving object with my eyes smoothly, but not follow a line on the floor without ‘stuttering’?(r/askscience)
It is a limitation of your brain.
Control of your extraocular muscles (the ones that move your eyes) is extremely complicated. There are two main pathways that control the two things you mention.
The smooth pursuit pathway uses visual input to track objects and follow them. Because it relies on visual input, you cannot force yourself to make smooth movements with your eyes. The voluntary eye movements use…
The saccadic pathway, which makes darting movements. These position your eyes very quickly.
The pathways involved in these are still being studied. For a more detailed discussion of the pathways, I recommend Martin’s Neuroanatomy
EDIT: there is an interesting disorder called ocular motor apraxia, in which the saccadic pathway doesn’t function. These people (usually children that grow out of it) rely entirely on their pursuit pathway. Once they fixate on an object, they can’t look away unless they turn their heads so far as to force their eyes to refixate.
Here’s a video showing a toddler with the disorder. Notice how she has to turn her head 90 degrees to look away from the ball.