It's not the max resolution of your screen that you set base and output settings to, it's the screen resolution you are using in your game (Options - Screen mode), if you don't do this the capture window will be larger than the video. The drag handles are simply used to zoom in (and out) so that will only make the final video more pixellated.

To improve your fps try one or more of the following if you haven't already;

turn off the big screen
reduce anti-aliasing or turn it off
reduce draw distance
turn off 3d trees
turn off the crowd