You have sufficient voltage to make sparks, illuminate a warning light, or what have you. Plus, the battery is new. The intermittent nature of the light, to start with, points to a lousy connection and/or wire. Arcing the starter should at least make sparks, even if it doesn't spin.
Start with the (I know, it's new) ground cable. Make sure it is onto something that is clean itself, and well-attached to the engine and/or the frame. Same for the positive cable to the starter. Test them both for continuity and resistance, just in case you got a bum cable.
Test the voltage at the key switch, and see what you're getting there, and compare it to the battery's voltage directly on the terminals. If you've gotten voltage to the switch, test each of the other terminals (3? 2?) in various positions (On, start, pre-heat if on the key etc) and see if they show voltage. If not, the key switch is defective.
I would try a jumper wire instead of the screwdriver, just to ensure you're getting good power to where you want it.
If everything is good but the starter doesn't want to work, pull it of and clean all the mounting surfaces in case they are corroded or fouled. You can even bench test it with a battery or battery booster directly, so see if you can get it to work. If none of that works, but your electrical system is fine, you're up for either a rebuild of your current starter or a replacement. Good luck!