I think Boss has nailed this one. It is normal for the contact to wear, and while it is wearing, you may find that "pulling harder" will work, as you described. Then, when you changed the switch, it was different enough that it didn't fully connect with the copper contact point very much at all, so you get sparks, but not enough amperage to turn the starter over. I'm betting with Boss that a new contact will fix this starter. However, if you take it in to a specialist, they can clean up any other issues that go along with wear and time, so that you get back a well functioning starter.
'52 Cub ("Great Personality") 148xxx
'48 Cub with FH ("Gunny Cub") 38xxx
'57 Lambretta (a slow work in progress)
'74 Triumph TR6 (Mama's toy)