Been busy with other things, so I haven't done much work on this the last couple weeks.
BUT...
I've found I can have many many many more asteroids on the screen at once by using a very simple trick: Any asteroid further than a certain distance is now just a sprite. Simple, stupid, but effective.