How big are the biggest rocks?
I've run rototillers in rocky soil before, where no rock was bigger than about baseball size. It worked fine. It was bumpy as hell, and there was lots of stop-and-go driving to pick up the big rocks in order to move forward, but the tiller didn't stall or explode or anything. Get a tiller with big spacing between the blades. It's the spacing that will determine how well it can work over or through rocks.
Before tilling, I also went over the ground first and used a garden fork to dig up the bigger rocks that had any exposed surface. I used a shovel, too, but the fork is better for breaking up the surrounding soil before shoveling.
The things I'd be most leary about rototilling are cable, wire, rope, and stuff like that. It'll get wound around the axle pretty quickly, and the tiller blades have enough leverage to make it a real bear to get off manually. A pair of diagonal wire-cutters is better than scissors for cleaning up mistakes.
And make sure you have several spare shear pins on hand before you start. You'll almost certainly break a few, and they're better to have and not need, than need and not have.
