Please don't put a brass bush in an aluminium bore, ever. That just makes a neat little voltaic cell which eats the aluminium whenever it gets water on it.
There is a good chance the bore can just be polished. If not, I suggest you use a thin steel bush, that is a transition fit on the axle. That way you won't much care whether it becomes frozen into the aluminium wheel, since you won't want to take it apart there anyway, you'll just slide the steel bush off the steel axle.
