Always centre punch the bolt before starting to drill to stop it wandering off like that. I reckon with a punch mark at an angle into the hole you have created you could probably get a small centre drill to start then go to a larger drill and slowly bring it to the upright position enough to get an easy out in and remove the old bolt. If you have only gone down a short way you probably have enough thread left in the hole to make a bolt hold. Use a Loctite thread retaining compound to make sure it doesn't come loose. You will probably need to go to an engineering supplies company such as Blackwoods for a centre drill or order one of Ebay. If you are not familiar with them it is a short drill with a small diameter that then steps up to a larger diameter. They are used in lathes to put a centre hole in metal for locating a live or dead centre.
