Before you begin, make sure the area is dry. 

Using a chisel or hammer, chip and knock away loose concrete from the site.  This includes loose concrete and rocks in the hole as well as around the hole.