Harpur was born on 23 January 1813 at Windsor, New South Wales, the third child of Joseph Harpur — originally from Kinsale, County Cork, Ireland, parish clerk and master of the Windsor district school — and Sarah, née Chidley (from Somerset; both had been transported.) Harpur received his elementary education in Windsor. This was probably largely supplemented by private study; he was an eager reader of William Shakespeare. Harpur followed various avocations in the bush and for some years in his twenties held a clerical position at the post office in Sydney.