As far as I know, the process is something along these lines:
1. Members of a party declare candidacies for the leadership role.
2. The members go through a campaign not unlike the one to get elected to a government position.
3. The party's members vote on who they want as their leader.
4. Whoever gets the most votes becomes the leader.
Basically, anyone who's a member of the party can declare candidacy, I think, although if you're not fairly high up, you probably don't have that much of a chance.
They usually have a number of ballots, depending on if a candidate can get over 50% of the vote in the first round. They keep knocking off candidates until one gets over 50%.
As for leaders who have no seats in the House, often they will wait for the next election, by-election, or occaisonally a member of their party will resign to open a seat up for a by-election.