TLDR:

His goal was to release Lucifer, which required breaking the Seals. To do this, he needed two things. Firstly, he needed to raise Lilith, as her death was the final scene. Secondly, he needed to raise a vessel for Lucifer to possess when he was finally released.

Long Answer:

In the Season 4 finale, Lucifer Rising, we see Azazel possessing a priest in 1972 and arriving at St. Mary's Convent. He slaughters eight nuns there and, using a dead nun, is able to speak directly to Lucifer. He tells him he has done this to speak to him, to help free him:

AZAZEL Father, look... I'm not exactly the praying type, but still... I made the sacrifice. I got you a bagful of nuns. So, uh... can you hear me? Can you whisper through the door?
DEAD NUN CHANNELING LUCIFER (in distorted gender-ambiguous voice) I'm here, my son.

Lucifer however, tells him that 66 Seals need to be broken that hold him captive - and that only Lillith can do this. However, he is told to find Lucifer "a special child":

AZAZEL So, uh... how do I bust you out?
LUCIFER Lilith.
AZAZEL Lilith? Father, she's... trapped neck-deep in the pit. It won't be easy.
LUCIFER Lilith. Lilith can break the seals.
AZAZEL Yeah, okay. But what do I do?
LUCIFER You must find me a child. A very special child.
AZAZEL (Eyes flash yellow) What do you mean? What child?

So at this point, his plan is starting to become clear. He needs to get Lilith out of whatever pit she is in and he needs to find Lucifer a special child.

With regards to the "special child"

Azazel then begins to go around the country and make deals with parents, allowing him to feed his blood to their babies later and make them develop supernatural abilities. Their abilities were strengthened depending on how much demon blood they consumed (as Sam's demise throughout Series 4 showed).

All in all, these were all the known "special" children (taken from the wiki):

• Sam Winchester
• Max Miller
• Andrew Gallagher
• Ansem Weems
• Ava Wilson
• Jake Talley
• Lily Baker
• Scott Carey

More information on them can be read on the linked wiki page. Whilst we originally saw their powers grow, the conversation from the Season 4 finale revealed their true purpose definitively - they were all potential vessels for Lucifer. Azazel had them battle at the end of Season 2 to find out who was the strongest (and thus most worthy) of the honour.

With regards to Lilith

Lilith is a little complicated to get into detail on, as the reason for being in the pit isn't explained on the show. When Sam kills her, this exchange does take place though:

SAM (in denial) No, no, no. No, he -- Lilith -- I stopped her. I killed her!
RUBY (fervently) And it is written that the first demon shall be the last seal. And you bust her open...

In other words, she was the first demon - the first ever. She was created by Lucifer himself. Whilst this doesn't explain why she is buried in the pit, this tallies with the real-world religious sources on Lilith, where she is regarded as one of the first demons (and in some sources, she's even considered one of the wives of Adam).

Conclusion

Azazel's ultimate goal was to release Lucifer. To do this, he needed two things. Firstly, he needed to raise Lilith, as her death was the final scene. Secondly, he needed to raise a vessel for Lucifer.

Ultimately, given his demise in Season 2, we never saw the power dynamic between himself and Lilith. However, the Companion Series for Supernatural does describe that the white-eyed demons were the highest rank there could be, so it seems likely she would outrank him.

Although initially, these special children were candidates for leading an army of demons in a war against humanity and God, Azazel's true goal was to create a suitable vessel for Lucifer when he returned.

http://supernatural.wikia.com/wiki/Special_children

Azazel needs a general to command the demon army he is planning to release from Hell.

In All Hell Breaks Loose (Part I), Azazel pits his special children against each other in a kill-or-be-killed, Battle Royale-like contest.

SAM: What happened to you?

AVA: Nothing!

[SAM stares her down until a minute later, she drops the act, and laughs a little.]

AVA: Had you going though, didn’t I? [wiping her eyes] Yeah. I’ve been here a long time. And not alone, either. People just keep showing up. Children, like us. Batches of three or four at a time.

SAM: You killed them? All of them?

AVA: I’m the undefeated heavyweight champ.

SAM: Oh, my God.

AVA: Don’t think God has much to do with this, Sam.

SAM: How could you?

AVA: I had no choice. It's me or them. After a while, it was easy. It was even kind of fun. I just stopped fighting it.

Azazel explains during the episode that he needs one soldier; the best of the best, and someone with leadership skills.

AZAZEL: I’m trying to help you. That’s why we’re talking. You’re the one I’m rooting for.

SAM: What’s that supposed to mean?

AZAZEL: Welcome to the Miss America pageant. Why do you think you’re here? This is a competition. Only one of you crazy kids is gonna make it out of here alive.

SAM: I thought we were supposed to be—

AZAZEL: Soldiers in a coming war? That’s true. You are. But here’s the thing: I don’t need soldiers. I need soldier. I just need the one.

SAM: Why?

AZAZEL: Well, I couldn’t just come out and say that, could I, Sam? I had to let everyone think they had a fighting chance. But what I need ... is a leader.

SAM: To lead who?

AZAZEL: Oh, I’ve already got my army. Or, I will soon, anyway.

SAM: You son of a bitch.

AZAZEL: Honestly, I’m surprised you hadn’t guessed. I mean, why do you think so many children flamed out already? Max Miller and Andy’s brother, what’s-his-name? They weren’t strong enough. I’m looking for the best and brightest of your generation.

Season 3 Spoilers ahead

This plan was already known by the demons, but Azazel's death in All Hell Breaks Loose (Part II) leaves a gap in the leadership of the newly released demon army.

From Sin City

DEAN: So, the gate opened. The demon army was let out. What now, huh? I'm not seeing a big, honking plan here.

CASEY: Honestly, there was a plan. Azazel was a tyrant, but ... he held us all together.

DEAN: Azazel?

CASEY: What, you think his friends just called him “yellow eyes”? He had a name. After you did him in, it all fell apart.

DEAN: (smiling) Sorry about that. So, what? No chain of command?

CASEY: There was. It was Sam. Sam was supposed to be the grand pooh-bah and lead the big army, but ... he hasn't exactly stepped up to the plate, has he?

DEAN: Thank God for that.

CASEY: Again with God. You think this is a good thing? Now you've got chaos, a war without a front, hundreds of demons all jockeying for power, all fighting for the crown. Most of them gunning for your brother. For the record, I was ready to follow Sam.

And later, we find that Lilith has taken control of the demon army.

DEAN: Oh, good. 30. 30 [demons] all gunning for us. Who sent them?

RUBY looks at SAM, who is now standing in the doorway.

RUBY: You didn’t tell Dean?

DEAN looks at SAM, confused.

RUBY: Oh, I’m surprised.

DEAN: Tell me what?

RUBY: There’s a big new up and corner. Real pied piper.

DEAN: Who is he?

RUBY: Not he. Her. Her name is Lilith.

DEAN: Lilith?

RUBY: And she really, really wants Sam’s intestines on a stick. ‘Cause she sees him as competition.

Sam and Dean were genetically engineered by Heaven to be the vessels. I believe the fact that Sam was part of Azazel's plan was more of a coincidence. Cas even says he knows what he did to Sam, but not why. I think it was more that Azazel was trying to make a power play and ended up failing when he inadvertently got in the way of the Winchester brothers.

Really that whole plot gave us two things: the beginning of the show and the reason why John got into hunting, and Sam's psychic abilities (now whether those were always "in" him and just heightened by demon blood or simply because of it I'm not sure)

If Azazel was just trying to create a vessel for Lucifer, Sam wouldn't have been nearly as important. It needed to be the blood line of Mary and John, not the psychic stuff (also a note since Dean obviously did not need any outside force to be a vessel).