I'm sorry - I know that isn't the news you were expecting. I am a bit surprised the doctor is doing 5 more days of Clomid - that isn't an approach I've heard before. For me, I ended up getting several days of Follistim injections.
As for why not wait a couple of weeks . . . well, the theory is that if it takes the follicles a long time to grow/develop, they are not "good" quality, and will not be as likely to result in a

.
From my non-professional perspective, it sounds like you are not a good Clomid responder (which is the case for me, too). Clomid does indirect stimulation - it lowers estrogen to trick your body into producing all kinds of other hormones. Some people just don't respond well to that approach, but do really well instead with direct stimulation (FSH injections). That was the case for me.
GL to you . . . I hope this does the trick for you, and I'm sorry your body isn't cooperating.