I've done co-sleeping with a newborn and toddler, and we all slept fine (probably because if baby woke up I nursed, lol.) You will all probably be doing lots of napping those first couple weeks.

The only problem I could see is if you get up every time baby cries, the toddler might get a bit jelous and try to get you to come back to your bed.