Can babies / kids come down in other weeks?

No. Children 5 and under are only permitted in Tiny Tots weeks and in Whole of Lodge weeks.