There is slack - but the slack is in the house schedule not the human. A house can sit for several work hours with nothing going on and not care. The humans working on the house do not like being idle - they are working for money and when they have no work they get no pay and in turn that makes it hard for them to pay their own bills. The bank cares a bit about idle time, but the interest charges from a few hours here and there of idle time isn't that much and so that is where builders put the slack in. Of course those interest charges still add up and so there is effort to reduce them but the costs of scheduling a crew and then not being ready for them adds up even more than interest.