"must have known"

This is indeed speculative, although it is a somewhat reasonable assumption. If an answer choice had said the argument was flawed as it relies on an assumption about the beliefs of builders, that may be correct. However, that was not what E said.