You can only upgrade the builders hut with diamonds, not buying the diamonds just means that you are limited to 2 builders.
The Magic Academy also has a limit on the amount you can upgrade it without diamonds.
There are ways to get free diamonds:
1) You start off with 200 diamonds, and there...