I feel as if I need some sort of additions to the game wrt. attacking/defending. Like soldiers and barracks or something.
That's actually what I'm working on right now.
I'm adding heroes to the game (Automated little villagers who can't work, but defend the village), and the influence tab that will allow you to try to nudge your little guys to do things you want. But by design, it won't always work.
Also going to be adding barracks and basic fighters too, although I'm not sure if they'll make it into this next update.