Way too difficult. Even with a maxed out castle, your archers just cannot defend the base. There are way too many enemies, and at day 4 they take more than one hit. They also climb over the walls, as if they weren't powerful enough. The archers need to be more powerful, or the enemies need to be less powerful, or the castle needs to accommodate more archers. Even then the workers try to fight, or repair walls as they're being broken, and end up dying pointlessly.
Don't even try taking more land. The barriers there are out of the range of your main castle, and fall without any opposition. Meanwhile, the workers are roaming around further from the main castle because they consider all territory safe territory. When the enemies break the wall, the workers try to fight instead of run. Then the land is lost. I've only seen a successful harvest of the farms once. You get three coins. For the effort it takes, you might as well get 50.
The game just needs a ton of balancing. It looks really promising, and it's already very interesting.