A good start; it's still pretty basic, but it's already fun.
I really like the 1 gem / 4 rocks limit -- it makes for interesting choices.
Can't decide whether I like the RNG wave order -- it's fun not knowing what's coming, but sometimes when the RNG is already refusing to give you any obsidian, then it rolls a high armor boss you can't touch, it just feels mean.

A few bugs:
You can place two gems (and four rocks) before the initial wave, by selecting a gem, rerolling, placing the first gem, then selecting one of the rorolled gems and placing it as well.
Enemies always path to top right corner -- on maps where your castle is somewhere else, this breaks the game.
Sometimes, the game freezes when an enemy steps on one of the damage tiles built into the level. It seems specific to the level, or possibly the type of damage tile, as most levels with tiles work fine. I noticed it on the level where the whole map is covered in Blackstone except paths between monster camps, the waypoint, and the castle. IIRC explosive damage tiles.
Inventory seem to persist when restarting the game. I started a game on the easiest setting, just to see what it was like, and played through the tutorial. Then I restarted on second-highest setting, and I started out with the same upgrades in inventory.

One gameplay complaint:
Infinite rerolls before the first wave makes for tedious play. Optimal strategy is to reroll 100+ times to get 4 of a kind (or 1000+ to get 4 of a specific color, probably aquamarine), and stack them for a tier 3 gem. And with the two-gem bug, it's even worse: optimal play is to reroll for 4 of a kind, stack them for a tier 3, reroll for 4 of a kind again, place the first t3 gem, stack the new gems, and place the new t3 gem. Consider limiting the initial draw no more than 2 of a single color, or limiting the number of rerolls.