I generally agree with what rubikks said, but...
The tune could stay if it was some atmospheric game set in for example 1920s. I LOVE the idea of playing as a referee. I haven't seen it ever before. Keep working on it. Graphics don't have to be exceptional, you can even keep everything minimalistic, like a bunch of black-and-white dots (representing players) moving round a grey rectangle (representing a football pitch).
Maybe keeping graphics very simple would help. I see you're not an artist :P so, if you want the game to look serious, try doing that. If you take it as a long-term project, then you can use its potential and create something big.