I went back to school for game design when I was 25. I'm 29 now, and almost finished with degree... I work full time, go to school full time, have a steady relationship, have many bills to pay... and still find time to work on flash projects.
That being said though...
If I could go back to being 18-19, I would have focused ALL of my time into my dreams, instead of drudging along "trying to make it" in the professional world.
I really wasted all of my time from 18-25, which was probably the BEST time for me to just try something I wanted to do... because it was before I had the responsibilities now.
As much as I'm an advocate of getting a degree to secure your future, I say Go for it. You're young, and there's plenty of time to recover. If it doesn't work, just go back to school when you're 25.
=)