Hmmm....
Well, I do get the joke, but it's not particularly funny. It came off as a little awkward, not in a good way. I think that things like timing, and some sort of flow would be essential in a joke like this.
The ol' turnaround gag requires you to catch someone by surprise, even shock them. But this...for whatever reason...didn't work.