Nice Idea But Needs Work
You need to find a way to make it more difficult. After getting used to the abrupt start its easy to remember the numbers. Its always just 5 of 4 numbers and the numbers are intuitively placed. If you're going to stick with numbers make sure that there are more doors than numbers that need to be remembered and that the numbers are not in the same order for every set of 5.
Or maybe just drop numbers altogether and go with something more difficult like colors or even colored numbers. Simple example with four doors:
Remember: 1 (red) 2 (red) 2 (blue) 1 (blue)
Doors Shown: 1 (red) 1 (blue) 2 (red) 2 (blue)
Of course it should get a lot more intricate than this but I hope you get the idea. Basically, even if people remember the order of the numbers they'll also have to remember the color of those numbers and if the order of the doors shown switches every set of four numbers (in this case) they will not easily get used to sending the guy to the correct door each time.
If you remake this let me know. Best of luck.