richieguitar is right; sometimes the bounce physics are off. I just had one come at the wall at about a 70 degree angle, and left closer to 30 degrees, both players fairly far away from the thing. Guessing the wall is composed of objects, and the puck hit a seam?
Pong is hard to reinvent. I'm not sure you managed to capture it well in this iteration, and the circular player-pieces don't help the sense of a loss of agency (which the speed also contributes to). All in all it feels skillless, which is unfortunate. If it were slower it might be suitable for a touchscreen application, but mice and trackballs and touchpads just can't be precise and reliable enough for what this game takes.
Recommended playing: Shatter, on PSN (and probably Steam, too). Based heavily on pong as well, and there's a lot of things it does that make it a game with some control but without losing a sense of chaos most times.