I use a Line6 Toneport, and it's true that the Line6 interfaces have tone direct monitoring (at least mine does) that offers zero latency.
However, it ONLY works on the guitar line in, every other line-in bypasses the tone direct system. Also, as far as I understand it, the tone-direct line only works for monitoring, and probably only through the line6 software, so if you're using something else, such as amplitube, guitar rig, etc. then I dont think you get the tone-direct monitoring, just regular asio.
I run 44 khz @ 24 bit / 512 buffer and dont experience any latency issues in windows vista using the Line6 supplied drives.
However, I have been able to tighten latency settings further using Asio4All, it's vastly superior to the line6 drivers in most cases.
I haven't seen anyone mention it, but is your computer up to the task of virtual amp simulation? If you're using settings with a lot of effects, especially things like reverb and delay, it will bring your computer to its knees and WILL introduce very bad latency in excess of one second.
Try using simple amp models with no effects applied (just an amp, nothing else loaded up) and see if your latency is any better. If it is, then your computer is the problem, if it's the same, then you've got a problem somewhere else.
Also, for the record, Line6 interfaces are pieces of shit, I can't wait to get rid of mine and buy something half decent. They're good enough for dicking around but if you're serious about recording stuff, spend a bit more and get a good interface with good preamps, because Line6's are noisy as hell.