Basicly, you have your tile coordinate system (space) in which (0,0) refers to top left of tiles, and (x,y) refers to the top-left of the tile in the y'th row and x'th collumn
You then have the projected space which is the result of transforming the tile coordinate system so that it is in screen coordinates for rendering basicly.
You can see the two spaces in the flash, and how moving mouse around in the bottom right one displays the position in tile-space and vice-versa. The top left is moved and scaled so that it isn't super tiny as it would be if represented properly in screen space (Since without scaling and moving it, the tile-space image would only cover an 8x8pixel area at top left of screen)
Link to Flash
You can also see the two transformation matrices, T is the one that goes from tile space to projected space, and T^-1 is the other way round.
R refers to the value of the red vector in the projected space, and B the value of the blue vector in the projected space with P being the coordinate of the origin in projected space.
You can see how going from a position of (x,y) in tile space, gives you a position in the projected screen space of (P + xR + yB) and that is the basis of the transformations:
Letting the values of P R and B be represented as Px,Py Rx,Ry Bx,By the two transformations become the matrices
= [Rx Bx Px]
T = [Ry By Py]
= [0 0 1 ]
and
-1 [a b c]
T = [d e f]
= [0 0 1]
where you can calculate the values for T^-1 as:
var idet:Number = 1/(Rx*By-Bx*Ry);
var a:Number = By*idet;
var b:Number = -Bx*idet;
var c:Number = (Bx*Py-By*Px)*idet;
var d:Number = -Ry*idet;
var e:Number = Rx*idet;
var f:Number = (Ry*Px-Rx*Py)*idet;
and hence you're two transformations on the vector (x,y,1) as being
//From tile to projected
var x2:Number = Rx*x+Bx*y+Px;
var y2:Number = Ry*x+By*y+Py;
//From projected to tile
var x2:Number = a*x+b*y+c;
var y2:Number = d*x+e*y+f;
if you project mouse coordinate into tile space, and take the integer part; you will have the indices relating to which tile it is on: so if the coordinate of mouse in tile space is
(12.68,7.18) the tile it is over is in the 7th row, and 12th collumn