Programming is as much about the theory as it is about the actual typing.
You need a theory first, which for you is along the lines of: Select posts from database where the category there in is equal to the ID provided and limit the output to 5.
So, that would be
mysql_query("SELECT * from posts WHERE categoryid='$catid' ORDER BY ID DESC LIMIT 5");
So now you have your query, you need to manipulate the data, so you need to create a loop that goes through each post that matches the criteria provided through the query and echos the data. Look up: mysql selecting data, select WHERE, order by, limit and echo.