I have been working with SQL 2000 and since that knowing that SQL server ill use and release memory depending on its requirement and unitization. This is not true with SQL 2008 while it is installed on Windows server 2008.
SQL 2008 will utilize as much memory assign to SQL Server while high resource oriented task running. It will never releasing this memory back to system though task finish. If any other task needed memory SQL internally flush out the older data from the memory. To release memory reserved by SQL 2008 you need to re-start SQL services.
I also observed sometime in task manager also assigned memory not showing against the SQL Server.