This probably is a case of SQL Server grabbing as much memory as it is allowed to - which is by design. Right-click on the server in the object explorer in SSMS and select properties. In the memory tab, it will show you the maximum memory SQL OS is allowed to take. You would want to reduce that making sure that you leave enough memory for Windows and for other applications.
Google for "configuring maximum memory in sql server" and you will see MSDN guidelines on how to and what to set the maximum memory to.