The Major differences are:
In a Web Application project everything is pre-compiled all the codebehind pages will be compiled into a .dll ---- In a Web Site Project nothing in the project is pre-compiled, the compiler will compile everything to ensure it is valid but none of the compiled pages are uploaded. When a user first attempts to access the site each page is compiled into its own dll. This means in a Web Site Project you are able to upload a single codebehind file.
Namespaces - In a Web Application project namespaces are created by default in a Web Site Project they are not.
Project files - A Website Project does not have a "cproj" file a Web Application project does.
Converting to a Web Application project can be more difficult then you think especially if you rely heavily the appcode folder.
I personally prefer a Web Application projects I find them easier to use and less annoying to deploy. I would personally only use web site project on something very small and simple.
Extra reading from MSDN