Web Site is what you deploy to an ASP.NET web server such as IIS. Just a bunch of files and folders. Theres nothing in a Web Site that ties you to Visual Studio (theres no project file). Code-generation and compilation of web pages (such as .aspx, .ascx, .master) is done dynamically at runtime, and changes to these files are detected by the framework and automatically re-compiled. You can put code that you want to share between pages in the special App_Code folder, or you can pre-compile it and put the assembly in the Bin folder.
The main difference with Web Sites is that when you build the project all the code files are compiled into a single assembly, which is placed in the bin directory. You dont deploy code files to the web server.
Deploying shared code files is generally a bad idea, but that doesnt mean you have to choose Web Application. You can have a Web Site that references a class library project that holds all the code for the Web Site. Web Applications is just a convenient way to do it.
By having all code files compiled into a single assembly, including codebehind files of .aspx pages and .ascx controls, in Web Applications you have to re-build for every little change, and you cannot make live changes. This can be a real pain during development, since you have to keep re-building to see the changes, while with Web Sites changes are detected by the runtime and pages/controls are automatically recompiled.
Having the runtime manage the codebehind assemblies is less work for you, since you don't need to worry about giving pages/controls unique names, or organizing them into different namespaces.
Im not saying deploying code files is always a good idea (specially not in the case of shared code files), but codebehind files should only contain code that perform UI specific tasks, wire-up events handlers, etc. Your application should be layered so that important code always end up in the Bin folder. If that is the case then deploying codebehind files shouldn't be considered harmful.
Another limitation of Web Applications is that you can only use the language of the project. In Web Sites you can have some pages in C#, some in VB, etc. No need for special Visual Studio support. Thats the beauty of the build provider extensibility.
Also, in Web Applications you don't get error detection in pages/controls as the compiler only compiles your codebehind classes and not the markup code (in MVC you can fix this using the MvcBuildViews option), which is compiled at runtime.
Building a Web Application is faster than building a Web Site, specially for large sites. This is mainly because Web Applications do not compile the markup code. In MVC if you set MvcBuildViews to true then it compiles the markup code and you get error detection, which is very useful. The down side is that every time you build the solution it builds the complete site, which can be slow and inefficient, specially if you are not editing the site. l find myself turning MvcBuildViews on and off (which requires a project unload). On the other hand, with Web Sites you can choose if you want to build the site as part of the solution or not. If you choose not to, then building the solution is very fast, and you can always click on the Web Site node and select Build, if youve made changes.
In an MVC Web Application project you have extra commands and dialogs for common tasks, like Add View, Go To View, Add Controller, etc. These are not available in an MVC Web Site.
If you use IIS Express as the development server, in Web Sites you can add virtual directories. This option is not available in Web Applications.
Because the programmers write the application, the application is then built. The test team tests the application on the test system. Then the customer installs the applications. That LAST think you want is anyone making live changes!
I'm not saying making live changes is good or bad, just that webapp projects doesn't support them.
Which is a good thing IMHO. It makes a mockery of any code, test, release process which has to be the bedrock of any serious application
To me having the choice is the best, on websites you can always inherit from a pre-compiled base class if you want to. There are many languages/frameworks (e.g. PHP) where people are used to the idea of deploying source code. That doesn't mean those are not 'serious' applications.
"Actually, you don't manage those DLL's, [...] you don't even have to know they exists. Not an issue." - Until the framework gets confused, doesn't clean the old versions up correctly, and starts throwing compilation exceptions with conflicting names all over the site... You can add error detection of the markup through the use of a WebDeployment project. I'm also not sure on your last point "with websites you can use IIS as the server", you can do this with a Web Application as well - and I have projects like this where the project is part of a larger web application.