If you get the Visual Studio 2012 or 2013 web installer then it uses the BITS service to download the setup files.
The BITS service is used to download files but throttles the download so it doesn't overwhelm your bandwidth.
If you are also downloading lots of other things like windows updates then the visual studio downloads have to take their time in the queue with everything else.
If you want to download to go quicker, you can disable and stop the BITS service, the installer will fall back to traditional HTTP can be much faster (depending on what other things are trying to use BITS).