At the top Home level of Report Manager in security>properties give a Windows group you've setup specifically for reporting the Browser role. Now below 'Home' create folders that will contain each subset of reports and set the security on each of these to only the Windows group that you want to access them. So that all users can access the top level of Report Manager through AD add each of your windows groups to this top level windows group.
By doing this all users in all groups will be able to access the Report Manager homepage but they will only be able to view folders and content they are authenticated to.