What you can do is put a password for a media set for a given database. And only the user with proper permissions to restore a database must know the password you specified in the backup set.
This way you can restring the user that restores a database