Its not possible to share the same DB on both clusters.
essentially each instance runs with its own resources, i.e SAN drives which are only accessible by one active instance at a time. The easiest thing is setup your first instance so its running as its own cluster group.
So have your OLTP db running on that one.
Maske sure you can successfully fail back and forth both the windows and SQL Clusters. Within Windows 2K8 failover manager you cannot failover the windows cluster via the GUI and it has to be done at the command line.
Once thats all working well install your secondary instance and repeat the process.
After all that is OK setup your replication. It is much easier than it sounds and the SQL Server 2008 MS Resource kit book explains this pretty much step by step.
MCITP SQL 2005, MCSA SQL 2012