September 8, 2009 at 12:29 am
I have designed a database architecture to meet our business requirement and keep database load on two separate servers so that users requests and load can be distributed for high performance and reliability.
I have attached the design diagram for easy to understand. There are two separate servers. Each server has database, One (Frontend db - subscriber) and one (Backend-Publisher) db. Frontend db would keep only last 7 days data into it and it will synchronize once per day with Backend server. All the rest of data would be available on Backend server since the application is online year 2006.
Initially we had one database server and there were 5000 users hitting site at a time, but now we have to achieve a target of 100,000 online users hitting site at a time. So we have decided to change the design database and application - here i am just asking about DB design. To keep application db lighter we have decided to keep only last 7 days data, coz some users has long session and application would be updating live session on the fly.
Please look into this design and if you have any question let me know.
Following are my questions for which your answers will be very helpful
1. Does any aspect of this design lead us to a disaster if we implement it? If yes please point it out and let us know why it could be a disaster and if there is any better solution exist.
2. Do you think if this architecture will involve heavy maintenance cost which could be a nightmare for whole project?
Shamshad Ali.
September 16, 2009 at 7:50 am
First, if this is going to support 100,000 people and it's worth a lot to your company, you ought to engage a consultant that works with systems like this. Someone with experience under high loads.
I'm confused. Why is the backend system a publisher and the front end a subscriber? If the last 7 days is kept on the front, I'd think that was the publisher.
Why merge? and why synchronize once per day?
You might want to read about some of the MySpace architecture adn tricks they went through as they grew.
September 18, 2009 at 5:02 am
First of all, Thanks you very much for you time you looked my question and replied.
First, if this is going to support 100,000 people and it's worth a lot to your company, you ought to engage a consultant that works with systems like this. Someone with experience under high loads.
Is it possible to forget any consultant at this moment? Can not someone help here on this forum? How will one get experience if no one is going to provide consultancy.
I'm confused. Why is the backend system a publisher and the front end a subscriber? If the last 7 days is kept on the front, I'd think that was the publisher.
Let me clear your confusion. The problem is we have to keep small load on FrontEnd Cluster and if we configure Publisher on Frontend it would have overhead to replicate data to backend.
Why merge? and why synchronize once per day?
You have not clearly read the points I mentioned in diagram. Merge replication is a requirement for few tables that could be updated from both Frontend cluser as well as from Backend Cluster. So there is no any easy way to merge data.
Next, to overcome the load on live db server data being replicated once at night From frontend to Backend. Admin users use Backend only for reporting purpose or for uploading training material videos so if today's users tracking data is missing in admin reprorts until it synchronize admin users can compromise this.
You might want to read about some of the MySpace architecture adn tricks they went through as they grew.
We studied http://highscalability.com and come up with above solution as i mentioned and wanted to get help on this design.
You can also review my complete question at:
Shamshad Ali.
September 18, 2009 at 10:00 am
People on this forum provide help for free. We don't do consultancy or design. That's what a consultant if for. You pay for their expertise, and they provide you information. You ought to be learning and asking them why they do things along the way. We provide short, relatively quick opinions and answers. Not detailed review of designs.
A publisher doesn't add a lot of load, but if you are sending data back from the front end, even with merge, it is a publisher. If it overloads your server, likely your hardware is not adequately sized.
September 19, 2009 at 1:09 pm
People on this forum provide help for free. We don't do consultancy or design. That's what a consultant if for. You pay for their expertise, and they provide you information. You ought to be learning and asking them why they do things along the way. We provide short, relatively quick opinions and answers. Not detailed review of designs.
Thanks for giving me straightforward reply. Can you help me findout a consultant who can help me and take interest on finding the solution?
A publisher doesn't add a lot of load, but if you are sending data back from the front end, even with merge, it is a publisher. If it overloads your server, likely your hardware is not adequately sized.
If the distribution is there with Publisher than it would be load on Frontend which we are assuming to face 100,000 no. of online users. Also frontend would keep only last 7 days data and purge 8th days data, we will disable delete trigger of replication so that it won't remove data from Backend during synchronization with Backend. Backend has all data for reporting purpose. Right now we are not going for complete data warehouse. If you need any further details let me know. I will be happy to explain.
Shamshad Ali.
Viewing 5 posts - 1 through 4 (of 4 total)
You must be logged in to reply to this topic. Login to reply