Post reply

SSIS 2012 to MySQL data load performance

  • February 22, 2016 at 2:35 am

    #324439

    Hello,

    We have a data warehouse built on SQL Server 2012 in Amazon EC2 environment and we have another database on MySQL on Amazon RDS environment.

    We are trying to export more than 5 million records from SQL server to MySQL using SSIS. This data load process takes about 2 hours to complete.

    But when we try to export the data from SQL Server 2012 on EC2 environment to another SQL server database on RDS environment, the data load is getting completed in less than 4 minutes.

    Is there any way we can achieve the same performance with MySQL too? Please help.

    Regards

    Navaneeth.S

  • February 22, 2016 at 7:35 am

    #1860086

    there are a bunch of settings you can do in MySQL to improve the loading of flat files; back when i loaded that huge Ashley Madison dump for fun, i discovered how painfully slow the default settings are.

    The settings and notes i have are on my home laptop, so i cannot provide details until tonight, but

    google a bit for "mysql data load slow" or similar criteria, and see what settings are recommended.

    https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=mysql%20data%20load%20slow

    Lowell

    --help us help you! If you post a question, make sure you include a CREATE TABLE... statement and INSERT INTO... statement into that table to give the volunteers here representative data. with your description of the problem, we can provide a tested, verifiable solution to your question! asking the question the right way gets you a tested answer the fastest way possible!

  • February 22, 2016 at 7:59 am

    #1860094

    It's funny how Ashley Madison is becomming the defacto sample database for MySQL. :crazy:

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • February 23, 2016 at 2:35 am

    #1860315

    Thanks for your feedback. We will try this and post our comments.

    With Regards

    Navaneeth.S

Viewing 4 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply