Import and upgrade strategy question

  • I've had this discussion with the lead programmer at work about how we need to start creating our database with scripts instead of deploying an actual database, and then doing updates to the database with scripts as we deploy new versions of our software. Thankfully, he had this in mind already. But there is something we disagree on.

    We have to have an import program to get legacy users' data into our new database structure. His thinking is that we should keep our importer up to date and every time we make a change to the database, the import program needs to be modified so that legacy users can be imported into the latest version of our database

    My feeling is that the importer should get their data into the database as it is at this moment, and then run all the scripts we've created since to get it up to date. It seems like double work to me to have to create the scripts *and* keep an importer up to date.

    Again, my idea of keeping the importer up to date is simply having it run the new scripts at the end after it imports into the db schema we have now, and his idea is that it needs to import directly into the latest db schema.

    Thoughts?

  • brandonmooreis (1/14/2011)


    I've had this discussion with the lead programmer at work about how we need to start creating our database with scripts instead of deploying an actual database, and then doing updates to the database with scripts as we deploy new versions of our software. Thankfully, he had this in mind already. But there is something we disagree on.

    We have to have an import program to get legacy users' data into our new database structure. His thinking is that we should keep our importer up to date and every time we make a change to the database, the import program needs to be modified so that legacy users can be imported into the latest version of our database

    My feeling is that the importer should get their data into the database as it is at this moment, and then run all the scripts we've created since to get it up to date. It seems like double work to me to have to create the scripts *and* keep an importer up to date.

    Again, my idea of keeping the importer up to date is simply having it run the new scripts at the end after it imports into the db schema we have now, and his idea is that it needs to import directly into the latest db schema.

    Thoughts?

    Is the data constantly changing in the old system? If so, you'll need to import again later.

    Will you continue to import data from the legacy system after go-live? If so, you'll need to keep the importer up to date.

    It depends on when the cutoff for the legacy data occurs. If it's never cutoff, keep it up to date. If it's already in lock down mode, do the import and be done with the antiques. 🙂


    - Craig Farrell

    Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

    For better assistance in answering your questions[/url] | Forum Netiquette
    For index/tuning help, follow these directions.[/url] |Tally Tables[/url]

    Twitter: @AnyWayDBA

  • I tend to like scripts for this, instead of a separate program. Especially as you'll start to have multiple versions of things quickly since SQL Server changes rapidly. Scripts seem to be easier and more flexible to quickly deploy, and even have the client make changes on their end if needed.

  • Thanks Craig. This legacy product is not going to be changing anymore. That's why I agree we should make an importer to the current version of the new product and be done with it. But not all legacy customers will upgrade immediately or at all, and if they upgrade after we've made changes to the new product I think we should just use the importer to get the legacy data into the schema the new database was in at the time the importer was made, and then run scripts to get it up to date.

    This is for a pos system. And I understand we should keep an importer up to date that imports from something like spreadsheets so that customers coming from other systems can get their data into our system. But it's obviously different when you're getting data from your own legacy product b/c your responsible for getting 'all' the data... not just 'most' of the important stuff. And I think that's where my coworkers disconnect is... he's thinking we need an importer that stays up to date with our most current schema all the time. But it's different when you're talking about upgrading a database that is literally what our current database came from and just needs to have the scripts run on it.

  • brandonmooreis (1/16/2011)


    This legacy product is not going to be changing anymore. But not all legacy customers will upgrade immediately or at all,

    ... <other information>

    You've got other issues then if you're dealing with a deployable vendor design.

    A) You need an importer finalized from and to each deployable version of the schema.

    B) You need to have a mid-step process as you version so that some poor soul on 2.0 who wants to go to 6.0 can, and you don't have to have converters to deal with data from the beginning of time. (Think 2.0 - 3.0 - 4.0 - 5.0 - 6.0 - 6.32). You need to keep a stable data path up.

    C) I personally would keep the importer up to date in this current state, and keep it versioned as you create client deployable versions.


    - Craig Farrell

    Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.

    For better assistance in answering your questions[/url] | Forum Netiquette
    For index/tuning help, follow these directions.[/url] |Tally Tables[/url]

    Twitter: @AnyWayDBA

  • Craig, I don't think you understand...

    All that's needed to get someone already on the system to the current version (as far as the db is concerned) is to just run the sql scripts. No importer is necessary for that. And because of this there's no need to keep an importer for legacy users up to date with our current version... it just needs to import them into the 1st version and then we run the scripts to get them up to date.

    I see no advantage to reproducing what we do in the scripts in the legacy import program. It would just be extra work for no good reason.

Viewing 6 posts - 1 through 5 (of 5 total)

You must be logged in to reply to this topic. Login to reply