July 7, 2016 at 10:08 pm
Comments posted to this topic are about the item The Need for Database Intelligence Software
July 8, 2016 at 2:26 am
Decent idea, but I have to raise an eyebrow at the practical side - given how so far most ideas to intelligently even identify what you might be writing in Word fail so miserably. I think I'd probably stick to lovingly crafting every bit until I could be convinced that the AI would cotton on correctly.
July 8, 2016 at 3:32 am
Too bad a bunch of developers assume this intelligence is already present in today's database engines, using code first to persist their objects without further considerations about consistency and performance. It is exactly that attitude towards data that requires this type of intelligence. Data is no longer seen as a separate asset that should be carefully crafted and maintained but rather as some byproduct of applications that unnecessarily burdens the developer.
An engine can only infer data types based on the set of values present in a column, but cannot force input validation. Hence date columns might contain values like 'tomorrow' or 'unknown' and the applications will not contain a mechanism to deal with data value errors because the developer didn't bother in the first place. Let alone primary and foreign keys. Although a combination of column values might uniquely identifying each 'object' today, you can never be sure about tomorrow. He'll just add an [ID] column to each row and everything just works, no questions asked. Relations place an extra responsibility on the developer to add and remove 'objects' in a specific order. Adding relational constraints after the fact will almost certainly lead to unexpected foreign key violations in an application and the developer will justly blame the database.
Of course data is still a valuable asset, but now you need a data analyst to be able to distill some useful information from your application databases. I assume a 'smart' engine could make this task easier allowing some Excel-savvy employee to create reports without consulting a data analyst to prepare his data. It fits well in the central role of today's application developer delivering solutions as fast as possible without too much consideration on future developments. Don't get me wrong, I am a developer too, sometimes this approach seems to be the only way to deal with expectations of customers and managers.
Maybe in the near future we will no longer place indexes on our tables, because the engine will take care of that based on some workload analysis. That would fit well in the declarative philosophy of SQL leaving it up to the engine to determine the best way to gather the results for your queries. Too many application databases are still 'just a bunch of tables' without any constraints assuming the application will ensure the integrity of the stored data. If you ever have build reports on these databases you know that every assumption about validity will fail soon or later and they will blame you for their incorrect results anyway. An engine might be able to infer some additional metadata from the available data but will suffer from the same glitches described earlier. But these are just my two cents (maybe three) ...
July 8, 2016 at 5:29 am
And woe to the professional database developer who tries to protect data with things like constraints and well-honed coding practices only to be dismissed as a waste of perfectly good oxygen by the next code-first EF practitioner.
If such an AI as described would also provide the only consumable data access layer rather than allowing randomly generated T/SQL being sent from application land it might have a chance of improving the data experience and protecting the data asset. After all, without data an application is simply a bag of bits with nothing to do.
------------
Buy the ticket, take the ride. -- Hunter S. Thompson
July 8, 2016 at 6:03 am
Alerting the DBA to newly minted but inappropriate objects would be preferable to a system that overrides the humans. Anyone who has had a tool automatically launch a re-org in the middle of heavy business activity has been cured of auto-management of their databases. Yes, these things can be tuned to operate within acceptable parameters, but the carnage on the way may put some enterprises out of business.
The other side of my mouth replies: Yes, but for companies that cannot afford a full IT staff (or in particular, a DBA), such a system - with built-in intelligence would be perfect, or near perfect.
Bottom line--system, tell me what is needed. I will decide, when/how/or even if an action is warranted.
Orthodox cowards make good DBAs. (It is fun to take a little risk, but control is our life's blood.)
July 8, 2016 at 6:12 am
To paraphrase what is said about fusion energy, this seems in the future, and always will be. Given that wizards for basic things aren't quite as good as humans, I can't ever see this coming.
Lots of good comments by the previous posters. A better way to describe DBAs as "orthodox cowards" should be found, however. It is anything but cowardly to stand up for the correct solution.
July 8, 2016 at 8:54 am
I think this a pretty fantastic vision, really. It's easy to get caught up in the practical challenges of what it would take to even take a meaningful step in that direction, but it's a great concept.
I can't help but think of when cars were first built, the idea of anti-lock brakes was not even technologically feasible. When the technology began to make it possible, people reared back, saying "I know when to let off the brakes when I start sliding! As a driver of a car, that's part of the knowledge that I must posess!" But eventually, the technology made it's way into every car out there, and now it just happens automatically, and works well, whether or not you're a "good" driver.
The same could be said of many features in a car - the advances over the ages have made is so our cars make a great deal of small decisions for us to aid an easier, safer ride, in a vehicle that requires less maintenance and last longer. We're even beginning to enter the foray of driverless cars, because ultimately cars were not invented to be driven, but to provide transportation, and the driver was a necessity of the time.
But also every other technology out there. There will be challenges, yes. There will be bumps in the road, yes. It will take time, yes. But this is a fantastic vision, and it takes this kind of vision in the face of it being so impractical in the present, to drive the technology (and the culture surrounding it) forward.
I like it.
July 8, 2016 at 9:09 am
I consider database design and tuning to be an art, not a science. As such, creating software to do these things would be rather difficult.
I wish it was a reality! Great idea. When do we start designing it???
Michael L John
If you assassinate a DBA, would you pull a trigger?
To properly post on a forum:
http://www.sqlservercentral.com/articles/61537/
July 8, 2016 at 9:12 am
For every far off idea that now seems a reality, there are many that remain a far off idea. When doing historical research for my master's thesis, I remember thumbing through military magazines from the 1950's that envisioned that by the year 2000 each infantry soldier would have their own flying platform. We're years beyond and have nothing remotely like that. But we have things that no one at the time was thinking about.
You just have to think how often you use the built in wizards for something once you've gotten any kind of good with the tool. And this vision is far more complicated than what current wizards are designed for.
Not dissing the idea. But we should recognize the limits of what automation can do. If people can't agree on these things, do we really expect a computer will always have the "right" answer.
July 8, 2016 at 1:17 pm
RonKyle (7/8/2016)
To paraphrase what is said about fusion energy, this seems in the future, and always will be. Given that wizards for basic things aren't quite as good as humans, I can't ever see this coming.
Did anyone of you ever stumbled upon those fancy recommended indexes from the Database Engine Tuning Advisor? You will recognize their obscure names immediately. I happen to see them more often lately, mostly on databases I was called upon because a developer cannot solve a performance problem. I do not know how many cases pass by unnoticed, when it did increase the performance of certain queries just enough to keep me out of the way.
Just to warn you: Whenever some wizard sounds like a solution to a database problem, some developers will use it even if it does not fulfill its promise. And don't you dare to tell them what is wrong with all this litter and clutter when they are asking you for help. Just fix that performance problem, they will take care of the rest. Hopefully most developers do not fall in this category, but if you have ever met one, you know exactly what I am talking about.
July 8, 2016 at 1:55 pm
The Need for Developers Who Are Literate About Relational Database Design
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
July 15, 2016 at 9:53 pm
liver.larson (7/8/2016)
I think this a pretty fantastic vision, really. It's easy to get caught up in the practical challenges of what it would take to even take a meaningful step in that direction, but it's a great concept.
"Fantastic" strikes me as the right word! I don't believe that it's feasible to have database intelligence software that can decide what constraints need to exist without first being told about the business logic that determines data integrity. The creator of the database will have to have some means of describing that logic to the AI, and that will not be much removed from specifying constraints. Similarly, type information deduced from an early set of data requires that the early set is genuinely big enough to ensure that the chosen type is correct, and generating such a dataset is probably only possible by first understanding what type of data will in in what column in each table, and since the creator needs to have the type information to generate the data set he might as well just tell the software which types to use.
I can't help but think of when cars were first built, the idea of anti-lock brakes was not even technologically feasible. When the technology began to make it possible, people reared back, saying "I know when to let off the brakes when I start sliding! As a driver of a car, that's part of the knowledge that I must posess!" But eventually, the technology made it's way into every car out there, and now it just happens automatically, and works well, whether or not you're a "good" driver.
I don't think that's a good analogy for schema design; some things can be automated - index creation and deletion, which statistics should be generated how often and from what sort of data sample, recovery plan creation and validation that restore is possible re all things that can probably be automated (but the system has to be told what risk levels are allowable for which data and what performance levels are required for the various workloads so again it can't just happen by magic) - but feeding in the information required for chema design is non-trivial, even if one does it on the level of describing a single supertable with column type descriptions and a set of functional dependencies, a set of multivalued dependencies, and a set of join dependencies so that the software can convert the supertable into a set of tables and constraints conforming to 6NF (or should it only go to 5NF? or 4NF? or EKNF?) which is probably the only way (short of specifying all the tables and all the constraints) to describe the required business logic that determines data integrity.
....
....
.... There will be challenges, yes. There will be bumps in the road, yes. It will take time, yes. But this is a fantastic vision, and it takes this kind of vision in the face of it being so impractical in the present, to drive the technology (and the culture surrounding it) forward.
I like it.
I think it's overambitious, too close to a sentient AI concept; but a lot of it will happn, although human input to tell this software what to do will continue to be required until we have genuinely sentient AIs (and that may never happen).
Tom
July 15, 2016 at 9:58 pm
Eric M Russell (7/8/2016)
The Need for Developers Who Are Literate About Relational Database Design
We also have a Need for DBAs Who Are Literate About Relational Database Design.
Are developers really any worse? 🙂
Tom
July 15, 2016 at 11:11 pm
TomThomson (7/15/2016)
Eric M Russell (7/8/2016)
The Need for Developers Who Are Literate About Relational Database DesignWe also have a Need for DBAs Who Are Literate About Relational Database Design.
Are developers really any worse? 🙂
Oh yeah...
--Jeff Moden
Change is inevitable... Change for the better is not.
November 8, 2016 at 4:03 am
Is the problem an issue with defaults? Too often defaults are accepted which allow for working (but poorly implemented) systems. Remove the defaults and then people have to engage their brains.
This is the same concern that I have with dynamic languages (interpretted, parsed, etc.).
In the past too many issues have been caused by inferred types. In dynamic languages you find the issue pretty quickly. With databases less so. Also the problems might just* be performance.
*This is a hidden issue and, potentially, can be a more difficult issue to resolve in a production database.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 15 posts - 1 through 15 (of 15 total)
You must be logged in to reply to this topic. Login to reply