April 15, 2019 at 12:00 am
Comments posted to this topic are about the item Building a Database Engine
April 15, 2019 at 1:38 pm
Writing a DB engine would be a daunting task indeed -- but at least one would have a SQL STANDARD on which to follow. One of my biggest regrets with regard to the (programming) industry is mentioned in the following article:
https://theuknownprogrammer.blogspot.com/2017/07/40-years-of-programming.html
I wish we had a STANDARD for programming languages -- issues such as the declaration of variables, assignment operators, comparison operators and loops -- the author of the above article is correct -- why do we need so many different ways to accomplish the same task? Having a universal programming SYNTAX could have put us light-years ahead.
Just my opinion -- I suspect now we'll have an endless off-topic debate about why and how one language is so much better than another (even though they produce the same end result).
April 15, 2019 at 2:36 pm
My first, real programming job was in the private sector; manufacturing. This was before the company adopted a SQL-based RDBMS. We worked on VAXes, where the OS was VMS (if anyone still knows about VMS, I'd like to know). Back then I wrote a whole system which used a technology VMS had called ISAM files. (It's been long enough ago that I no longer remembered the name of the technology; had to look it up.) I wrote it in C for VMS. Wrote the CRUD operations; but not to the extent that person did who wrote that series of articles, trying to replicate SQLite. Still, it was a lot of work and at the time I enjoyed doing it.
But would I want to do it again? Absolutely not.
Kindest Regards, Rod Connect with me on LinkedIn.
April 15, 2019 at 3:39 pm
Think of all the millions of man hours that have gone into SQL Server, Oracle, PostGres, MySQL, Teradata, Neteeza etc. Imagine waking up one morning and thinking that you could do better!
Either you have a very specific use case which doesn't require 99.9% of what one of the main protagonists need or you are Michael Stonebraker.
I've a theory that a DB has to get beyond version 8 before it can be said to be mature. SQL Server 9.0 was SQL2005. That was a huge step change. I've seen blog posts saying that Oracle wasn't that great before V8.0.
April 15, 2019 at 4:03 pm
Writing a db engine is hard, and I think it's an interesting exercise just because it shows you that. When I tried some mySQL in 2002/3/4, it felt way behind SQL Server. Didn't feel like something I'd even want to work with or use in production. I think SQL felt that way in 4.2 and a little in 6.5, but 2000 (7 was a a point release for me), was pretty good. 2005 was certainly a
I do think it takes time to mature software, but there are also endless ways people want to work with code. Lots of things in C, or Fortran, or Cobol were confusing and hard to understand. There are certainly good arguments about why declaring or not declaring variables matters. I don't know that we should have a standard way, as that inhibits research and experimentation into new languages. That being said, I do wish that the designers of languages would early on be more consistent.
April 15, 2019 at 6:03 pm
I am retired but back in the nineties I had designed and implemented a junction box to contain cross references between all kinds of standards documents and military specs (citations to references and vice versa using short cryptic structured keys, not the actual lengthy published document numbers). It was done with Ingres on DEC Alpha machines. I wrote a script to recalculate/update the junction box but it took FOREVER to run (it ran for weeks and never completed). I tried and tried and consulted with several people to improve the Query Plan but apparently the process was much too complex for the Ingres to digest and handle.
Finally I proposed to my manager that this process be taken out of Ingres and updated using C programs on flat files and sort utilities. It ran in mere minutes on 800K documents with maybe 4M or more references. It was so amazing that we decided to also offload several other DB tables (different kinds not related to xrefs) to run through their own flat files and sorting processes. Then the resulting flat files could be bulk loaded back into respective DB tables. It was a deep relief to be able to do "end of issue" processing (every 60 days) on a very brief expeditious tight schedule.
April 16, 2019 at 1:35 pm
At least in my observable universe, most limitations that developers struggle with daily are at the logical level of SQL coding or at the physical modeling level, rather than being a fundamental limitation of the database engine itself. Major database platforms like SQL Server or Oracle will do what you want and how you want it, if you just take time to familiarize yourself with all the features and follow best practices. Instead of just migrating a database from one release and hardware environment to another, think about refactoring.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
April 16, 2019 at 2:44 pm
Rod, I did work many years ago in C on the DEC VMS. We used a database called S1032. The company that owned s1032 sold it to CompuServe as they were heavily invested in we. We used DEC FORMS for the user interface. It was surprisingly fast. The one thing about DEC was all their tools worked very well together. I have since returned to work for the same company and we still have the 30 feet of cabinets that stored all the DEC manuals. No internet to look up answers in those days. Dif thru the manual and figure it out!
April 16, 2019 at 3:00 pm
With 30 feet of printed manuals, I'm guessing a moderate sized IT department had something like a technical reference librarian who could research stuff and keep it organized.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
April 16, 2019 at 7:40 pm
In order to practice writing some code, it's not just the documentation (or lack thereof) that can be a challenge, but also the sheer number of layers you need to master to get anything done. And each one can change overnight and break things.
http://www.commitstrip.com/en/2016/05/10/a-moment-of-nostalgia/
April 17, 2019 at 3:21 pm
There have been so many new database engines introduced over the past decade, I don't see the need for a new one. When building out an enterprise scope solution, the database administrators need to choose the right engine for the task at hand rather than relying on one engine for all needs. ETL bulk staging, transactional processing, micro-services, aggregate reporting, documents and images, embedded devices: these are all very different data access patterns, and sometimes you need to think outside the SQL Server box.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
Viewing 11 posts - 1 through 10 (of 10 total)
You must be logged in to reply to this topic. Login to reply