September 16, 2015 at 1:18 pm
And 15 year old technology.
A Saturn V rocket is 40+ year old technology. Can it still be used to get to earth orbit and provide the astronauts with the vehicles to get to the moon and back? Yes.
In fact, because NASA has been screwing around trying to reinvent the wheel, we have not gone back to the moon. since the 70's. NASA wants to build a bigger rocket to take everything they need to the moon to setup a permanent moon presence.
There are a lot of strategies to go to the moon and back which they could have used the Saturn V rocket for.
We did not get to see these other strategies played out, other than Lunar Orbit Rendevous (LOR).
All I am doing is using Saturn V Technology (e.g. MS-Jet Engine), but using a new strategy never tried, nor communicated to consider.
September 16, 2015 at 1:24 pm
erichansen1836 (9/16/2015)
Addicted -Although I do wonder how your solution could be EASALY Used out of the Box to allow your database to be collocated on instances around the world with a Transaction in instance Peru being to instance Sweden? Does it have built in Monitoring and optimization?
You have no idea what you are talking about Addicted because you have not been actively following the Thread. You act as if this is a turn-key solution to roll off the retail store shelves.
I never presented this solution this way.
All I have shown is how it can be done, because it is a method not brought to the attention of the world before.
I have used MS-Jet Engine/ODBC enabled databases since Access 97 came out, both on Windows Peer-to-Peer and Windows Server Networks, but only until 2014 did it ever occur to me that a huge database could be maintained using *.MDB files as partial tables. And I discovered on my own that the tools needed to build this database already come factory installed on Windows 7 Home Premium (i.e. ODBC Admin Utility, MS-Access ODBC Driver), and JetComp.exe compact/repair utility is available for free download at Microsoft, and it is not hard to find a Free programming language with ODBC functionality (such as ActiveState Win32 Perl and Dave Roth's Win32::ODBC module and Aldo Calpini's Win32::GUI Native Windows GUI widget tool chest).
NOTE: If you don't like this idea, I have a second one to offer for Perl users.
Use Fixed-Length record TEXT files as I have demonstrated for *.MDB files, but use persistent Perl SDBM database files (tied to program hash tables) for immediate random access indexing to fixed-length record offsets(in bytes). These indexes can easily be rebuilt from the data in the Text Files and dont even need to be backed up . IMPORTANTLY, this is perhaps a true "NO SQL" system, and you don't need Data Access Components (DAC). When a Perl application user-interface to this database is compiled to EXE, it is a completely standalone system, and easily installed to client PCs. This is also a FREE/NO COST database access system for end-users (which you can develop for them), and to repeat, does not require Perl runtime or development components be installed on client PCs or the Server. And yes, this can be a relational database.
And just how much time would it take to build the functionality needed to safe guard the data, ensure database performance, meet the stake holders requirements for RPO/RTO? Much of that is already built into SQL Server/Oracle/PostgreSQL/MySQL/name your commercial/FOS RDBMS of choice? Not everyone wants or needs to be proficient in writing these types of routines. If a person built such a system, who would take over maintaining it if the original developer is hit by (or purposely walks in front of) the proverbial bus? How many contractors do you know personally that could take over for you if it happened to you?
These are questions that I have asked (and asked) that you have ignored with the deftness of a seasoned politician. These are questions I would ask you as a stake holder building me an application. Do you avoid these questions with your clients?
September 16, 2015 at 1:28 pm
Wayne West (9/15/2015)
To be brutally honest, it sounds like you've done a heck of a job engineering job security for yourself at the expense of anyone else easily maintaining the system. I won't argue that it might be a fine solution for certain purposes, but of all the companies that I've worked for over the years can I think of a single project that I've been involved in or known about where it would be a good solution.
I remember one app in Access that a fellow carted around on a laptop, and while I'm not certain about Access being able to "shard" across multiple mdb's, it would still be a neat trick if that app could break through the filesize limit using the split database file trick.
I routinely have to query across (and union up) multiple tables, so its not like what he's doing is altogether foreign to me. For that matter, a decent migration path would be to start extracting from his current system to a server styled SQL system for some reporting purposes and maybe gear up to move portions to that. Heck, since he's already got a PERL code base, he's already invested in maintaining the application logic there, so another database server (and the experience in coding to it) in the mix could be a way to hedge against the unknown future supporability of Microsoft's jet.
Still, I'd probably rather use one of the free offerings to beat that 10gb express limit. I was interested in the OP's decision making why he didn't pick one of the free offerings that exist outside the MSSQL world that would have led to a more classic looking implementation, but I suspect this thread is getting a bit too noisy and bad mannered to allow me to get an answer 🙂
September 16, 2015 at 1:32 pm
Jeff Moden (9/15/2015)
J Livingston SQL (9/15/2015)
erichansen1836 (9/10/2015)
If you implement a Network database file system of 500 MDB files, each acting as a partial table within the 5 billion row database, each MDB file containing 10 million rows...
excuse me for being thick, but lets say for instance that I have a 50 million row table of sales transactions....do I assume that in your world of JET this would equate to possibly 5 separate MDB files?
if this assumption is correct, can you please explain how you query all 5 MDBs at once.....for example I need to retrieve all sales data for a specific customer....the actual transactions could be in any of the five MDBs?
I have no knowledge of PERL, so if this is a stupid question then please reply accordingly 🙂
And what code is it that will determine the partitioning of those files and decide when a file is "full" and create another one and add that to the partitioning scheme, whatever that may be?
use the stat function. D'OH.
September 16, 2015 at 1:38 pm
Still, I'd probably rather use one of the free offerings to beat that 10gb express limit. I was interested in the OP's decision making why he didn't pick one of the free offerings that exist outside the MSSQL world that would have led to a more classic looking implementation, but I suspect this thread is getting a bit too noisy and bad mannered to allow me to get an answer Smile
I have: use Fixed-Length record TEXT files as I have demonstrated for *.MDB files, and use persistent Perl SDBM database files tied to Perl program hash tables, for immediate random access indexing to fixed-length TEXT file record offsets (in bytes). No DAC components needed. Compiled Perl EXE application files are all that is needed istalled to the client PCs, and no runtime etc needed on the Server side either.
The problem with fixed-length TEXT file databases has always been their inability to be used in a relational database system with random access indexing to records. This fixes that problem.
This would be a TRUE, NO SQL system, because regular expressions are used for sequential processing of records, and tied hash table key/value pair lookups are used instead of SQL for random access.
September 16, 2015 at 1:40 pm
erichansen1836 (9/16/2015)
And 15 year old technology.
A Saturn V rocket is 40+ year old technology. Can it still be used to get to earth orbit and provide the astronauts with the vehicles to get to the moon and back? Yes.
In fact, because NASA has been screwing around trying to reinvent the wheel, we have not gone back to the moon. since the 70's. NASA wants to build a bigger rocket to take everything they need to the moon to setup a permanent moon presence.
There are a lot of strategies to go to the moon and back which they could have used the Saturn V rocket for.
We did not get to see these other strategies played out, other than Lunar Orbit Rendevous (LOR).
All I am doing is using Saturn V Technology (e.g. MS-Jet Engine), but using a new strategy never tried, nor communicated to consider.
Actually, from what I have read (several sources that I have no record of or can currently find), NASA does not have the technical knowledge to launch its last Saturn V rocket.
Of course, I may be wrong on this and what I read was a load of crap.
September 16, 2015 at 1:42 pm
erichansen1836 (9/16/2015)
Addicted -Although I do wonder how your solution could be EASALY Used out of the Box to allow your database to be collocated on instances around the world with a Transaction in instance Peru being to instance Sweden? Does it have built in Monitoring and optimization?
You have no idea what you are talking about Addicted because you have not been actively following the Thread. You act as if this is a turn-key solution to roll off the retail store shelves.
I never presented this solution this way.
All I have shown is how it can be done, because it is a method not brought to the attention of the world before.
I have used MS-Jet Engine/ODBC enabled databases since Access 97 came out, both on Windows Peer-to-Peer and Windows Server Networks, but only until 2014 did it ever occur to me that a huge database could be maintained using *.MDB files as partial tables. And I discovered on my own that the tools needed to build this database already come factory installed on Windows 7 Home Premium (i.e. ODBC Admin Utility, MS-Access ODBC Driver), and JetComp.exe compact/repair utility is available for free download at Microsoft, and it is not hard to find a Free programming language with ODBC functionality (such as ActiveState Win32 Perl and Dave Roth's Win32::ODBC module and Aldo Calpini's Win32::GUI Native Windows GUI widget tool chest).
NOTE: If you don't like this idea, I have a second one to offer for Perl users.
Use Fixed-Length record TEXT files as I have demonstrated for *.MDB files, but use persistent Perl SDBM database files (tied to program hash tables) for immediate random access indexing to fixed-length record offsets(in bytes). These indexes can easily be rebuilt from the data in the Text Files and dont even need to be backed up . IMPORTANTLY, this is perhaps a true "NO SQL" system, and you don't need Data Access Components (DAC). When a Perl application user-interface to this database is compiled to EXE, it is a completely standalone system, and easily installed to client PCs. This is also a FREE/NO COST database access system for end-users (which you can develop for them), and to repeat, does not require Perl runtime or development components be installed on client PCs or the Server. And yes, this can be a relational database.
I call this Joint Database Technology, taking two relatively weak database systems, but using them together to create a relatively strong database system.
I call MS-Jet Engine/ODBC databases Reduction Database Technology, because the MS-Access software is eliminated from the equation, freeing Jet Engine to be all it can be.
And you base this comment (shown below) on what, that until now he has remained silent?
You have no idea what you are talking about Addicted because you have not been actively following the Thread.
September 16, 2015 at 1:44 pm
erichansen1836 (9/16/2015)
Still, I'd probably rather use one of the free offerings to beat that 10gb express limit. I was interested in the OP's decision making why he didn't pick one of the free offerings that exist outside the MSSQL world that would have led to a more classic looking implementation, but I suspect this thread is getting a bit too noisy and bad mannered to allow me to get an answer Smile
I have: use Fixed-Length record TEXT files as I have demonstrated for *.MDB files, and use persistent Perl SDBM database files tied to Perl program hash tables, for immediate random access indexing to fixed-length TEXT file record offsets (in bytes). No DAC components needed. Compiled Perl EXE application files are all that is needed istalled to the client PCs, and no runtime etc needed on the Server side either.
The problem with fixed-length TEXT file databases has always been their inability to be used in a relational database system with random access indexing to records. This fixes that problem.
Well my choice would be Postgresql, was that not a good choice when you started this project? Have you considered it recently? I don't have much experience with it, and none really on windows, although I suspect it can't be that bad, heck, they have 5 (two more unsupported) of the past releases available for windows.
September 16, 2015 at 1:49 pm
Actually, from what I have read (several sources that I have no record of or can currently find), NASA does not have the technical knowledge to launch its last Saturn V rocket.
Of course, I may be wrong on this and what I read was a load of crap.
Sounds about right Lynn.
And the United States government does not retain the knowledge of past wars (Florida Seminole War) in order to avoid the same mistakes which were made in Vietnam and Iraq.
One problem I see with IT world is that the past is not appreciated.
For example, in the early 1980's I was taking a Fortran class in college and we had to use punch cards and wait our turn in line to run our jobs on the computer. Our instructor told us to be sure our programs were perfect initially to avoid having to wait in line after corrections were made. Now days, computer programs seem to be written by constant feedback (regarding errors) from local QA people to overseas contractors writing the code until all errors are eliminated. Now how is that going to produce efficient code?
September 16, 2015 at 1:53 pm
Well my choice would be Postgresql, was that not a good choice when you started this project? Have you considered it recently?
I have never investigated Postgresql, although the name I have seen couple times.
I check into it.
September 16, 2015 at 1:57 pm
patrickmcginnis59 10839 (9/16/2015)
erichansen1836 (9/16/2015)
Still, I'd probably rather use one of the free offerings to beat that 10gb express limit. I was interested in the OP's decision making why he didn't pick one of the free offerings that exist outside the MSSQL world that would have led to a more classic looking implementation, but I suspect this thread is getting a bit too noisy and bad mannered to allow me to get an answer Smile
I have: use Fixed-Length record TEXT files as I have demonstrated for *.MDB files, and use persistent Perl SDBM database files tied to Perl program hash tables, for immediate random access indexing to fixed-length TEXT file record offsets (in bytes). No DAC components needed. Compiled Perl EXE application files are all that is needed istalled to the client PCs, and no runtime etc needed on the Server side either.
The problem with fixed-length TEXT file databases has always been their inability to be used in a relational database system with random access indexing to records. This fixes that problem.
Well my choice would be Postgresql, was that not a good choice when you started this project? Have you considered it recently? I don't have much experience with it, and none really on windows, although I suspect it can't be that bad, heck, they have 5 (two more unsupported) of the past releases available for windows.
Fortan code? Punch cards?
This thread is starting to sound like the movie Field of Dreams when Kevin Costner first tries to get James Earl Jones to come with him.
Jones starts spraying him with bug spray whilst shouting "Go back to the 60's!"
Please, go back to the 60's with this solution, and take Celko with you!
Michael L John
If you assassinate a DBA, would you pull a trigger?
To properly post on a forum:
http://www.sqlservercentral.com/articles/61537/
September 16, 2015 at 2:04 pm
The nice thing about Joint Database Technology is that it is portable to any O/S platform.
The user-interface could be portable too if you used Perl TK module instead of Perl Win32::GUI module.
And on UNIX systems, you are not limited to 4 GIG per each fixed-length TEXT file.
On UNIX systems you can have huge text files much larger than 4 GIG, although there is a limit on Perl SDBM files of 2 GIG using them as indexes to text file records.
For 4 GIG text files on Windows O/S, my SDBM database files never got larger than 1 GIG, so there is room for growth. Perhaps on UNIX you could take the text files to 8 GIG and the corresponding SDBM index file to 2 GIG.
For Perl SDBM index files, the key/value pair is such that:
the Key is one or more fields/columns in the text file records, and
the Value is the record offset in bytes where you randomly SEEK to in your Perl code.
The Key can be made up of partial fields/columns too.
EXample KEY: LastName(first 4 chars)|FirstName(first 4 chars)|ZipCode|etc.
September 16, 2015 at 2:09 pm
Michael L John (9/16/2015)
patrickmcginnis59 10839 (9/16/2015)
erichansen1836 (9/16/2015)
Still, I'd probably rather use one of the free offerings to beat that 10gb express limit. I was interested in the OP's decision making why he didn't pick one of the free offerings that exist outside the MSSQL world that would have led to a more classic looking implementation, but I suspect this thread is getting a bit too noisy and bad mannered to allow me to get an answer Smile
I have: use Fixed-Length record TEXT files as I have demonstrated for *.MDB files, and use persistent Perl SDBM database files tied to Perl program hash tables, for immediate random access indexing to fixed-length TEXT file record offsets (in bytes). No DAC components needed. Compiled Perl EXE application files are all that is needed istalled to the client PCs, and no runtime etc needed on the Server side either.
The problem with fixed-length TEXT file databases has always been their inability to be used in a relational database system with random access indexing to records. This fixes that problem.
Well my choice would be Postgresql, was that not a good choice when you started this project? Have you considered it recently? I don't have much experience with it, and none really on windows, although I suspect it can't be that bad, heck, they have 5 (two more unsupported) of the past releases available for windows.
Fortan code? Punch cards?
This thread is starting to sound like the movie Field of Dreams when Kevin Costner first tries to get James Earl Jones to come with him.
Jones starts spraying him with bug spray whilst shouting "Go back to the 60's!"
Please, go back to the 60's with this solution, and take Celko with you!
I remember punch cards, also paper tape. While in High School I loved taking the chads to the High School football games and using them like confetti. Others around us didn't like it, hard to get of their hair.
September 16, 2015 at 2:11 pm
And I also remember using Telex systems to send messages instead of email.
You had to type a message that the machine punched onto a narrow yellow paper tape.
Then you loaded that tape and called a phone number, and when connected, you sent the tape message
and it printed out at the destination on a typewriter.
September 16, 2015 at 2:12 pm
erichansen1836 (9/16/2015)
The nice thing about Joint Database Technology is that it is portable to any O/S platform.The user-interface could be portable too if you used Perl TK module instead of Perl Win32::GUI module.
And on UNIX systems, you are not limited to 4 GIG per each fixed-length TEXT file.
On UNIX systems you can have huge text files much larger than 4 GIG, although there is a limit on Perl SDBM files of 2 GIG using them as indexes to text file records.
For 4 GIG text files on Windows O/S, my SDBM database files never got larger than 1 GIG, so there is room for growth. Perhaps on UNIX you could take the text files to 8 GIG and the corresponding SDBM index file to 2 GIG.
For Perl SDBM index files, the key/value pair is such that:
the Key is one or more fields/columns in the text file records, and
the Value is the record offset in bytes where you randomly SEEK to in your Perl code.
The Key can be made up of partial fields/columns too.
EXample KEY: LastName(first 4 chars)|FirstName(first 4 chars)|ZipCode|etc.
Picking and choosing what questions or comments you will respond? I asked several important questions. Your failure to answer them if I was looking to you to provide me with a turn-key solution would have me walking you out the door, contractor or FTE. Care to go back and answer my questions?
Viewing 15 posts - 76 through 90 (of 245 total)
You must be logged in to reply to this topic. Login to reply