October 13, 2010 at 10:01 am
Really? Do you have any stats/research that backs up that assertion? I'm not an expert on what developers do but I have extensive experience as a DBA and managing groups of DBA's and I have to say that we spent very little time re-working our modeling techniques. We had standard tools and techniques. We used them. Once in a blue moon we re-examined our methods agains the latest and greatest tools and techniques and adjusted as necessary.
I'm not sure what 'reinventing database access' means. We had some utility programs that generated CRUD code for tables and/or views if we needed it, but that was a very small portion of the work done by the DBA staff.
We did have standards about how the DBMS was used. We had those standards because often we had little control over how the development groups wrote their applications. We opted to put the business logic near the data to help ensure consistency across the application portfolio. It worked.
So, from my perspective, you're solving a problem that may not really exist on the database side.
RAP does not really address the problems of DBAs, whose primary job is to maintain existing applications. RAP is about new application development, where someone has to decide exactly how to do all the things that the article lists. I assure you, from many many years of application development experience, that these are the things that consume most of new application developers' time.
October 13, 2010 at 10:36 am
RAP does not really address the problems of DBAs, whose primary job is to maintain existing applications. RAP is about new application development, where someone has to decide exactly how to do all the things that the article lists. I assure you, from many many years of application development experience, that these are the things that consume most of new application developers' time.
I guess that's where the disconnect comes from. My experience has mostly been in situations where the DBA team, or the data architecture team, did the data models for new applications. In most of the shops I've been in, DBAs didn't just keep things running, they were active participants in the application development process. They were involved from the very beginning, so we didn't see a lot of the thrashing around trying to figure out the tools or the modeling paradigm or the access pattern.
I think you're making a large assumption about the role of a DBA that may not be universally true. In organizations where the DBAs just keep the lights on and don't have any real understanding of the data over which they watch, you have bigger problems than the choice of application development tools.
October 13, 2010 at 3:50 pm
It is not true to say that:
"Unfortunately in this day & age we still don't have relational databases that allow us to divide our various objects into different name spaces"
Sql Server has Schemas that can be used to place objects in! Tables and SP and Functions. Look at AdventureWorks for examples.
Oracle (swearing tis tis) has tablespaces and packages,but the effect is the same.
So it is preferred that prefixes aren't used and schemas are.
But this article doesn't take other aspects of architecture into account.
For instance:
- data modeling with Entity Framework or nHibernate
- error and exception handling logging and reporting
- message management (error and non-error)
And these are relevant for a small to medium size application
Then what if the application is successful and needs to scale up and be extended.
can it:
use services as a layer of Interfacing
be seperated into different teirs that work in a application farm with a load balancer
call other services without too much plumping
use other tools or techiniques such sockets communication, multiple level threading, multiple UI eg ASP.net, Silverlight
All these need to be considered when considering architecture for a even a small application, else you are planning NOT to be successful or locking the applicaiton out of being extendable or scalable
regards
GregJF
PS
Ford now make all kinds of cars. If Henry was still alive, we would be all driving black cars that look alike.
Or Ford would be bankrupted long ago
Honda succeeds because it is innovative and changes its achitecture often, Honing, honing,honing!:-)
October 13, 2010 at 4:10 pm
David
I think your approach is Ill convieved and out dated.
Today we write applications for the Enterprise or for the Sector(eg small retail).
All the "Aspects", "Prespectives" and "Viewports" of software architecture should be taken into account when design such applications.
The small retailer stills need auditing, although not as much, as the big enterprise retailer.
A better architectural approach to build upon a framework like .Net or Java and contruct the "Aspects", "Prespectives" and "Viewports" into a enterprise framework, that would suit a Enterprise or a Sector of the economy.
Think BIG! Design to Grow
regards
GregJF
October 13, 2010 at 4:25 pm
GregFrazer (10/13/2010)
It is not true to say that:"Unfortunately in this day & age we still don't have relational databases that allow us to divide our various objects into different name spaces"
Sql Server has Schemas that can be used to place objects in! Tables and SP and Functions. Look at AdventureWorks for examples.
Oracle (swearing tis tis) has tablespaces and packages,but the effect is the same.
So it is preferred that prefixes aren't used and schemas are.
One part of the purpose of the RAP prefixes, as illustrated in an earlier blog, is to allow you to differentiate multiple objects that really should have the name except for their basic natures (stored procs versus functions, for example). That is not something we would generally want to split across schemas (procs in one, functions in another). Another purpose of the prefixes is to simply allow you to categorize your objects without having to put them in different schemas.
But this article doesn't take other aspects of architecture into account.
For instance:
- data modeling with Entity Framework or nHibernate
- error and exception handling logging and reporting
- message management (error and non-error)
And these are relevant for a small to medium size application
RAP has its own built-in ORM that is far simpler than Entity Framework and nHibernate. In fact a specific objective of RAP is to allow you to build smaller apps without needing to read a 900-page instruction manual or build huge non-type-checked XML files. Exception logging and reporting are Enterprise Library add-ons that live in their own schemas and can be added to a RAP application.
Then what if the application is successful and needs to scale up and be extended.
can it:
use services as a layer of Interfacing
be seperated into different teirs that work in a application farm with a load balancer
call other services without too much plumping
use other tools or techiniques such sockets communication, multiple level threading, multiple UI eg ASP.net, Silverlight
Like all new products developed on a limited budget, RAP does not claim to be everything to everyone. I don't know the extent to which it can do all these things. Its purpose is to demonstrate what can be done automatically with a radically higher degree of standardization. If I were required to meet all these requirements right out of the box on the first try, then certainly I should give up and go home immediately.
Honda succeeds because it is innovative and changes its achitecture often, Honing, honing,honing!
Like me, Honda also did not start out with a full product line. Honda's first product was piston rings. Honda succeeded, step by step, by applying Deming's theory of quality management, which focuses entirely on eliminating useless and counterproductive variation in manufacture. That's why you can drive a Honda 200,000 miles without repairing it, and that in turn is why Honda and its Japanese siblings have totally upended the American automobile marketplace. American companies are constantly revising and honing their product lines too. Unfortunately they just don't get the "elmination of variation" thing (because their corporate managements are more into sales than engineering) and so they are closing plants and dealerships while the Japanese and Koreans are taking over.
October 13, 2010 at 4:28 pm
GregFrazer (10/13/2010)
A better architectural approach to build upon a framework like .Net or Java and contruct the "Aspects", "Prespectives" and "Viewports" into a enterprise framework, that would suit a Enterprise or a Sector of the economy.
OK .. let's see your implementation that can be generically applied to all reasonable-sized database-based apps and generate most of the code for you! I'm ready to download it & try it out.
October 13, 2010 at 6:10 pm
David
I just would not attempt to build an application or framework that is
"generically applied to all reasonable-sized database-based apps and generate most of the code for you"!!
I, and many others, would build and do build frameworks for industy sectors and enterprises.
These frameworks have "best pratice" (very debatable term) princples and patterns applied so that developers can utilise them as TOOLS and still write code that meets business needs.
These can then become the standards for those enterprises and maybe even the industry sectors.
Affording newcomers, and old hands, in that enterprise the standards, the tools and the framework to codify the business logic and rules inside a scalable, extensibly, interfaced and may even service oriented frameworked solution.
Try reading more on private clouds (eg,Programming WCF, Juval Lowy, 3rd edition).
Your idea is good just not the bit about "generically applied" AND "generate most of the code".
Turn it into a framework without the code generators.
Code generators can cause more trouble than they are worth. eg when they contain a bug, when they need to be changed or extended.
I am sorry, but I think your approach is incorrect. No amount of code I post is going to change that.
There is no silver bullet that does all thing for all programmers.
Embedding yourself in good architecture that is based on prinplces, patterns and practices
( read "Design Patterns: Elements of Reusable Object-Oriented Software",Gof4; "Patterns of Enterprise Application Architecture",Martin Fowler; and even "Architecting Microsoft .NET Solutions for the Enterprise",Dino Esperato
OR
Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives - Nick Rozanski
)
regards
GregJF
October 13, 2010 at 10:30 pm
GregFrazer (10/13/2010)
Your idea is good just not the bit about "generically applied" AND "generate most of the code". Turn it into a framework without the code generators.
Greg: I am thinking that one of us is missing the other's point here because we seem to be talking past each other. RAP does not presume to implement any particular functionality any more than a complier presumes to. RAP's entire purpose is to create applications (or frameworks or whatever) in a highly consistent manner rather than in the handcrafted manner that is pervasive today. To argue against the use of RAP (and everything like it) is analogous to arguing against the use of compilers and claiming that we'd be better off building and using application-specific libraries written in assembly language.
RAP is intentionally application-agnostic like a compiler is intentionally application-agnostic. It has to do with systemetizing the plumbing of applications, not defining their functionality.
Whether you are writing a "framework" or an active library component (like Microsoft's logging or authentication libraries that have their own databases), or whether you are writing apps that make use of those things, somebody somewhere has to actually be designing schemas, coding databases, coding data layers, building business rules and writing user interfaces. We have a choice here: we can code those things by hand (the "whatever the team comes up with" standard) or we can code them mostly automatically with a degree of precision that no human team could possibly attain. RAP's purpose is to do that.
So RAP has nothing to do with subscribing to (or not subscribing to) the use of frameworks and libraries and patterns and practices, except of course that it is based on the .NET framework. You are perfectly free to use RAP to build database-based libraries using whatever patterns and practices you choose.
Code generators can cause more trouble than they are worth. eg when they contain a bug, when they need to be changed or extended.
You've lost me here completely. Every compiler is a code generator. Would you advocate that we abandon compilers because of the possibility of a bug? Like a compiler, RAP generates hundreds or thousands of completely formed database and data layer routines and objects whose consistency and reliability would be quite beyond the capability of any human team to maintain. A RAP-generated database will contain precisely zero coding errors (and in the unlikely event of a bug, it would be easy to fix the bug and then truly be at zero coding errors). In constrast, I have never seen any significantly-sized human-maintained database that I would estimate to be anywhere near that level of error freedom.
Embedding yourself in good architecture that is based on prinplces, patterns and practices
There is nothing about RAP that precludes the use of principles, patterns and practices. The upper tiers of RAP in fact implement a strict MVC model, so RAP has a built in "pattern". Like MVC itself, RAP's implementation of it is application agnostic and so it is applicable to essentially all apps. RAP also contains its own highly reliable ORM, which is a library of sorts (a ton of code that people don't have to rewrite every time) that does a far better job of database interaction than I would expect any human team I've ever worked with to be able to generate.
So I guess I'm missing the point. There is nothing about the use of RAP (just as there is nothing about the use of compilers) that would preclude anything you're talking about. All the things you're talking about have to be implemented using some technology and RAP presumes to improve that technology (much as compilers are regarded as an improvement over assemblers).
October 14, 2010 at 2:13 am
David
So I and my customers invest in RAP(if not money, then time and effort) and rely on your assurances.
We do this for months or years.
Then we need to build a Silverlight front end that is quite complicated and needs to use data graphs to relate data and then send that the database. In Entity Framework 4, that is called a self tracked entity.
Does RAP do that?
Then we get successful and people want to get data from out App.
In WCF we would build an interface and expose it as a service.
Does RAP do that?
Then we get busy and we need to scale our app (code) on to an application farm and govern traffic by load balancers
With WCF and stateless code and BigIP(for eg) we can and have!
Does RAP do that?
Then we wish to performance monitor our applicaition
Take it international with Globalization.
Does RAP do these?
OR do we have ot after many months refactor our code out of rap and build a non-RAP framework to get these things done.
OR do we have to wait until you put these things into RAP
BTW code generators that make other C#/VB.net are NOT the same as compliers. Compliers have syntax checking, type checking etc.
These C#/VB.net code generators make LOGICAL errors that are near impossible to debug.
And any major enhancements, like those above need MAJOR updates to the code Generator.
You certainly haven't conivnce me.
Also in the example you got us to load some SQL scripts.
Which created 222 Stored Procedures and 132 Functions
So your claim of no database programming is not evident.
Also what if we need to add a table or to add a column, does that need SP changes?
Do we need to go back to RAP to get another release?
This is asking more questins than it answers
Regards
GregJF
Regrads
October 14, 2010 at 9:32 am
I've been using https://sourceforge.net/projects/mygeneration/ for a while now. It's not identical to RAP but it's a code generator. The way I have designed my templates solves about 90% of the things you listed. My templates create a MVVM WPF/Silverlight application.
It takes a database schema that is well defined.
It creates:
* the stored procedures that application will need to access the data
* the the Model classes for each database entity
* * The model classes have built in validations that are derived from the schema such as null checks, length checks, or even constraints added using CHECK at the database level. This allows the application to tell you when something is wrong even before you have saved it. (What are the chances you wouldn't mess one of those up if you did that by hand ?)
* the DAO interfaces that the application will use to ask for data
* * a SQL DAO implementation that will be used by the WCF service
* * a WCF DAO implementation that the application can use to connect to my WCF service
* the WCF Service
* the management layer that tracks all the changes to my object and keeps everything in sync
* the various view models for managing each entity
* the XAML for the views (very generic)
* the thread manager to make sure all blocking calls run async so the application is always responsive.
I can generate about 90% of the code for my applications with a push of a button. Everything is consistent from one application to another. If you add a column to a table, or change a definition some where just regenerate everything and you are done. No having to spend hours going through thousands of lines of code updating everything.
I can spend core of my time worrying about the user interface and how the user interacts with my application instead of all the implementation details.
If a bug is found you just fix it in the template and regenerate again and you are done.
Overall it's a beautiful concept.
October 14, 2010 at 12:00 pm
GregFrazer (10/14/2010)
DavidYou certainly haven't conivnce me.
Greg: Truly and honestly, I am glad that there are some projects where relatively advanced needs like yours are being met, and where the programmers are so good and so consistent that they don't need code generators to help them out. I would say that RAP has little to offer you.
Not all of us are so fortunate however. I operate in environments where the databases don't have explicitly declared primary keys, foreign keys, indexes, and so forth, and the whole thing more resembles a collection of "flat files" than a database. I would guess that at least half the application programmers out there couldn't tell you what an ORM is, let alone enumerate the features of Entity Framework. The vast majority of programmers working today probably could not tell you what "first normal form" is, nor would they care enough to tolerate an explanation. Almost all the schema desigs I've seen feature grossly inconsistent uses of keys and status fields and are consequently a nightmare to maintain. The code (both in the database and the app) is generally a pile of undocumented spaghetti that requires hours for me to discern the intended funcionality. Outside of the few best-managed companies, his is the industry norm I'm afraid.
RAP is intended to raise the standards of the enormous bottom end of the industry by doing automatically what apparently so many programmers cannot or will not do for themselves. American industry is losing vast sums of money due to the poor design, inefficency and high error rate of the software that it pays people to develop. In my opinion, easily half the running apps today are dismal design failures whose shortcomings are not obvious only because most American management is basically uninterested in either application design or quality control ... and because there are no obvious alternatives. And I'm not even counting here the apps that were such total failures that they never even entered production.
It seems to me that all of American business is potentially poised for a smackdown comparable to what happened to our auto industry. If most of our underlying apps and schemas resemble most of the stuff I've seen, our companies will be fundamentally unable to respond to any challenge from anyone using any serious software quality control mechanisms. Could there come a day when essentially all American business is noncompetitive due to poor software app quality? Fifty years ago if you had suggested that the American automobile market would someday be dominated by non-Americans, and that they would dominate predominantly based upon their quality control (which derives from consistency), you would have been laughed at.
So please bear with me. Most app designers couldn't design a comprehensive auditing system on a dare; RAP gives it to them for free. Most apps don't do concurrency management; RAP provides it automatically. Most apps don't dare to ever actually delete records, thereby defeating referential integrity and smothering live data in obsolete data; RAP solves all these problems automatically. The list goes on and on.
So no, RAP is not trying to compete with top-end designs requiring the full features of Entity Framework. RAP is at the other end, trying to clean up the enormous mess.
October 14, 2010 at 3:15 pm
Thanks David, for such a fine article. After decades of quality training and Six Sigma classes, I believe your predictions about the demise of the unproductive American programmer may happen. It is ego that refuses to give up creativity in deference to proven designs. Much of the repetitive code where I work, is generated by CodeSmith and by in-house tools which allow us to deliver standardized apps very quickly.
Reducing variation should be the primary goal, if you want to deliver bug-free code quickly.
October 14, 2010 at 4:18 pm
David
I have been coding to databases for nearly 30 years. (I don't come from America(nth, sth or in between))
And I agree with you about most "designers". They need a re-education of what programming is!!
I think nHiberate and Entity Framework are better solutions(tools) than using stored procedures to maintain a database. Although I have reservations about their ability to Create a database or tables therein.
But you need to know the RDBMS to make good design decisions (eg refain from Views in SQL SERVER, embrace them in Oracle; same as with CURSORS. not for SQL SERVER but great in ORACLE) and you need to know the business domain to be able to optimise those databases for best effect.(eg look how a CLUSTERED indexes affect Inserts and therefore which keys to use as CLUSTERED, most of the time never the auto-generated number key.)
But if programmers use these data modelling tools they will not return to SPs. Except when the modelling tools can't do what a SP/Function can, or for elaborate reporting.
So I suppose that my approach to build Enterprise frameworks and educate/mentor young programmers to take a life approach to their own learnings. I just don't like spoon feeding people, even if it is good for them.
As far as management... Don't get me started else this will actually get much longer and dirtier.
As I said before, in general your Idea is good but it shouldn't impose a non-industry naming convention, have the ability to swap out tiers(refactor) for a better solution, be scalable(application farming), be expandable(add more features, coding and datastore), be able to be comunicated with(services) and most of all, be totally maintainable(no black boxes).
I would rather start programmers off by teaching these things than to hide these things from them.
Best of Luck
GregJF
October 15, 2010 at 8:33 am
David Ziffer (10/14/2010)
... American industry is losing vast sums of money due to the poor design, inefficiency and high error rate of the software that it pays people to develop. In my opinion, easily half the running apps today are dismal design failures whose shortcomings are not obvious only because most American management is basically uninterested in either application design or quality control ... and because there are no obvious alternatives. ...
and
It seems to me that all of American business is potentially poised for a smackdown comparable to what happened to our auto industry.
I believe the "smackdown" has started but the alternative is faring no better. When the small engineering company I worked for was acquired by a large corporation, my development team was disbanded by corporate decree that software development had to be outsourced. Efficiency, quality and our high performance user front ends disappeared as Indian code factories cranked out so-called industry standard apps. The "corporate" goal to reduce admin user time per technical report succeeded (10 mins. to 7 mins.) while ignoring professional user time (30 mins. to 120 mins.). The defense was that the outsourced app "met industry standards" and complied with corporate policy for development. Of course the people who decreed this policy will never have to use the app or take responsibility for the reduced productivity of the professional staff.
The net result, in my opinion, is to produce low quality apps with the illusion of lower costs. Your approach can change that, but it is not complete. Good development tools can still produce junk, faster and cheaper, whether they are used here or in India. That being said, at least good development tools allow the designers to focus on what makes a good application, and it isn't the hand-wired vacuum tube breadboards they are designing now.
I'll certainly give RAP a fair try. It looks promising provided I keep in mind that meeting the criteria for a productive application (from both a user and maintenance standpoint) is still my responsibility.
October 16, 2010 at 6:56 am
ismell (10/14/2010)
I've been using https://sourceforge.net/projects/mygeneration/ for a while now. It's not identical to RAP but it's a code generator
Thank you for this reference. I was unaware of this and will check it out. -the author
Viewing 15 posts - 16 through 30 (of 42 total)
You must be logged in to reply to this topic. Login to reply