June 28, 2021 at 12:00 am
Comments posted to this topic are about the item Where Does NoSQL work well?
June 28, 2021 at 8:21 am
Kristof Kovaks produced a good thumbnail sketch of various NOSQL solutions and has updated his page at least once. https://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
Web session info and state then something like REDIS works well. AWS DynamoDB provided you understand its use case and pricing structure. This is Key-Value pair NOSQL.
I feel that anything requiring an Entity Attribute Value model would be a candidate for a Document DB. Open ended surveys or surveys where the answers to a question can alter the questions you are going to be asked.
Searchable log files is a case where something like ElasticSearch can be very helpful. Let's face it SQL Server's full-text search has not had much love and attention for a very long while.
Graph databases are ones where I feel there is a lot of potential but now we have graph capabilities in SQL Server. I think we need some really good examples and resulting shift in thinking to get the most out of these. Because these are based on sound mathematical theory (just like RDBMS) these I think will grow.
But at the back of my mind is an increasingly loud voice telling me that the problems we experience in the data world have got nothing to do with technology.
June 28, 2021 at 11:10 am
But at the back of my mind is an increasingly loud voice telling me that the problems we experience in the data world have got nothing to do with technology.
- It's our approach to data quality
- It's the way we talk (or more accurately don't talk ) to each other. Conways law
- It's our understanding of the businesses we work for
- It's our ability to apply critical thinking and perform analysis of problems/solutions
- It's our ability to sell our ideas and solutions
David, I heartily agree with your assessment. I have been concerned with the more recent thinking that 'anyone can do it'. I don't have the background and training to determine if it is an innate trait or if it is due to adequate training ( I actually did get my degree in behavioral sciences and education), but I know there are people who 'have it' and people who don't. Back in my days managing a shop there were several times I hired new folks only to have them prove to be not capable of the performance we needed, and sometimes I had to let them go and be replaced.
Even now in our advanced years I see the differences between my wife and myself in thinking patterns. She was a graphic designer with years of computer graphics work, who started, operated, and sold her design studio with a number of employees of her own. She is the artistic one with whom I could never compete, and I am the problem-solver who figures out how to make things work.
And sometimes it IS the way we talk ( and think ). It's more than knowing things. It's being able to put that knowledge into the proper format and context to be communicated. Many times I find that while I have a whole background of thoughts in my head I don't always choose the right ones or the correct amount of relevant detail to express.
Rick
Disaster Recovery = Backup ( Backup ( Your Backup ) )
June 28, 2021 at 1:35 pm
I think that NoSQL environments have their use cases and shine well in those. I think the biggest problem is when we try to use NoSQL tools to replace RDBMS tools. They are fundamentally different. The reverse is true as well. RDBMS tools should not be used when a NoSQL tool is a better fit. For example, Hadoop with Hive is not a replacement for an RDBMS just as an RDBMS storing documents is not a replacement for MongoDB. People need to stop being over zealous and use the right tool for the right job. Life is not a one-size-fits-all scenario!
LinkedIn: https://www.linkedin.com/in/sqlrv
Website: https://www.sqlrv.com
June 28, 2021 at 2:03 pm
Where I work, we have a customer support call center application and also a mobile app where customers can login and look at their account info and payment history. Initially, these apps were hitting the data warehouse, but the volume of queries (> 1,000s per second) was proven to be untenable, even when the database was highly optimized with covering indexes and < 10ms runtime. All it took was a skew in cardinality or minor change to a query (like adding a single column) for performance of the application to get jammed up - which you totally don't want to happen when customers are on the line.
Currently, the backend is Cosmos DB, where all CRM data for a customer is contained in a single JSON document, and a replication process built on Kafka steams real-time updates from the EDW and other transactional databases. So, what Cosmos DB provides is high concurrency, consistent performance (because each query is a single read), and resilience or availability. Also, because it's hosted in Azure, the request units per second (RU/s) can be easily scaled up and down to support peak business hours or unexpected spikes in usage.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
June 28, 2021 at 2:56 pm
Eric,
In you situation, your essentially extracted the customer-centric data into a single document per customer then used a document database for that very purpose. Good use there considering the data is slowly changing and it collects all data per customer together in one place for multiple inquiries. And you have the bonus of not impacting your DW with customer inquiries!
That's a great example of using multiple technologies in a correct manner to solve the problem!
LinkedIn: https://www.linkedin.com/in/sqlrv
Website: https://www.sqlrv.com
June 28, 2021 at 6:31 pm
a relational store is likely better
It is important to remind everyone that a relational DBMS is not a way of "storing" data. The relational model provides a logical interface to data - and has nothing to say about physical storage. The data can be stored however the implementer chooses - that's the beauty of the thing.
I have worked with NoSQL "DBMS"s in the past. Hierarchical data structures like COBOL file descriptions for example or key value pairs on computers with 28kb per partition that can't support ISAM.
NoSQL is far more "traditional" than the RDBMS.
I think NoSQL can be safely ignored as jumping on any of the myriad NoSQL busses will inevitably lead to a career terminating dead end.
People have been touting various things that were going to replace the RDBMS ever since I started in the early 90s. None of them have ever really come to anything. The reason for that isn't that people are conservative, it is because the relational model is a brilliant idea with a very long future ahead of it.
June 28, 2021 at 7:11 pm
a relational store is likely better
It is important to remind everyone that a relational DBMS is not a way of "storing" data. The relational model provides a logical interface to data - and has nothing to say about physical storage. The data can be stored however the implementer chooses - that's the beauty of the thing.
I have worked with NoSQL "DBMS"s in the past. Hierarchical data structures like COBOL file descriptions for example or key value pairs on computers with 28kb per partition that can't support ISAM.
NoSQL is far more "traditional" than the RDBMS.
I think NoSQL can be safely ignored as jumping on any of the myriad NoSQL busses will inevitably lead to a career terminating dead end.
People have been touting various things that were going to replace the RDBMS ever since I started in the early 90s. None of them have ever really come to anything. The reason for that isn't that people are conservative, it is because the relational model is a brilliant idea with a very long future ahead of it.
Will, I agree that the no-sql route will probably be a dead-end, or a niche for a non-database professional. I started in the late 1960's with flat-files that we physically sorted into many orders, probably the outgrowth of card cabinets with drawers of cards that we sorted on the old mechanical sorting machines, then indexed flat-files, hierarchical databases, and relational databases. Instinct tells me the relational model is going to be here for a very long time into the future. Anything other at this point in time I see as a place for non-db-professionals.
Rick
Disaster Recovery = Backup ( Backup ( Your Backup ) )
June 29, 2021 at 5:20 am
David.Poole
Graph databases are ones where I feel there is a lot of potential but now we have graph capabilities in SQL Server. I think we need some really good examples and resulting shift in thinking to get the most out of these. Because these are based on sound mathematical theory (just like RDBMS) these I think will grow.
The question is whether graph DBMSs can do things that at RDBMS cannot do, or if they can do the same things in a more concise and flexible manner?
Functional programming has a sound mathematical basis, but for data management is clearly less flexible than the relational model.
SQL is just once possible language to use for an RDBMS and a language more faithful to the relational model and that removes much of the redundancies and inconsistencies of SQL would be a great step forward (a relational NoSQL DBMS)
Regarding performance and scalability there is no need to abandon the RDBMS to achieve this - with an RDBMS the implementer is completely free to choose whatever physical method of storage they find most appropriate - without changing the interface (SQL in the case of SQL DBMSs).
I am a bit puzzled by Hadoop - programmers write programs that are submitted as jobs - this seems like a return to 1970s style mainframe batch programming - something the we plucky young bucks at the start of the eighties swept away with online, real time systems.
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply