Post reply

The Cloud as a Foundation

  • February 9, 2022 at 12:00 am

    #3980799

    Comments posted to this topic are about the item The Cloud as a Foundation

  • February 9, 2022 at 3:19 am

    #3981428

    I think that the cloud has some fantastic capabilities and opportunities.  Despite some of my posts, my big concern isn't so much about the cloud, it's about the "shiny new object" syndrome that a lot of people seem to suffer.

    For one company that I work for, they had a developer make a fascinating tool using Python and a couple of other "hooks" to import data data into SQL Server.  They worked on it for quite a while and with a little more work (moth or two?), it'll be done.  What's my concern there?  Basically, the rewrote BULK INSERT in Python. <headdesk><facepalm> 😀

     

    --Jeff Moden

    RBAR is pronounced "ree-bar" and is a "Modenism" for Row-By-Agonizing-Row.
    First step towards the paradigm shift of writing Set Based code:
    ________Stop thinking about what you want to do to a ROW... think, instead, of what you want to do to a COLUMN.

    Change is inevitable... Change for the better is not.

    Helpful Links:
    How to post code problems
    How to Post Performance Problems
    Create a Tally Function (fnTally)

  • February 9, 2022 at 2:49 pm

    #3981522

    I think I would have reservations regarding entrusting my data to a platform called 'Snowflake'.  And maybe even more so to a platform called 'Cockroach'.

    Rick
    Disaster Recovery = Backup ( Backup ( Your Backup ) )

  • February 9, 2022 at 6:44 pm

    #3981581

    ".. Snowflake is something that a company can run on AWS, Amazon, or GCP .."

    Maybe you meant to say Azure instead of "Amazon", but Snowflake can be hosted in Azure as well. We're actually researching Snowflake as a potential alternative to our current EDW in SQL Server (and maybe Azure SQL Analytics is an option as well). I'll be involved in a proof of concept implementation this quarter.

    When comparing database platforms, especially NoSQL or cloud native options, one thing to consider is that most are optimized for  specific data access patterns, and the underlying architecture excels for some use cases - while being very unsuited for others.

    For example, Snowflake doesn't create indexes. Instead it splits up the data into 10 MB "micro-partitions" and clusters the data horizontally across distributed nodes. This is all done auto-magically, and that is it's big selling point. Little to no database administration as we know it (physical data modeling, configuration, etc.) is required.

    I'm just now getting my feet wet, but apparently it's optimized for extremely fast bulk ELT and OLAP or data science workloads using ANSI SQL or Spark, but not recommended for application (ie: select for customer 123) type queries. So, the trade off is that it's not as flexible as SQL Server. My assumption at this point is that if Snowflake is used as the core EDW, then it will serve as a hub, and various application centric datamarts will need to spoke off from it. The datamart or microservices layer, that is where SQL Server, Azure SQL, or Azure CosmosDB would be better suited.

    "Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho

  • February 9, 2022 at 6:59 pm

    #3981587

    Eric M Russell wrote:

    ".. Snowflake is something that a company can run on AWS, Amazon, or GCP .."

    Maybe you meant to say Azure instead of "Amazon", but Snowflake can be hosted in Azure as well. We're actually researching Snowflake as a potential alternative to our current EDW in SQL Server (and maybe Azure SQL Analytics is an option as well). I'll be involved in a proof of concept implementation this quarter.

    Good catch. Yes, I meant Azure and edited the piece. Snowflake works everywhere.

Viewing 5 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic. Login to reply