Microsoft has just announced Microsoft Fabric during the Microsoft Build conference. This is a unified analytic solution of the era of AI.
Microsoft Fabric is an end-to-end, unified analytics platform that brings together all the analytics tools that organizations need. It covers several core workloads: data integration, data engineering, data warehousing, data science, real-time analytics, and business intelligence — all served together in a single Software-as-a-Service (SaaS) solution, sharing one user experience and common lake-centric architecture (OneLake), using open data formats, and having security and governance managed centrally.
With Microsoft Fabric, customers no longer need to stitch together individual analytics services from multiple vendors. Instead, they can use a single product with unified experience and architecture that empowers every team in the analytics process with the role-specific experiences they need, so data engineers, data warehousing professionals, data scientists, data analysts, and business users feel right at home.
Why do we need a unified analytic solution? The answer is simple and straightforward – research shows “the amount of data created over the next three years will be more than the data created over the past 30 years.” Investing in the technology you need to make real-time, data-driven decisions without compromising security or governance is critical for the organization’s long-term success.
What is important is that businesses today face a variety of data and analytics challenges that can be difficult to navigate.
Here’s how Microsoft Fabric can begin to address these challenges.
First, implementing an open and scalable analytics environment with built-in security, governance, and compliance capabilities enables you and your team to have peace of mind while increasing your efficiency.
Establishing a unified center of enablement has become an increasingly essential component of decision making. With democratized access, let your teams easily employ a wide range of easy-to-use analytics tools that reduces time to powerful insights.
Most importantly, having a governed source of truth ensures organizations operate in a standardized way and use the same data––making it easier to keep secure and govern.
The current state of data, analytics, and BI
The current state of data, analytics, and business intelligence can be described as being compartmentalized, insecure, costly to oversee, and inaccessible to non-technical users.
By using comprehensive data and analytics technology, customers can overcome these challenges and gain a competitive advantage:
- Currently, organizations rely on a variety of siloed solutions and data. With an integrated, SaaS-based suite, all end-users have access to the same, unified data in an integrated suite.
- Today, managing data comes with serious security risks. An integrated data and analytics suite with built-in, industry-leading security, compliance, and governance, gives your teams peace of mind.
- More than ever before, there is an overwhelming demand for data stewards at every organization. An easy-to-use data & analytics platform empowers users in various non-technical roles to leverage analytics for increased business agility.
- It’s no secret that legacy systems come with costly integration and ongoing maintenance.Therefore, it is imperative that businesses have access to a cost-transparent suite with reduced overhead.
Introducing Microsoft Fabric
Fabric brings together existing offerings like Data Factory, Synapse, and Power BI into a single unified product for all your data and analytics workloads:
- Data Factory (data integration)
- Synapse Data Engineering
- Synapse Data Warehouse
- Synapse Data Science
- Synapse Real-Time Analytics
- Power BI (Business Intelligence)
- Data Activator (coming soon)
From a Data Integration standpoint: Microsoft Fabric offers comprehensive data migration and integration to enable a lake-first pattern. Azure Data Factory connectors enable data integration, while Azure Synapse Link connectors enable “no code” and “always synchronized” data integration for operational databases.
From a data organization perspective, Microsoft Fabric leverages OneLake: This ensures that all data is ingested into a data lake on Azure Data Storage Gen––a cost- and performance-optimized storage service––for the most demanding business intelligence, machine learning, and artificial intelligence workloads.
On an Analytics front: Data scientists can bring their preferred compute frameworks, languages, runtimes, and tools to the data lakehouse and further enhance the data through feature engineering and statistical techniques.
When it comes to Business Intelligence: The Microsoft Intelligent Data Platform (MIDP) offers best-in-class integrated solutions to responsibly democratize business intelligence with self-serve tools and experiences for data analysts and data citizens.
And lastly, from a Governance standpoint: Microsoft Purview then provides a single pane governance solution to help effectively scan and manage your data estate––even as it grows and scales.
Common analytics scenarios
Microsoft Fabric supports four common end-to-end analytics scenarios:
- Lakehouse
- Data Warehouse
- Data Science
- Real Time Analytics
Keep in mind that while these are the most common scenarios supported by Microsoft Fabric, there are many others that technical and business end-users can navigate through.
Lakehouse
The Microsoft Fabric Lakehouse analytics scenario makes it so that data can be ingested into OneLake with shortcuts to other clouds repositories, pipelines, and dataflows in order to allow end-users to leverage other data.
Once that data has been pulled into Microsoft Fabric, users can leverage notebooks to transform that data in OneLake and then store them in Lakehouses with medallion structure.
From there, users can begin to analyze and visualize that data with Power BI using the see-through mode or SQL endpoints.
Data Warehouse
The Data Warehouse analytics scenario takes existing sources that are mounted, while pipelines and dataflows can bring in all other data that is needed.
IT teams can then define and store procedures to transform the data, which is stored as Parquet/Delta Lake files in OneLake.
From there, business users can analyze and visualize data with Power BI, again using the see-through mode or SQL endpoints.
Data Science
The Data Science analytics scenario can be ingested similarly to the Lakehouse and Data Warehouse paths.
Once the data is ingested, it is cleaned and prepared using notebooks and then stored in the Lakehouse with medallion structure.
After the data is cleaned and stored, machine learning models can be trained and tested directly on the Lakehouse.
Like the other analytics scenarios, business users can analyze and visualize the data with Power BI using the see-through mode or SQL endpoints.
Real Time Analytics
Unlike the Data Science, Lakehouse, and Data Warehouse analytics scenarios, streaming data can be ingested into the Microsoft Fabric in several ways to achieve real-time analytics.
Users can leverage Event Hub, IoT Hub, pipelines, dataflows, notebooks, or open-source products like Kafka, Logstash, and more.
Once ingested into the Microsoft Fabric, streaming data can be stored in Kusto DB and mirrored into Lakehouse. After the data has been stored, machine learning models can be trained and tested directly on the Lakehouse with experiments.
Like the other scenarios, business users can analyze and visualize the data with Power BI using the see-through mode or SQL endpoints. Data can also be exposed through KQL or notebooks using Spark.
Useful links
Here are the links you should visit now:
- Explore the product http://aka.ms/try-fabric
- Q&A webinars https://aka.ms/fabric-webinar-series
- Latest announcements https://aka.ms/fabric-tech-blog
- Implementation guide https://aka.ms/Getting-Started-eBook
What’s next
Stay tuned for more Fabric blog posts. I will go through all the scenarios showing you on practical examples how to start and implement them.