Thank you for joining in this course. In this educational announcement, I want to explore a possible future of data analysis using Microsoft.
It used to be that Microsoft’s main cloud for data analysis and other functions was Microsoft Azure. However, Microsoft has started to de-emphasize Azure for data analysis.
- Last year it retired its DP-500 exam “Microsoft Certified: Azure Enterprise Data Analyst Associate”.
- It is also going to retire its very popular DP-203 exam “Data Engineering on Microsoft Azure” on 31 March.
So what is it focusing on instead for data analysis? The answer is: Power BI and Microsoft Fabric.
- Last year it launched the DP-600 certification “Fabric Analytics Engineer Associate” as a direct replacement for the DP-500 exam.
- Now it has launched the DP-700 certification “Fabric Data Engineer Associate” at the same time as retiring the DP-203 exam.
Microsoft has said that it has no plans to retire Azure services such as Azure Synapse Analytics. However, I think that its additional investments will be more weighted to Fabric than to Azure data analysis services.
Why should you study Microsoft Fabric?
In its blog, Microsoft has said “The next versions of our big data analytics products are now a core part of Microsoft Fabric.” By studying Fabric, you will be future-proofing your skills.
Also, if you use Power BI, it integrates natively into Microsoft Fabric. You can create Fabric items such as lakehouses and data warehouses to store data. They automatically create a default semantic model, which can be used in Power BI.
As part of their commitment to making Fabric Microsoft’s analytical products, Microsoft has recently added its popular SQL Database, originally in Azure, into Fabric.
What data analysis services are available in Power BI and Microsoft Fabric?
In the above graphic you can see some of the data analysis services in Microsoft Fabric.
- First, you start with data sources – either structured, semi-structured or unstructured batch data, streaming data, or data which is held in other data sources, and accessible in Fabric.
- Then you ingest the data into Fabric, using pipeline, dataflows or eventstreams, or accessing the data using shortcuts or have mirrored data.
- Then you transform the data, querying batch data in lakehouses using PySpark or in data warehouses using SQL, or querying streaming data using eventhouses.
- Finally, you expose batch data, either using a lakehouse or data warehouse SQL endpoint, or creating Power BI reports and dashboards using their default semantic model. You can also expose streaming data using a KQL queryset, a Real-time Dashboard, or by again creating Power BI reports and dashboards.
How can I learn how to use Power BI and Microsoft Fabric? And what do I need to learn?
Microsoft has an exam relating to Power BI – the PL-300 “Microsoft Power BI Data Analyst”. It consists of four parts:
- Prepare the data using Power Query/Get and Transform. This can use sources such as text files, Excel spreadsheets, and databases.
- Model the data. This expands the data into a semantic model, which can include calculated columns and measures using DAX.
- Visualize and analyze data. This is the analysis of the data using Power BI visuals, visible to the end user.
- Manage and secure Power BI, in both Power BI Desktop and the Power BI Service.
In the above graphic, all this equates to the PBI service in the Expose column, and dataflows in the Transform column.
Microsoft also has two exams and four Microsoft Applied Skills relating to Fabric. This means that you can study a wide area of Fabric, or a narrow area. The exams are:
- DP-600 “Implementing Analytics Solutions Using Microsoft Fabric”. This equates to the services highlighted in the graphic above, together with the Eventhouse and KQL Queryset.
- DP-700 “Implementing Data Engineering Solutions Using Microsoft Fabric”. This equates to the first three columns in the graphic above. It covers pipelines, dataflows, eventstreams, shortcuts and mirrored databases to ingest data, write PySpark code in notebooks, SQL code in notebooks and the SQL Analytical Endpoint in Lakehouses and Data Warehouse, and KQL code in Eventhouses.
The four Microsoft Applied Skills relating to Fabric are:
- Implement a lakehouse in Microsoft Fabric. This uses SQL or PySpark to query and manipulate the data.
- Implement a data warehouse in Microsoft Fabric. This uses SQL to transform and query data.
- Implement a Real-Time Intelligence solution with Microsoft Fabric. This uses KQL to process data in an Eventhouse which has been ingested and transformed in an Eventstream.
- Implement a data science and machine learning solution with Microsoft Fabric. This uses scikit-learn and SynapseML to train, track and score a model using previously imported data.
How can I learn the skills for Power BI and Microsoft Fabric?
We have three video courses relating to Power BI and Microsoft Fabric:
- PL-300: Microsoft Power BI Data Analyst. This covers the four parts for Power BI: prepare the data, model the data, visualize and analyze data, and manage and secure Power BI.
- DP-600: Fabric Analytics Engineer Associate. This follows on from the PL-300 exam, and covers a fifth part for Power BI, in addition to using SQL to query lakehouses and data warehouses, and KQL to query eventhouses.
- DP-700: Fabric Data Engineer Associate. This standalone course uses PySpark and SQL to query lakehouses and data warehouses on data which has been imported using Dataflow Gen2 and pipelines, together with eventstreams, eventhouses, KQL querysets, and more.
For more about our video courses, please click on the above links.
Conclusion
In summary, it appears, in terms of Microsoft data analysis, that Azure is the past and the present, but Microsoft Fabric is the future.
If you are currently creating data analysis in Microsoft, and especially if you are using Azure, then you should investigate Power BI and Fabric’s services. This will allow you to expand your existing skills with new ways of transforming, analyzing and presenting data.