I had an amazing time adventuring to Wales with other members of the data community last week! It was my first international conference, and I had heard so many incredible things about it, but still it blew away all my expectations. I’m so honored to have met so many incredible people throughout the week! Thank you to new friends and old for making it a conference to remember.
Below are the links to GitHub for my session as well as some notes from the sessions I attended. Disclaimer, these notes are not even close to the real thing. Be sure to check out these sessions if you see them being held at your local user group or SQL Saturday in the future! I tend to take very bullet point oriented notes to best organize my thoughts, hopefully these spark your interest and guide you to professionals who know about specific technologies within the data realm.
Check out the agenda and feel free to reach out to folks with topics you’re interested in! They may have their slide deck easily accessible to folks who may not make it to the conference.
Power BI Meets Programmability with Yours Truly
Thank you so much to everyone who made it out to my session!
It was incredible to have 48 people in person and another 21 virtual! It was a humbling experience to see that room fill up and know they were all there to learn something new and exciting to help with their day-to-day work. Can’t wait to look through the feedback and learn where to tweak my presentation to make an even bigger impact in the future!
- Github with slide deck, sample C# code, and sample PBIX file (with Excel file that powers it): Github Link
- Category of blog posts related to TOM and C#: https://dataonwheels.wordpress.com/category/c-tom-and-power-bi/
- Fun tip someone showed me after the session:
If you use /* and */ to comment out chunks of code (like I do frequently during the session), you can use /* and –*/ so that you only need to comment out the /* to run that block. Pretty neat!
Supercharge Power BI with Azure Synapse Analytics with Mathias Halkjaer
Limitations of PBI
- Source load minimization doesn’t exist in most cases (lots of queries against the same source)
- Out of sight transformation layer
- Not best tool for data warehouse tasks
- No logs or output from quality checks
- Doesn’t handle humongous data very well
Enchantments needed for PBI from Synapse
- Time-travel (historical data & change tracking)
- Enlarge (massive scale)
- Counter spell (reverse ETL)
- Wish (supports multiple programming languages)
- Divination (AI)
- Regenerate (CI/CD and source control)
Azure Synapse
- A toolbox full of tools: serverless sql, data warehouse, spark engine, etc.
Data Lake
- Landing zone for data
- Cheap
- Redundant
- Quick and dirty analytics
- Performance
- Most efficient way to load data into data warehouse
- Easily consumed
Report Design & Psychology (Quick Tips) with Jon Lunn
Dual process theory: intuitive vs attentive
You can mentally process up to 15 items intuitively. Past that it gets pushed to attentive.
Things can move from system 2 (attentive) to system 1 (intuitive) with repetition (aka driving or typing).
Layout – left to right, top to bottom. Move most important KPI to top left. Remove distractions (excess colors, 3d visuals, pie charts, etc). Titles in upper left make it easier to know what’s contained within the visual.
Tip – to visualize dots use dice patterns to move it to a system 1 process.
Ratios (how many, relative) vs percentages (how likely, chance risk probability). Both are good options to contextualize a number more. We usually understand ratios much better because how many is more intuitive than how likely.
Intuitive systems are more prone to errors or bias. If you want to hide something, go for percentage.
Use the power of 10’s (aka 8 out of 10 preferred it). Round and use a ratio to move it into intuitive process.
Power BI Datamarts What, How, Why with Marthe Moengen
You can manage RLS roles within datamart, join tables via GUI, and join with similar UI to PBI desktop modeling view. In preview only. You can use as a SQL endpoint to connect to your datamart.
Why? Access data through a SQL endpoint. Do simple aggregation using SQL script instead of M. There are potential challenges since this is in preview.
It keeps getting easier to move Oracle and Open Source workloads to Azure with David Levy
- Cloud migration phases:
- Discover
- Assess
- Migrate
- Optimize
- Secure & Manage
- Azure Migrate: central hub of tools for datacenter migration
- Primary tool for discover and assess
- Azure Database Migration Service – Gen 2 has much more control for end users to say how many resources that item gets
- Use Oracle Assessments in Azure Data Studio to get recommendations for sku size
- Oracle databases in Oracle cloud can use OCI Interconnect to pull data back and forth as Azure IaaS
- A lot of people move to PostgresSQL to pay less and get open source resources
- Migration strategy
- Rehost
- Refactor
- Rearchitect
- Replace
- No better way to run software than by SaaS. Why invest the time and resources when the vendor can maintain this for you and allow you to focus on your business?
- The Oracle Assessments in Azure Data Studio can see all the database details, SKU recommendations, and Feature Compatibility. Each section has a migration effort attached, very nice
- Azure is a great place for Postgres SQL:
- High availability
- Maintained
- PostgresSQL migration extension can run assessment in ADS and understand the migration readiness
- Integrated Assessment
- Easy set up and config in ADS
- To migrate postgres
- Pg_dump scripts out the PostgresSQL schema
- Pg-sql allows creating databases and tables
- Use the PostgreSQL to Azure Database for PostgreSQL Online Migration Wizard
- You can start cutover with pending changes
- Azure Database for MySQL
- Flexible server is now available, allows mission critical apps to use zone redundancy and fine-grain maintenance scheduling
- Azure Database Migration Service (ADMS)
- Migrates schema an data
- Preview = replicate changes
- You can do this as an online migration
- Optimizing your environment
- Costs during & after migration
- During: Azure TCO Calculator, Azure Migrate, Azure Hybrid Benefit & Reserved Instances, and join Azure Migration & Modernization Program
- Do reservations early, you’ll lose money otherwise
- AMMP = Azure Migration & Modernization Program, provides coaching with best practice
- During: Azure TCO Calculator, Azure Migrate, Azure Hybrid Benefit & Reserved Instances, and join Azure Migration & Modernization Program
- Make sure you get good data for trouble times so your migrated instance is ready for the worst case scenario
- Execute iteratively
- Costs during & after migration
5 Things You Can Do to build better looking reports – James McGillivray
- Design is subjective
- No clear end point
- No one solution
- Design is often seen as less important
- Blog: jimbabwe.co.za
- Tip Techniques
- Grid Layout
- Pro: left brain, less layout tweaking happens, looks professional
- Con: can look boxy,
- Gutters = white space between visuals
- Margins = white space along edge of report
- Maximize Real Estate
- Pro: bigger visuals = more value, fewer visuals = less distractions
- Con: often hard to convince stakeholders, all questions not answered by default
- Beautiful Colors
- Pro: use color to convey meaning, highlight data, show continuous data, qualitative sequential divergent
- Con: many pitfalls, accessibility concerns, can be polarising
- Consider the Audience
- Pro: most important elements first, remove unneeded elements, hierarchy view (summary on top, grouped middle, detail on bottom)
- Grid Layout
Identifying and Preventing Unauthorized Power BI Gateways with Angela Henry
- The problem: available to all, gateways are not only for PBI, results in chaos
- How do we find gateways?
- Manage connections & gateways. Let’s us identify gateways. Need to be Azure AD global, PBI service admin, or Gateway Admin
- Reason to restrict: governance