Using Spark Connect from .NET

Introductory Ramble Spark Connect In July 2022, at the Data and AI summit, Apache Spark announced “Spark Connect,” which was a way of connecting to Apache Spark using the...

Just Buy the Keyboard

The holidays have passed and it’s a new year. You probably have a gift card or two and haven’t decided how to use it yet. Allow me to help:
Buy...

Architecting Zero Downtime Deployments at THAT Conference

Thanks to everyone that came to my talk at THAT Conference. The deck is here, if you want to download it and use it as a resource, or to...

Fabulous Stuff in Fabric – Part 3: Shortcuts

A while ago I had a little blog post series about cool stuff in Snowflake. I’m doing a similar series now, but this time for Microsoft Fabric. I’m not going...

Last Weeks Reading (2024-01-28)

📰 News Fabric end-to-end use case: Data Engineering part 2 – Pipelines If you’re new in Microsoft Fabric – check this series and build it with Sam. Set up...

Off to THAT Conference

I’m in Round Rock, TX today at THAT Conference. I attended THAT Conference in Wisconsin Dells last summer, and it was fun. I took my wife, but I wish...

Troubleshooting SQL Server Error 8623: The query processor ran out of internal resources and could not produce a query plan

SQL Server Error 8623

At least once day, an application for one of the databases I manage gets this error:The query processor ran out of internal...

Videos from MITRE ATT&CKcon 4.0

If you aren’t familiar with the ATT&CK framework and knowledge base, it’s a tool to help classify and understand cyberattacks. Recently (Oct 2023), ATT&CKcon 4.0 was held and MITRE...

A New Word: Lyssamania

lyssanmania – n. the irrational fear that someone you know is angry at you, that as soon as you wander into the room, you’ll be faced with a barrage...

Exploring the Spark Connect gRPC API

All Spark Connect Posts
Code
Goal of this post In the first two posts, we looked at how to run some Spark code, firstly against a local Spark Connect server and...