How to get data in a DataFrame via .NET for Apache Spark
When I first started working with Apache Spark, one of the things I struggled with was that I would have some variable or data in my code that I...
2021-01-17
2 reads
Ed Elliott
Git in 10 commands to do 99% of gitting and keeping out of trouble
Git is hard, probably harder than it needs to be but I have been using it for about 5 years and have a workflow that works for me. I...
2021-01-12
3 reads
TF-IDF in .NET for Apache Spark Using Spark ML v2
Spark ML in .NET for Apache Spark Apache Spark has had a machine learning API for quite some time and this has been partially implemented in .NET for Apache...
2020-12-19
75 reads
TF-IDF in .NET for Apache Spark Using Spark ML v2
Spark ML in .NET for Apache Spark Apache Spark has had a machine learning API for quite some time and this has been partially implemented in .NET for Apache...
2020-12-19
3 reads
Blog .NET for Apache Spark UDFs Missing Shared State
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
98 reads
Blog .NET for Apache Spark UDFs Missing Shared State
The Problem When you use a UDF in .NET for Apache Spark, something like this code:
class Program { static void Main(string args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select");...
2020-10-29
2 reads
Spark .NET - Spark ML TF-IDF
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
40 reads
TF-IDF in .NET for Apache Spark Using Spark ML
Spark ML in .NET for Apache Spark Spark is awesome, .NET is awesome, machine learning (ML) is awesome, so what could be better than using .NET to write ML...
2020-07-07
8 reads
TF-IDF in .NET for Apache Spark Using Spark ML
Last Updated: 2020-10-18
NOTE: What you read here was before .NET for Apache Spark 1.0 which includes everything we need to do this purely in .NET - in this post...
2020-07-07
17 reads
Approaches to running Databricks ETL code from Azure ADF
Databricks is fantastic, but there is a small issue with how people use it. The problem is that Databricks is all things to all people. Data scientists and data...
2020-06-03
192 reads
Blogs
Privacy Policy
We value your privacy. This policy explains what personal data we collect and why. What...
Why use XACT_ABORT?
This came up one day at my work when a developer was using it....
Using AI for Data Conversion
I’m sure I’ve never mentioned that I’m an amateur radio operator. Like Vegans and...
Forums
sp_lock memory leak
I spotted this in the release notes for CU17 in SQL Server 2022: KB3616559...
Migration related issue
I migrated DB server from 2017 enterprise to 2019 enterprise. After the migration, every...
How Much AI Code Would You Use?
Comments posted to this topic are about the item How Much AI Code Would...
Question of the Day
Browsing for Instances
I have a SQL Server instance that is not listening for connections on port 1433. By default, all ports are locked down on my server, except for the SQL Server port. What can I do to allow users to connect to the instance without knowing the port?
See possible answers