How do we test ETL pipelines? Part one unit tests
Why do we bother testing? Testing isn’t an easy thing to define, we all know we should do it, when something goes wrong in production people shout and ask...
2019-07-25
43 reads
Ed Elliott
How do we test ETL pipelines? Part one unit tests
Why do we bother testing? Testing isn’t an easy thing to define, we all know we should do it, when something goes wrong in production people shout and ask...
2019-07-25
spark-dotnet examples - reading and writing csv files
How do you read and write CSV files using the dotnet driver for Apache Spark?
I have a runnable example here:
https://github.com/GoEddie/dotnet-spark-examples
Specifcally:
https://github.com/GoEddie/dotnet-spark-examples/tree/master/examples/split-csv
Let’s take a walkthrough of the demo:
Console.WriteLine("Hello Spark!"); var spark...
2019-07-15
32 reads
spark-dotnet examples - reading and writing csv files
How do you read and write CSV files using the dotnet driver for Apache Spark?
I have a runnable example here:
https://github.com/GoEddie/dotnet-spark-examples
Specifcally:
https://github.com/GoEddie/dotnet-spark-examples/tree/master/examples/split-csv
Let's take a walkthrough of the demo:
Console.WriteLine("Hello Spark!"); var spark...
2019-07-15
1 reads
spark-dotnet how does user .net code run spark code in a java vm
Apache Spark is written in scala, scala compiles to Java and runs inside a Java virtual machine. The spark-dotnet driver runs dotnet code and calls spark functionality, so how...
2019-07-14
7 reads
spark-dotnet how does user .net code run spark code in a java vm
Apache Spark is written in scala, scala compiles to Java and runs inside a Java virtual machine. The spark-dotnet driver runs dotnet code and calls spark functionality, so how...
2019-07-14
2 reads
Spark and dotnet in a single docker container
I really like the new dotnet driver for Spark because I think it makes spark more accesable to devs who might not know pythpn or scala.
If you want to...
2019-07-11
14 reads
Spark and dotnet in a single docker container
I really like the new dotnet driver for Spark because I think it makes spark more accesable to devs who might not know pythpn or scala.
If you want to...
2019-07-11
Where do you do your database development? Hopefully not production
Here are three scenarios, if you work with SQL Server either as a provider of database environments (DBA) or a consumer of database environments for your application (developer) then...
2019-07-05
3 reads
Where do you do your database development? Hopefully not production
Here are three scenarios, if you work with SQL Server either as a provider of database environments (DBA) or a consumer of database environments for your application (developer) then...
2019-07-05
1 reads
Blogs
My 2024 in Data: Reading
By Steve Jones
This is my last week of the year working (I guess I come back...
My 2024 in Data: Speaking
By Steve Jones
This is my last week of the year working (I guess I come back...
Top 5 Advanced SQL Courses for 2025 (Must-Know)
Want to seriously boost your data skills? Mastering advanced SQL is the key, whether...
Forums
Need ADF pipeline suggestion to copy 1 million files
I have an ADF pipeline that copies Files from source to destination. Both Source...
Function Defaults
Comments posted to this topic are about the item Function Defaults
Question of the Day
Function Defaults
I have created this function in SQL Server 2022:
CREATE FUNCTION dbo.AddInt (@one INT, @two INT = 1) RETURNS INT
AS
BEGIN
RETURN @one + @two
END
How can I call this and invoke the default value for @two? See possible answers