Stairway to U-SQL Level 6: U-SQL Indexes and Partition Schemes

Question

Post reply

Stairway to U-SQL Level 6: U-SQL Indexes and Partition Schemes

Mike McQuillan

SSCertifiable

Points: 5999
More actions
August 10, 2016 at 12:01 am

#310382

Comments posted to this topic are about the item Stairway to U-SQL Level 6: U-SQL Indexes and Partition Schemes

Viewing 9 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic. Login to reply

akljfhnlaflkj SSC Guru Points: 76202 More actions · Answer 1

akljfhnlaflkj

SSC Guru

Points: 76202

August 10, 2016 at 6:44 am

#1894756

I need to go back and read the prior lessons.

Michael Rys SSC Enthusiast Points: 152 More actions · Answer 2

We have changed the syntax for Partitioning and Distribution. Can you please update your samples.

More details here: https://github.com/Azure/AzureDataLake/blob/master/docs/Release_Notes/2016/2016_08_01/USQL_Release_Notes_2016_08_01.md#start-of-deprecation-of-old-partitioned-by-syntax

Cheers

Michael

Mike McQuillan SSCertifiable Points: 5999 More actions · Answer 3

Hi Mike

Thanks very much for your updates. I'll take a look and will update this article in the next week or so.

Regards,

Mike.

nzhang6666 SSCrazy Points: 2217 More actions · Answer 4

For 070 script, it says file not found: /ssc_uk_postcodes/postcodes/postcodes.csv

I am following the series from start and have the project built on level 5. Thanks.

nzhang6666 SSCrazy Points: 2217 More actions · Answer 5

nzhang6666 (10/14/2016)
For 070 script, it says file not found: /ssc_uk_postcodes/postcodes/postcodes.csv
I am following the series from start and have the project built on level 5. Thanks.

It works now, but I change the path to where the postcode file saved locally, something like this:

DECLARE @DataFilePath string = "C:/Users/n.z/Downloads/usql_files/U-SQL_Files/Postcodes/Postcodes.csv";

also the output file path, something like this: TO "C:/Users/n.z/Downloads/usql_files/U-SQL_Files/Postcodes/knowsleyPostcodes.tsv"

Also found another way is to create folders under project, e.g. for the original script to work, need to create folders ssc_uk_postcodes/postcodes under project, and add the postcode csv in it. The output location will be (automatically) C:\Users.z\AppData\Local\USQLDataRoot\outputs

Mike McQuillan SSCertifiable Points: 5999 More actions · Answer 6

Hi nzhang6666

Good to hear you are following along, and sorry to hear you had some trouble with the files. I've just had a check and I've noticed that I don't specify how to add the files to the Visual Studio project in Level 5. I'll update the article to reflect this.

To follow the series exactly, you need to create a ssc_uk_postcodes folder within your Visual Studio solution, with appropriate sub-folders (counties, postcodes, districts, postcodeestimates). Then put each of the appropriate files in there. Which is what you mention in your second point.

Your change (where you've specified a local file path) will work locally, but it won't work if you try to run the code in Azure. Adding the above folders locally and in Azure will guarantee consistency in the scripts, as you're using relative instead of absolute paths.

I'll post a message here once the article has been updated. Thanks for your interest.

Best regards,

Mike.

guna.srikanth Grasshopper Points: 23 More actions · Answer 7

I just started on U-SQL and new to Microsoft Azure. Quick basic Question - If I need to insert any Data into U-SQL Database, first I need to copy over the contents of the Table from OnPremise Server to Azure Data Lake Store and then use U-SQL Script to insert the Data into a Indexed, Partitioned Table in U-SQL Database. I think I am having multiple Files for the same Data set when it comes to Azure Storage. Can we insert Data into U-SQL Tables (U-SQL Database) directly from onPremise Server ???

Mike McQuillan SSCertifiable Points: 5999 More actions · Answer 8

Hi Guna

Sorry for my delayed reply. Unfortunately you must copy your data up to the Data Lake first before you can insert data.

The only alternative is to run your database locally via Visual Studio and run queries from there. Of course, you're missing out on the benefits of using Azure doing that.

Regards,
Mike.