Stairway to U-SQL Level 3: Processing Multiple Files With U-SQL

Question

Stairway to U-SQL Level 3: Processing Multiple Files With U-SQL

Mike McQuillan

SSCertifiable

Points: 6000
More actions
July 6, 2016 at 12:00 am

#400362

Comments posted to this topic are about the item Stairway to U-SQL Level 3: Processing Multiple Files With U-SQL

Viewing 8 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic. Login to reply

John N Hick Ten Centuries Points: 1320 More actions · Answer 1

You are bleeding edge with this - U-SQL seems to resemble a cross between T-SQL and Powershell, with a bit of DOSShell thrown in. I don't know when our company will adopt Azure, but this is fun to learn - thanks!

Mike McQuillan SSCertifiable Points: 6000 More actions · Answer 2

Hi John

I'm having fun writing it too, glad to hear you're enjoying the series! Thanks for the kind words.

Regards,

Mike.

Kevin3NF SSCarpal Tunnel Points: 4765 More actions · Answer 3

Skipping header rows is available now:

https://github.com/Azure/AzureDataLake/blob/master/docs/Release_Notes/2016/2016_08_01/USQL_Release_Notes_2016_08_01.md#u-sql-built-in-extractors-now-support-skipping-header-rows-skipfirstnrows-parameter

------------------------------------------------------------------------------------------------Standing in the gap between Consultant and ContractorKevin3NF DallasDBAs.com/Blog Why is my SQL Log File HUGE?!?![/url]The future of the DBA role...[/url]SQL Security Model in Plain English[/url]

Mike McQuillan SSCertifiable Points: 6000 More actions · Answer 4

Kevin3NF - Friday, January 27, 2017 9:44 AM
Skipping header rows is available now:
https://github.com/Azure/AzureDataLake/blob/master/docs/Release_Notes/2016/2016_08_01/USQL_Release_Notes_2016_08_01.md#u-sql-built-in-extractors-now-support-skipping-header-rows-skipfirstnrows-parameter

Hi Kevin

Yes, this feature has been available for a little while now. I'm working on a couple of new articles at the moment, once they're out of the way I'll update the early articles and will incorporate this feature into a future step of the stairway.

Thanks for pointing it out.

Cheers,
Mike.

mortalic SSCrazy Points: 2068 More actions · Answer 5

It appears the wildcard syntax you're using is deprecated.

This seems to work now:
@results = EXTRACT postcode string, oacode string, total int, percentage int, filename string FROM "/oacodes/{filename:}{*}.csv" USING Extractors.Csv();

Mike McQuillan SSCertifiable Points: 6000 More actions · Answer 6

mortalic - Tuesday, October 17, 2017 11:54 AM
It appears the wildcard syntax you're using is deprecated.
This seems to work now:
@results = EXTRACT postcode string, oacode string, total int, percentage int, filename string FROM "/oacodes/{filename:}{*}.csv" USING Extractors.Csv();

Hi there

Sorry for my delayed response. Funnily enough, I was playing about with this feature recently, and found the same as you - everything works as expected now.
I'll update this article when I post up the next step in the stairway. Thanks for taking the time to comment.

Regards,
Mike.

mortalic SSCrazy Points: 2068 More actions · Answer 7

You bet, thanks for making this article. It was really helpful getting me started.