How do you handle SSIS package merging between two BI teams?

  • Phil Parkin - Thursday, October 18, 2018 7:13 AM

    xsevensinzx - Thursday, October 18, 2018 6:47 AM

    I mean, that's not a bad suggestion. Anything you can do in SSIS, I can likely do the exact same with TSQL. 😀

    OK, here's a nice starter for you. What T-SQL commands would you use to grab Google Analytics data (which requires a web service login using OAuth 2.0)?

    TSQL + Python script. Easy.

    Need that CmdExec though. Save as CSV and can even use Jeff's favorite, BULK_INSERT. That or have Python call a script itself to write the data using a stored procedure.

    In the event you cannot do that due to security reasons or the engine does not support it, then you put the Python script on a Windows job to download at the frequency you need and use TSQL to ingest the data once it's available on a Agent job. Remember, Python contains all you need to authenticate with OAuth and Google. You can even use SQL tables to store the secret keys and retrieved via stored procedure called by the Python script separately. 

    Need something more challenging here. :laugh:

  • Phil Parkin - Wednesday, October 17, 2018 8:15 AM

    Jeff Moden - Wednesday, October 17, 2018 7:48 AM

    Data Cruncher - Tuesday, October 9, 2018 6:55 AM

    How do you handle shared SSIS projects?

    It's easy for me.  I write stored procedures to replace them. 😉  Only 2 more to go and then we can turn off SSIS.

    Can we set your username to SSIS Troll😛

    Jeff's going out on Halloween as SSIS Troll... but what does that look like exactly?

  • xsevensinzx - Thursday, October 18, 2018 6:08 PM

    Phil Parkin - Thursday, October 18, 2018 7:13 AM

    xsevensinzx - Thursday, October 18, 2018 6:47 AM

    I mean, that's not a bad suggestion. Anything you can do in SSIS, I can likely do the exact same with TSQL. 😀

    OK, here's a nice starter for you. What T-SQL commands would you use to grab Google Analytics data (which requires a web service login using OAuth 2.0)?

    TSQL + Python script. Easy.

    Need that CmdExec though. Save as CSV and can even use Jeff's favorite, BULK_INSERT. That or have Python call a script itself to write the data using a stored procedure.

    In the event you cannot do that due to security reasons or the engine does not support it, then you put the Python script on a Windows job to download at the frequency you need and use TSQL to ingest the data once it's available on a Agent job. Remember, Python contains all you need to authenticate with OAuth and Google. You can even use SQL tables to store the secret keys and retrieved via stored procedure called by the Python script separately. 

    Need something more challenging here. :laugh:

    This is cheating. You said "... with T-SQL", not "... with T-SQL, combined with any other technology I care to choose".

    The absence of evidence is not evidence of absence.
    Martin Rees

    You can lead a horse to water, but a pencil must be lead.
    Stan Laurel

  • Phil Parkin - Friday, October 19, 2018 5:56 AM

    xsevensinzx - Thursday, October 18, 2018 6:08 PM

    Phil Parkin - Thursday, October 18, 2018 7:13 AM

    xsevensinzx - Thursday, October 18, 2018 6:47 AM

    I mean, that's not a bad suggestion. Anything you can do in SSIS, I can likely do the exact same with TSQL. 😀

    OK, here's a nice starter for you. What T-SQL commands would you use to grab Google Analytics data (which requires a web service login using OAuth 2.0)?

    TSQL + Python script. Easy.

    Need that CmdExec though. Save as CSV and can even use Jeff's favorite, BULK_INSERT. That or have Python call a script itself to write the data using a stored procedure.

    In the event you cannot do that due to security reasons or the engine does not support it, then you put the Python script on a Windows job to download at the frequency you need and use TSQL to ingest the data once it's available on a Agent job. Remember, Python contains all you need to authenticate with OAuth and Google. You can even use SQL tables to store the secret keys and retrieved via stored procedure called by the Python script separately. 

    Need something more challenging here. :laugh:

    This is cheating. You said "... with T-SQL", not "... with T-SQL, combined with any other technology I care to choose".

    Laugh, it's not cheating. SSIS is not using one set of technology to do all of this either. It's called Integration Services for a reason. It's just calling to similar functionality you would call out to using T-SQL. I'm forcing the integration so to speak. Anything you can do in SSIS, I can likely do not using SSIS.

Viewing 4 posts - 16 through 18 (of 18 total)

You must be logged in to reply to this topic. Login to reply