August 27, 2014 at 7:50 am
This was brought up in the automate everything article, so I figured I start a new thread on the topic.
Anyone here utilizing Python with SQL Server?
I ask simply because I use it a great deal. Python has been my go to language when piping data between platforms to my SQL Server instance. It's also been my go to language for API's from third-party vendors when piping in new data sources into my databases. And now, I'm starting to dive into the statistical packages of Python because it integrates so easily over R.
I would love to hear about any others using Python and what good or bad experiences they had with it.
August 27, 2014 at 8:31 am
We use Python to do all kinds of things, but most of our Python is used to automate ARC GIS processing.
August 27, 2014 at 8:54 am
Does anyone else in your shop have such a good handle on Python? If not, what will they do if you leave or get hit by the proverbial bus?
--Jeff Moden
Change is inevitable... Change for the better is not.
August 27, 2014 at 9:01 am
I saw IronPython a few years ago and it looked really interesting, especially with stats work. I haven't done much more than play with it, but I found Powershell to do lots of what I might do with it (non-stats related), so I play with that.
August 27, 2014 at 9:01 am
Yes, I'm a beginner, but we have several other Python users with varying skills in Python.
August 27, 2014 at 2:13 pm
Jeff Moden (8/27/2014)
Does anyone else in your shop have such a good handle on Python? If not, what will they do if you leave or get hit by the proverbial bus?
It's just me over here. I'm the DBA, developer, data scientist, floor sweeper, all-in-one guy. So no, no one else. I use it mainly due to the simplicity and the power behind it. I also use it because I use to work in the game industry where Python was used in a large number of backend systems in a number of AAA online games. Seemed like a good option to utilize being a few data nerd guys I knew also used it with Hadoop and R.
But then again, I rarely used Microsoft technologies until I fell in love with SQL Server. 😉
September 4, 2014 at 8:13 am
Does anyone else in your shop have such a good handle on Python? If not, what will they do if you leave or get hit by the proverbial bus?
Python programmers aren't hard to find, and most code is fairly easy to read and maintain. It's more of OSS/Linux programming language, but there's a large number of users on Windows and almost all of the third party libraries work on Windows as well.
I use Python for data analysis and utilities, it's another tool in my kit alongside Visual Studio and SQL Server. I'm also co-leader of the local Python User Group.
November 20, 2014 at 4:36 pm
I use python with SQL Server extensively. In fact, I have written about it before at https://www.simple-talk.com/sql/database-administration/python-for-the-sql-server-dba/ and I have used in in some of the examples for articles I have written here.
I primarily use pyodbc to interface SQL Server and my Python code, but that is not the only option out there.
---
Timothy A Wiseman
SQL Blog: http://timothyawiseman.wordpress.com/
December 21, 2014 at 3:48 am
I use Python for everything (related to administration) and most of the ETL stuff (using xsd and XML Bulk Load (SQLXML).
January 10, 2015 at 3:37 am
February 8, 2015 at 7:35 pm
I am using Python
February 9, 2015 at 8:48 am
I was involved in a project a month or so ago where we wanted to use edit distance (i.e. Levenshtein) to identify potential duplicates among a couple of million news articles pulled from the Web.
Since the data was stored in SQL Server and we had both a native T-SQL and a CLR-based Levenshtein function at our disposal we tried running the analysis using a T-SQL query. However, the performance was an impediment (using either version of the function).
I didn't have much experience with Python but I know some ETL engineers who consider it an important tool in their arsenal. So, I downloaded a few libraries and threw together a script in an hour or so. The performance was vastly improved (can't remember exactly, but I think the Python script was processing records about 5X faster than T-SQL, maybe more).
The biggest issue was getting all the ODBC configurations (pyodbc, FreeTDS, etc.) working on my Mac; but now that they're all playing nice I can see a strong argument for using Python scripts for data tasks that are not in T-SQL's wheel house.
February 16, 2015 at 9:52 am
I started using python about 6 months ago as we started using Amazon Web Services to support some of our online sites as well as the traditional onsite SQL servers, Web servers etc.
As a DBA I find it a great language to program in, there is so many modul;es out there to plug in and give you a broad range of functionality without coding it all yourself. Traditionally we used powershell, SSIS, vb, C# and command line programming to achieve all of our automation type work. But have started moving to python for everything.
The boto SDK for AWS is especially good, allowing us to build environments using all the different AWS services from single scripts that utilise custom modules we have created.
I even created a few scripts to automate data replication from our onsite SQL Servers to our AWS RDS MySQL instances and back using python. As the systems have no direct connection we can run the scripts from an intermediary server that can connect to both and do all the work required, hooking into S3 storage.
MCITP SQL 2005, MCSA SQL 2012
January 17, 2018 at 11:48 pm
This was removed by the editor as SPAM
January 18, 2018 at 6:17 am
Spam reported.
--Jeff Moden
Change is inevitable... Change for the better is not.
Viewing 15 posts - 1 through 15 (of 18 total)
You must be logged in to reply to this topic. Login to reply