January 4, 2016 at 4:57 pm
Eric,
Thanks for the insight.
Microsoft always seems under pressure to limit its free solutions in different ways over time. It is often hard to understand the logic of it unless someone walks you through it.
412-977-3526 call/text
January 5, 2016 at 8:49 am
The following is a recent (2014) article comparing features of SSCE (3.5 and 4), SQL Server Express, and SQL Server LocalDB to SQLLite.
http://erikej.blogspot.com/2011/01/comparison-of-sql-server-compact-4-and.html
There are JDBC (Java) providers for SQL Server, one provided by Microsoft in addition to a handful of 3rd party implementations, but they don't support SQL Server Compact Edition (.SDF files). If there were, then that would change the picture in terms of user base and practical application of SSCE in the embedded device and zero footprint space.
"Do not seek to follow in the footsteps of the wise. Instead, seek what they sought." - Matsuo Basho
January 7, 2016 at 11:24 am
SQLite is included with the default install of Python, so it makes for a easy data store when doing various tasks. I like to use it instead of Pandas for some data analysis use cases.
January 7, 2016 at 11:41 am
Interesting Chris!
So you just use the aggregations and grouping that are available in SQLite vs. pandas for some use cases? I'm just getting into using pandas, but have been using Python for a while. Still, sometimes I know I can write the T-SQL to aggregate and group on more quickly than I know how to do similar actions in pandas, and why bring the data out of the DB if the T-SQL is performing OK?
Do you use the native sqlite3 package or do you use an external package like SQLAlchemy?
Thanks!
Matt
January 7, 2016 at 3:33 pm
There are so many NOSQL solutions that it is easier to say what NODQL is not than what it is.
As a data architect one of my great frustrations was solutions that failed to consider what the end to end picture looked like. I can't blame developers because they get charged with building a point solution with tightly defined scope.
The problem comes when the data is needed in a downstream business process. I have rarely seen any consideration given to such a use case. The most I can hope for is for the front end system replicating its data as is to a backend staging area.
There seems to be an assumption that an automagical process will turn an ever increasing volume of data into a useable data asset to feed a BI facility. This leads to convoluted processes that are really coping strategies for poorly thought out data architecture
January 8, 2016 at 11:01 am
So you just use the aggregations and grouping that are available in SQLite vs. pandas for some use cases? I'm just getting into using pandas, but have been using Python for a while. Still, sometimes I know I can write the T-SQL to aggregate and group on more quickly than I know how to do similar actions in pandas, and why bring the data out of the DB if the T-SQL is performing OK?
It depends on the task and the data. Sometimes it's easier to create a few tables in SQLite and do a few queries then it is to deal with pandas. I know a lot more SQL and Python than I know pandas. It's when you want to do advanced stats and pretty charts, iPython and pandas wins.
January 8, 2016 at 5:55 pm
Ok, cool. Thanks Chris, that has been my approach as well. Glad I was not too far off the beaten path.
January 18, 2016 at 4:55 am
Definitely one to bear in mind. Often forgotten. Ever present.
Gaz
-- Stop your grinnin' and drop your linen...they're everywhere!!!
Viewing 8 posts - 16 through 22 (of 22 total)
You must be logged in to reply to this topic. Login to reply