Replacing BCP with SQLBulkLoad
-
Comments posted to this topic are about the content posted at http://www.sqlservercentral.com/columnists/spopovski/replacingbcpwithsqlbulkload.asp
-
This article assumes that XML is the way of the future, which is a false assumption.
XML has some significant problems.
1. Performance. I'm not talking about file size (and the resulting poor performance over networks), but about that fact that parsing XML is an intensive process. For one thing, you must read the entire data structure into memory because you must read to the closing tag. CSV has no such problems.
2. Interoperability. XML only works with new stuff and something that can parse XML. Sure it crosses platforms, but you'll find that CSV as a format is even more accepted.
Even one of the co-authors of the XML 1.0 standard has pointed out some problems with XML.
See
http://slashdot.org/article.pl?sid=03/03/18/0712248&mode=thread&tid=95&tid=156
That is not to say that XML doesn't have its place. Its place is in small applications that don't deal with much data, and/or have complicated data structures that need to be represented with XML.
My impression is that XML is touted as the wave of the future by an increasingly small number of people these days.
-
Im not much of an XML fan either, but in many cases the extra overhead is worth what you get out of it. Using the DOM requires you to read the entire file (an awesomely slow process with large files, I watched a demo where the memory usage ticked upward...and upward...forever), but SAX and maybe the .Net reader are designed to be lighter. CSV's are ok, but it's not very rich. My favorite data transfer format is MS Access!
Andy
-
Doesn't anyone translate these articles into proper English before posting them? The grammatical errors in this article drove me nuts.
-
This article could have been much better if it was edited and proof-read before posting. I gave up trying to decipher the spelling mistakes and errors, but I think the idea for the article was good.
I didn't know SQLXML had this functionality, but I wouldn't trust the code samples given in the article because of the poor English.
Edited by - cppwiz on 07/21/2003 08:53:54 AM
-
I got an error trying this. I 3 column table with hr, Source, and Description as the column headings:
hr
0x800401F3
Source
ODSOLE Extended Procedure
Description
Invalid class string
Thanks,
ThomBeaux
Thomas LeBlanc, MVP Data Platform Consultant
-
quote:
Inattention using master..xp_cmdshell can seriously endanger SQL Server securityThis article rightfully points out that you tend to avoid xp_cmdshell for security reasons, but then goes and uses the sp_OA* procedures, which you tend to avoid for security and stability reasons. Therefore, I don't think the author can cite security as a reason for using SQLBulkLoad over bcp.
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
Edited by - bkelley on 07/21/2003 09:34:06 AM
K. Brian Kelley
@kbriankelley -
My fault on the English - we haven't decided what to do just yet. Clearly if can't follow the article it's of no value, yet we don't have time to rewrite content that comes in from readers who use English as a second language.
Andy
-
I think the BULK INSERT statement is a good substitute to bcp. You avoid teh use of xp_cmdshell and also you avoid the overhead of having each field around tags. I personally think the best data interchange format is a pipe or comma delimited file. The question of portability doesn't arise as even with an XML file you still have to match the various tags. That said, XML does seem interesting and until now I haven't been convinced that it is the right format for transfer of large volume of data
-
quote:
I think the BULK INSERT statement is a good substitute to bcp.From a performance perspective, it should be slightly faster. BULK INSERT runs in the memory context of SQL Server whereas bcp does not. Both of these would be faster than DTS and the described method here.
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
K. Brian Kelley
@kbriankelley -
quote:
My fault on the English - we haven't decided what to do just yet. Clearly if can't follow the article it's of no value, yet we don't have time to rewrite content that comes in from readers who use English as a second language.Andy
No apologies needed. As an American that wishes to live in a WORLD community, I recognize that many of the people we have to work with will be using English as Second Language (ESL). Those of us in America need to be considerate and tolerant of those that make a great deal of effort to learn and understand our language.
A very interesting article in BusinessWeek a few months ago presented a story how an airline passenger from Taiwan (I think) could not make the stewardess understand her question in the best English she could muster. But a Japanese and Indonesian assisted, all using ESL, no one using their native languages, to make the stewardess understand the woman was asking where the restroom was. The point of the article is that most of the world uses ESL successfully in business and technology. But Americans have the most difficult time in both understanding and speaking with ESL people, because of our use of double-meaning words (e.g. right) or slang or idioms that have no sense in the rest of the world.
Bottom line, it is pure elitism to scoff at the open exchange of ideas because the elitist cannot take the time to read carefully and to understand what is said. Those elitists do not need apologies, but need scolding. This is a public forum, not a grammar contest.
Mark -
I have to agree with Mark on this one. I've watched people intentionally not listen to my mother because she was Japanese. It didn't matter that she was speaking perfectly clear English with only a hint of an accent. Because they said to themselves, "I won't understand her so I'm not going to even listen," they ended up not understanding her because they didn't listen.
My respect for the author writing an article in the first place. Those of us who do it on a regular basis know how hard it can be. It's that much harder when it's not in your native language.
K. Brian Kelley
http://www.truthsolutions.com/
Author: Start to Finish Guide to SQL Server Performance Monitoring
http://www.netimpress.com/shop/product.asp?ProductID=NI-SQL1
Edited by - bkelley on 07/21/2003 2:43:54 PM
K. Brian Kelley
@kbriankelley -
Well said. That reflects a lot of what we were thinking. The counterpoint (that we were thinking) is that technical material is hard enough to read/learn when written very clearly - obscure it and it becomes even harder.
Andy
-
Hi Brian,
quote:
My respect for the author writing an article in the first place. Those of us who do it on a regular basis know how hard it can be. It's that much harder when it's not in your native language.very well said!!!
In fact, that's what refrains me from submitting some article yet.
In addition, I'm at war with english punctuation. (to be honest, I'm with german to a certain degree, too
)Let' s look at this as a first try, which is always the hardest one.
And maybe as a suggestion to the SiteOwners, to review articles from non native english speakers before publishing. That means no proofreading or editing, just 'normal' reading.
Cheers,
Frank
--
Frank Kalis
Microsoft SQL Server MVP
Webmaster: http://www.insidesql.org/blogs
My blog: http://www.insidesql.org/blogs/frankkalis/[/url] -
Hi David,
quote:
This article assumes that XML is the way of the future, which is a false assumption.XML has some significant problems.
1. Performance. I'm not talking about file size (and the resulting poor performance over networks), but about that fact that parsing XML is an intensive process. For one thing, you must read the entire data structure into memory because you must read to the closing tag. CSV has no such problems.
funny anecdote on this happened yesterday.
I was in a project meeting with our IT staff. Basically to discuss how to get mutual fund quotes into our mainframe.
When it comes to the 'which format' question, I said with your post in mind, hey, let's do it with XML. It's universal, trendy blabla..
Before going on, I must say, I really like our mainframe guys. They are innovative, flexible, open for new thoughts.
Guess what they said!
No, no XML, only CSV, that just fine. We can reuse existing code.. and anyway who cares about XML.
Well!
Cheers,
Frank
--
Frank Kalis
Microsoft SQL Server MVP
Webmaster: http://www.insidesql.org/blogs
My blog: http://www.insidesql.org/blogs/frankkalis/[/url]
)Viewing 15 posts - 1 through 15 (of 17 total)
You must be logged in to reply to this topic. Login to reply