Stored procedures and Creating tables. (Problem).

• ben.brugman

  SSChampion

  Points: 13369

  More actions

  August 18, 2016 at 1:28 pm

  #1896239

  Steve Jones - SSC Editor (8/16/2016)

  ---

  This is really fascinating. Ben, thanks for a great discussion.

  IF you or anyone else wanted to write an article, or series, on dealing with CSV data in a few specific ways, I think it would be great. I know I've had issues at times with CSVs that were problematic to load.

  Thanks,

  Allthough this will take some time, I think I can do that.

  For parsing a 'proper' CSV file, I have most code in place.

  I allready started with a 'grande' cleanup of the code so that it is presentable.

  At the moment the total amount of code is 'large' and sometimes specific for my situation.

  Also the code contains a huge amount of trial code and benchmarking code.

  Parts of the code are still hardcoded for my specific situation.

  Also I use a lot of selfmade stored procedures and a 'adjusted' splitlong (was split8k) procedure.

  Creating an article has to be done in my spare time.

  Also creating the code has to be done in my spare time.

  Because for me this is a large project, I will need some feedback.

  Feedback on my English (not my native language).

  Feedback on code/script constructions.

  Feedback on the actual working of the code.

  I think this feedback is needed to get a better quality article.

  Improvements of the code of others is very welcome. (elegance, performance, readability)

  So please contact me,

  and inform me how this is to be done.

  As said I think I can do this, also I think that this will result in usefull code for many.

  But I do not make any promises.

  Ben

  At the moment the code is large, it consists of a large number of components.

  The important ones are:

  Getting the filenames which are present in a folder.

  Reading the files present in the folder.

  Processing one file at the time.

  Parsing it in several stages.

  Creating a table from the headers in the file.

  Creating the table from the content of the file.

  A large number of 'smaller' components are correcting splits which should not have been made.

  Handling 'white space'.

  Adding of extra columns for 'linenumbers' and 'PK' alternative.

  Intelligent datatype creation.

  Automatic (guessing) the delimitors determination.

  Removing the last (empty) line.

  Choosing the line with 'headers', if available.

  Choosing the line with 'datatype', if available.

  Choosing the line for the start of the data. (So that header/datatype or other lines can be skipped).