Exif data into a database.

  • ben.brugman - Monday, November 26, 2018 3:40 AM

    Eirikur Eiriksson - Friday, November 23, 2018 7:58 AM

    I must ask why you want to do this?
    😎
    The EXIF specs are 148 pages, you might have to spend the next few days (and nights) modelling it to fit into a relational database:pinch:

    Thanks for the document, did go through it. This is very interresting, but not needed for using the exif information. (But still interesting, thank you).

    Why?
    There is a 'shit'-load of information available in the Exif data. Having photography as a hobby, this is interresting information too me. Using a (SQL-server) database makes this information more accesseble. With lightroom you can make filters on the data to see the pictures (and how many) are 'left' after the filtering. With a database you can do so much more, combinations of the 'parameters' for example, to study the settings of large sets of pictures.

    One important goal is to collect the data of a large number of media and combine them in a single database. So that I can explore which pictures are not allready in my masterset of pictures. And then 'discard' the media which I do not have need anymore for. (I have a collection of CD-roms, flash drives and others containing backups of pictures and other pictures.) Going through them informs me if my master set is complete.

    Other questions are what are the lenses and settings which get used and how much. And how do I (and others) take pictures over time.

    Remark: Photography is a hobby, both the pictures and the meta (exif) data is totaly mostly insignificant. (But it's a hobby).

    My question to you is:
    Why the obsession with the WHY question? Very often this question is asked. No problem, but af the WHY is anwsered, the original question keeps hanging in the air. The impression I sometimes get is the the 'WHY' question is asked to avoid giving an anwser to the question. Or as 'a suggestion' that the you should not want what you asked for.

    Ben

    I suspected that you were doing photography, just about the only reasonable justification on embarking this kind of task 😉
    😎

    The reason for the WHY question is that it gives an insight, which normally is not transparent or obvious from the initial question.
    I have done similar in the passed for different purposes such as hobby photography, advertising, medical images etc., not too unfamiliar with the subject. 
    What are you using to extract the EXIF?

  • Eirikur Eiriksson - Monday, November 26, 2018 4:15 AM

    I suspected that you were doing photography, just about the only reasonable justification on embarking this kind of task 😉   😎
    What are you using to extract the EXIF?

    In the first message of this thread:
    -- The tool:
    -- https://www.sno.phy.queensu.ca/~phil/exiftool/

    -- Example for calls to the exiftool
    exiftool -csv -r -ext jpg J:\ >exif1_jpg.csv
    exiftool -csv -r -ext jpg C:\ >exif2_jpg.csv

    So the exif is for hobby only.
    For getting .CSV information in the database, in my direct surrounding, I have seen a dot net solution (1) based on Excel and .CSV files. This was dependend on the version of Excel and the region setting of the machine. And the performance was horrible. Somebody worked on this for 3 months and the result was 'shit'.
    I did build a version completely in SQL-server (2). There are limitations (2 Giga, not all characters set are correctly processed, works at about 100 K a second, can do EOL and delimitors, and some other stuf. File must be on the database machine.)
    Another college of my build a 'simple' version (3) that did not depend on 'SSMS' and did not need access to the database machine, but came with other (?) restrictions. (And I think it was not suetable for large files ???).

    So seeing that in my firm alone there are three (1)(2)(3) not completely satisfactory solutions, where also the Intelectual Property belongs to the firm. I would not be supprised that there are hundreds of solutions for the same problem. So I AM supprised that there is no <generic> solution available (as far as I know).

    The exif file set (from pictures) with a predictable structure (textqualifier is known, delimitor is known and <EOL> character is known), but with unpredictable columns (number of fields and which fields) was a very nice example to show the 'general' problem of getting .CSV's into a database. This is an example which does not te take to much knowledge to reproduce. (Everybody has pictures, access to the exiftool is available and most here know SQL-server and SSMS). So this is a very nice  example problem, to show general .CSV access problems.

    At my work I am in a bit of a lull, so have some time to spend a bit more of time on these types of problems. My hobby came with a nice example (the exif) example to address a problem which in my believe does occure regulary.

    Ben

    One problem I have not solved yet. Outputting a .CSV file from data using SSMS, which can be used for input in the solutions. (Problem is the output is in Unicode and the methods are all geared to single byte of UTF-8 character sets).

  • ben.brugman - Monday, November 26, 2018 5:23 AM

    Eirikur Eiriksson - Monday, November 26, 2018 4:15 AM

    I suspected that you were doing photography, just about the only reasonable justification on embarking this kind of task 😉   😎
    What are you using to extract the EXIF?

    In the first message of this thread:
    -- The tool:
    -- https://www.sno.phy.queensu.ca/~phil/exiftool/

    -- Example for calls to the exiftool
    exiftool -csv -r -ext jpg J:\ >exif1_jpg.csv
    exiftool -csv -r -ext jpg C:\ >exif2_jpg.csv

    Have you looked into using the option of exporting the EXIF as an XML? Cannot remember if Phil's tool does that, have used it many times but also many years ago.
    😎
    Much of the work I've done on this has been on Linux, latest applications using DarkTable which can produce the EXIF as an XML.

  • This was removed by the editor as SPAM

  • This was removed by the editor as SPAM

Viewing 5 posts - 16 through 19 (of 19 total)

You must be logged in to reply to this topic. Login to reply