Working with XML OPENXML vs nodes()

  • I am moving a discussion from a non-technical thread to here.  The question is how to use nodes() for processing XML data for import into SQL Server tables rather than OPENXML as it has been reported that OPENXML could be more resource intensive than using nodes().
    Starting with the following as the only way I could get nodes() to work to shred a sample (yet confidential) xml file to be shredded using nodes().  I had to eliminate the namespace information at the top of the XML.  The information I found in BOL was not helpful to me in trying to figure out how to incorporate namespaces I with nodes().

    The problem that I was running into with using nodes was getting it work with the XML data using namespaces, and the information in BOL really didn't help.  If I pulled out the namespace information in the XML file, I was able to use nodes to shred the xml.  With OPENXML is was actually easy to figure out the namespace usage where the namespace name is cidneNS.

    I am posting the opening of the top level XML here to show what I am working with regarding namespaces.  Understand I don't have control over this definition, I just have to work with it.
    What I start with:

    <ExploitationUXOReports xsi:schemaLocation="urn:cidneNS http://localhost/ws/xsds/2.1.8/CIDNE-Exploitation-UXO-2.1.8_WS_ENUM.xsd" xmlns="urn:cidneNS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">


    What it looks like to work with nodes:

    <ExploitationUXOReports>


  • The other issue I did see is that if node elements were missing the nodes() returned empty strings for character columns, 0 for integer columns, 1900-01-01 for date columns all of which is valid but if the element was missing, it could be false data.  Whereas OPENXML returns NULL which for some elements is actually valid in the sense the it is unknown at the time the XML data was created or is from a system where that data doesn't exist and will be added later in our system.  For some columns we have default values that I can substitute for the null values during the processing of the xml data.

  • Lynn,

    Can you look at your XML data in the GUI to get an idea of what nodes you'll need to query and how far down in the nestings you'll need to go? EDIT: I don't know if a separate document is openable in SSMS, but maybe you have another way of getting this information so you can see if you need one, two, or no namespaces.

    I don't know if the code I did as part of my Database Dependents Part 2 article will help you at all, but the namespace I used for that was pretty simple. Just "WITH XMLNAMESPACES ('www.microsoft.com/SqlServer/Dts'AS DTS)". The namespaces are great for identifying distinct node sections of the document you're planning to shred. In my article, I did a query with one namespace and another with two so that I could grab the nodes of two different nested sections. It worked great.

    On the other hand, when I'm querying an XML data type from a table, I don't even use namespaces. I do something like:


    SELECT *
    FROM (SELECT T.c.value('FirstColumnName[1]','varchar(20)') AS FirstColumnName,
                               T.c.value('SecondColumnName[1]','int') AS SecondColumnName
              FROM MyTable
              CROSS APPLY XMLDataColumn.nodes('row') T(c) 
             ) a
    WHERE FirstColumnName = 'SkippyOrJif';

    Second Edit: Is it possible to import the document as an XML data type and then query it from a table? It may be easier or more complicated, depending on how big the document is.

    Brandie Tarvin, MCITP Database AdministratorLiveJournal Blog: http://brandietarvin.livejournal.com/[/url]On LinkedIn!, Google+, and Twitter.Freelance Writer: ShadowrunLatchkeys: Nevermore, Latchkeys: The Bootleg War, and Latchkeys: Roscoes in the Night are now available on Nook and Kindle.

  • To extract nodes from an xml with namespaces, look at this site: xmlnamespaces,
    I can't try with your namespace without sample data
    Regarding missing nodes, using the value query with nodes should give a NULL, not empty strings or zero's.

  • Basically It's a two step process:
    1.  Affix the query that has the 
                                  xmlcolumn.node('/somexpath','type')
      syntax in it with an appropriate  WITH XMLNAMESPACES expression
    2. incorporate the namespace alias into your query or use wildcards

    Example:


    declare @x xml;

    set @x='<ExploitationUXOReports
         xsi:schemaLocation="urn:cidneNS http://localhost/ws/xsds/2.1.8/CIDNE-Exploitation-UXO-2.1.8_WS_ENUM.xsd"

           
          xmlns="urn:cidneNS"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <test> this is fun</test>
    <test> woohoo</test>
    <test> still here</test>
    <test> bored yet?</test>
    </ExploitationUXOReports>';

     

    WITH XMLNAMESPACES (
                     
    'urn:cidneNS' as testalias,
                     
    'http://www.w3.org/2001/XMLSchema-instance' as xsi)
    select a.value('.','varchar(50)') testval
    from @x.nodes('//testalias:test') x(a); --with specific namespace alias

    WITH XMLNAMESPACES ( 'urn:cidneNS' as testalias,
    'http://www.w3.org/2001/XMLSchema-instance' as xsi)

    select a.value('.','varchar(50)') testval

    from @x.nodes('//*:test') x(a)   --with namespace wildcard

    using the wildcard is obviously a bit of a gamble if you have lots of namespaces, since it will resolve to the  any elements matching your criteria.

    Notes: 
    1.In order to populate the WITH XMLNAMESPACES, you're looking to put in an alias for each "xmlns" entry you find. 
    2.  when you see   xmlns:<whatever>  , the <whatever> is the alias provided by the SOURCE system.  You could reuse those, or not.
    3.  the xmlns with no trailing alias is the "default" alias.  You can certainly replicate that choice as well if you wish, or just give it your own value and refer to it using yours.

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

  • The requirement for the XML coming from external sources is that the XML must include the cidneNS namespace.  If this namespace is not included in the XML (as I showed) it will be rejected.
    This namespace is used for all elements.
    The following is a sample XML sans all data.  There are additional sub-tables for the main report that may or may not be present in other files as well some of the sub-tables represented in this file could be missing in other files as many of them are optional.  In addition there could be additional columns for each table that may not be represented in any given XML file.  I need to account for all the columns as many are optional and are not required.  Some of these may also have defaults if not provided but as even the current process inserts null values where these defaults should actually be used but the column, though not in the XML file, is included in the actual INSERT into the database.  I can code the process to use a defined default if the value to be inserted is NULL.  There are other changes that I have found to work with such as if the TheaterFilter is null or empty to use the default TheaterFilter for the server the file is being imported.  Use this TheaterFilter or the TheaterFilter provided in the many report table for the TheaterFilter in all sub-tables even if different in the XML file (that is the requirement).  I know how to go back to the main element where it should be used instead of the element in sub-portion being processed.  And the beauty is it works with multiple reports being processed (I duplicated a single report in a file and made changes to the duplicated data so I could ensure data flowed appropriately).


    <ExploitationUXOReports xsi:schemaLocation="urn:cidneNS http://localhost/ws/xsds/2.1.8/CIDNE-Exploitation-UXO-2.1.8_WS_ENUM.xsd" xmlns="urn:cidneNS" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
      <ExploitationUXO>
        <AimingPointDetails></AimingPointDetails>
        <Analysis></Analysis>
        <CallSign></CallSign>
        <CivilianActivity></CivilianActivity>
        <CivilianKIA></CivilianKIA>
        <CivilianWIA></CivilianWIA>
        <Classification></Classification>
        <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
        <CoalitionKIA></CoalitionKIA>
        <CoalitionWIA></CoalitionWIA>
        <DateDiscovered></DateDiscovered>
        <DateDiscoveredZulu></DateDiscoveredZulu>
        <DateInitialPublished></DateInitialPublished>
        <DatePosted></DatePosted>
        <DateUpdated></DateUpdated>
        <displaySerial></displaySerial>
        <DisplaySummary></DisplaySummary>
        <DisplayTitle></DisplayTitle>
        <EODFOBDeparture></EODFOBDeparture>
        <EODMissionComplete></EODMissionComplete>
        <EODNotificationDTG></EODNotificationDTG>
        <EODSceneArrival1></EODSceneArrival1>
        <EODSceneDeparture></EODSceneDeparture>
        <EODTeam></EODTeam>
        <FiringPointDetails></FiringPointDetails>
        <FiringPointDetailsMGRS></FiringPointDetailsMGRS>
        <FPDisMGRS></FPDisMGRS>
        <HostNationKIA></HostNationKIA>
        <HostNationWIA></HostNationWIA>
        <IEDEvent></IEDEvent>
        <isDeprecated></isDeprecated>
        <LastTimeRouteCleared></LastTimeRouteCleared>
        <LightConditions></LightConditions>
        <MarkingDetails></MarkingDetails>
        <MGRS></MGRS>
        <OriginatingSite></OriginatingSite>
        <OriginatingSystem></OriginatingSystem>
        <OriginatorGroup></OriginatorGroup>
        <OriginatorName></OriginatorName>
        <OriginatorUnit></OriginatorUnit>
        <PrimarySafeAreaMGRS></PrimarySafeAreaMGRS>
        <PSAisMGRS></PSAisMGRS>
        <ReferencePoint></ReferencePoint>
        <ReleasableDataClassification></ReleasableDataClassification>
        <ReleasableDataClassificationReleasabilityMark></ReleasableDataClassificationReleasabilityMark>
        <ReportingUnit></ReportingUnit>
        <ReportKey></ReportKey>
        <ReportType></ReportType>
        <RoutePatrolInterval></RoutePatrolInterval>
        <SecondarySafeAreaMGRS></SecondarySafeAreaMGRS>
        <ServiceAffiliation></ServiceAffiliation>
        <SiteMarked></SiteMarked>
        <SSAisMGRS></SSAisMGRS>
        <TrackingNumber></TrackingNumber>
        <UpdatedByGroup></UpdatedByGroup>
        <Weather></Weather>
        <TheaterFilter></TheaterFilter>
        <Device>
          <Classification</Classification>
          <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
          <DatePosted></DatePosted>
          <DateUpdated></DateUpdated>
          <DeviceKey></DeviceKey>
          <FirstResponderTargeted></FirstResponderTargeted>
          <HowDeviceWasFound></HowDeviceWasFound>
          <isDeprecated></isDeprecated>
          <OriginatingSite></OriginatingSite>
          <OriginatingSystem></OriginatingSystem>
          <OriginatorGroup></OriginatorGroup>
          <OriginatorName></OriginatorName>
          <OriginatorUnit></OriginatorUnit>
          <PartialBuild></PartialBuild>
          <Placement></Placement>
          <PostBlast></PostBlast>
          <ReleasableDeviceClassification></ReleasableDeviceClassification>
          <ReleasableDeviceClassificationReleasabilityMark></ReleasableDeviceClassificationReleasabilityMark>
          <ReportKey></ReportKey>
          <SafeArea></SafeArea>
          <Suicide></Suicide>
          <UpdatedByGroup></UpdatedByGroup>
          <Munitions>
            <Category></Category>
            <Classification></Classification>
            <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
            <Country></Country>
            <DatePosted></DatePosted>
            <DateUpdated></DateUpdated>
            <Details></Details>
            <DeviceKey></DeviceKey>
            <isDeprecated></isDeprecated>
            <ItemDescription></ItemDescription>
            <ItemMarkings></ItemMarkings>
            <LotNumber></LotNumber>
            <Manufacturer></Manufacturer>
            <Measure></Measure>
            <MunitionKey></MunitionKey>
            <MunitionsType></MunitionsType>
            <Name></Name>
            <OriginatingSite></OriginatingSite>
            <OriginatingSystem></OriginatingSystem>
            <OriginatorGroup></OriginatorGroup>
            <OriginatorName></OriginatorName>
            <OriginatorUnit></OriginatorUnit>
            <Packaging></Packaging>
            <Quantity></Quantity>
            <ReleasableMunitionsClassification></ReleasableMunitionsClassification>
            <ReleasableMunitionsClassificationReleasabilityMark></ReleasableMunitionsClassificationReleasabilityMark>
            <ReportKey></ReportKey>
            <SerialNumber></SerialNumber>
            <ShippingDocuments></ShippingDocuments>
            <SizeUnit></SizeUnit>
            <UpdatedByGroup></UpdatedByGroup>
            <Fusing>
              <Classification></Classification>
              <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
              <DatePosted></DatePosted>
              <DateUpdated></DateUpdated>
              <FusingCountry></FusingCountry>
              <FusingDetails></FusingDetails>
              <FusingKey></FusingKey>
              <isDeprecated></isDeprecated>
              <MunitionKey></MunitionKey>
              <OriginatingSite></OriginatingSite>
              <OriginatingSystem></OriginatingSystem>
              <OriginatorGroup></OriginatorGroup>
              <OriginatorName></OriginatorName>
              <OriginatorUnit></OriginatorUnit>
              <ReportKey></ReportKey>
              <UpdatedByGroup></UpdatedByGroup>
            </Fusing>
          </Munitions>
          <RenderSafe>
            <Classification></Classification>
            <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
            <DatePosted></DatePosted>
            <DateUpdated></DateUpdated>
            <DeviceKey></DeviceKey>
            <isDeprecated></isDeprecated>
            <OriginatingSite></OriginatingSite>
            <OriginatingSystem></OriginatingSystem>
            <OriginatorGroup></OriginatorGroup>
            <OriginatorName></OriginatorName>
            <OriginatorUnit></OriginatorUnit>
            <RenderSafeDetails></RenderSafeDetails>
            <RenderSafeKey></RenderSafeKey>
            <ReportKey></ReportKey>
            <RSP></RSP>
            <UpdatedByGroup></UpdatedByGroup>
          </RenderSafe>
        </Device>
        <Location>
          <Classification></Classification>
          <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
          <DatePosted></DatePosted>
          <DateUpdated></DateUpdated>
          <isDeprecated></isDeprecated>
          <LocationKey></LocationKey>
          <MGRS></MGRS>
          <OriginatingSite></OriginatingSite>
          <OriginatingSystem></OriginatingSystem>
          <OriginatorGroup></OriginatorGroup>
          <OriginatorName></OriginatorName>
          <OriginatorUnit></OriginatorUnit>
          <ReportKey></ReportKey>
          <UpdatedByGroup></UpdatedByGroup>
        </Location>
        <RenderMain>
          <Classification></Classification>
          <ClassificationReleasabilityMark></ClassificationReleasabilityMark>
          <DatePosted></DatePosted>
          <DateUpdated></DateUpdated>
          <EquipmentUsedName></EquipmentUsedName>
          <EquipmentUsedType></EquipmentUsedType>
          <isDeprecated></isDeprecated>
          <OriginatingSite></OriginatingSite>
          <OriginatingSystem></OriginatingSystem>
          <OriginatorGroup></OriginatorGroup>
          <OriginatorName></OriginatorName>
          <OriginatorUnit></OriginatorUnit>
          <RenderSafeKey></RenderSafeKey>
          <ReportKey></ReportKey>
          <UpdatedByGroup></UpdatedByGroup>
        </RenderMain>
      </ExploitationUXO>
    </ExploitationUXOReports>

    This is the code generated dynamically to process the main report from the XML shown above using the OPENXML.  You can see that there are a lot of columns in the table not represented in the XML.  As each report could have different columns represented even within a single file, I have to use this shotgun approach.  The current processing, also using OPENXML, is written in RBAR fashion where each table and sub-table is processed one row at a time.  I am moving this to a set based process such that if there are multiple reports in a single file each table has all data inserted at once instead of one row at a time in each table for each report.

    select
      TableName = @Table -- debug code added to show me the table currently being processed
      , [AimingPointDetails]
      , [Analysis]
      , [CallSign]
      , [CFCIVABD]
      , [CFCIVKIA]
      , [CFCIVWIA]
      , [CivilianActivity]
      , [CivilianKIA]
      , [CivilianWIA]
      , [Classification]
      , [ClassificationAlias]
      , [ClassificationCaveat]
      , [ClassificationCodeWord]
      , [ClassificationDeclassifyOn]
      , [ClassificationDomain]
      , [ClassificationReleasabilityMark]
      , [ClassificationReleasabilityMarkAlias]
      , [CoalitionKIA]
      , [CoalitionWIA]
      , [DateDiscovered]
      , [DateDiscoveredZulu]
      , [DateInitialPublished]
      , [DateInitialPublishedZulu]
      , [DatePosted]
      , [DatePostedZulu]
      , [DateUpdated]
      , [DateUpdatedZulu]
      , [DisplayDateZulu]
      , [DisplayIcon]
      , [displaySerial]
      , [DisplaySummary]
      , [DisplayTitle]
      , [EODFOBDeparture]
      , [EODFOBDepartureZulu]
      , [EODMissionComplete]
      , [EODMissionCompleteZulu]
      , [EODNotificationDTG]
      , [EODNotificationDTGZulu]
      , [EODSceneArrival1]
      , [EODSceneArrival1Zulu]
      , [EODSceneArrival2]
      , [EODSceneArrival2Zulu]
      , [EODSceneDeparture]
      , [EODSceneDepartureZulu]
      , [EODTeam]
      , [EquipmentUsedName]
      , [EquipmentUsedType]
      , [EventCategory]
      , [FiringPointDetails]
      , [FiringPointDetailsMGRS]
      , [FPDisMGRS]
      , [FPDLatitude]
      , [FPDLongitude]
      , [HostNationKIA]
      , [HostNationWIA]
      , [IEDEvent]
      , [isDeprecated]
      , [IsTearLine]
      , [LanguagesKey]
      , [LastTimeRouteCleared]
      , [Latitude]
      , [LightConditions]
      , [Longitude]
      , [MarkingDetails]
      , [MGRS]
      , [Narrative]
      , [OriginatingNation]
      , [OriginatingNetwork]
      , [OriginatingSite]
      , [OriginatingSystem]
      , [OriginatorGroup]
      , [OriginatorName]
      , [OriginatorUnit]
      , [PrimarySafeAreaMGRS]
      , [PSAisMGRS]
      , [PSALatitude]
      , [PSALongitude]
      , [ReferencePoint]
      , [ReleasableDataClassification]
      , [ReleasableDataClassificationReleasabilityMark]
      , [ReportingUnit]
      , [ReportKey]
      , [ReportType]
      , [RoutePatrolInterval]
      , [SafeAreaMGRS]
      , [SecondarySafeAreaMGRS]
      , [ServiceAffiliation]
      , [SigactReportKey]
      , [SigactTrackingNumber]
      , [SiteMarked]
      , [SSAisMGRS]
      , [SSALatitude]
      , [SSALongitude]
      , [TeamLeaderComments]
      , [TearLineClassification]
      , [TearLineContact]
      , [TearLineReleasability]
      , [TearLineSourceReportDateUpdated]
      , [TearLineSourceReportKey]
      , [TheaterFilter] = ISNULL(NULLIF([TheaterFilter],''),@DefTheaterFilter)
      , [Title]
      , [TrackingNumber]
      , [UpdatedByGroup]
      , [UpdatedByName]
      , [UpdatedByUnit]
      , [Weather]
      , [WITAnalysis]
      , [WITComments]
      , [WITDTGArrival]
      , [WITDTGArrivalZulu]
      , [WITDTGDeparture]
      , [WITDTGFOBDeparture]
      , [WITDTGMissionComplete]
      , [WITDTGNotification]
      , [WITDTGReportComplete]
      , [WITEvidence]
      , [WITTeam]
    from openxml(@hdoc,N'cidneNS:ExploitationUXOReports/cidneNS:ExploitationUXO')
    with (
      [AimingPointDetails] nvarchar(max) 'cidneNS:AimingPointDetails'
      , [Analysis] nvarchar(max) 'cidneNS:Analysis'
      , [CallSign] nvarchar(100) 'cidneNS:CallSign'
      , [CFCIVABD] int 'cidneNS:CFCIVABD'
      , [CFCIVKIA] int 'cidneNS:CFCIVKIA'
      , [CFCIVWIA] int 'cidneNS:CFCIVWIA'
      , [CivilianActivity] nvarchar(100) 'cidneNS:CivilianActivity'
      , [CivilianKIA] int 'cidneNS:CivilianKIA'
      , [CivilianWIA] int 'cidneNS:CivilianWIA'
      , [Classification] nvarchar(50) 'cidneNS:Classification'
      , [ClassificationAlias] nvarchar(50) 'cidneNS:ClassificationAlias'
      , [ClassificationCaveat] nvarchar(500) 'cidneNS:ClassificationCaveat'
      , [ClassificationCodeWord] nvarchar(50) 'cidneNS:ClassificationCodeWord'
      , [ClassificationDeclassifyOn] nvarchar(50) 'cidneNS:ClassificationDeclassifyOn'
      , [ClassificationDomain] nvarchar(1) 'cidneNS:ClassificationDomain'
      , [ClassificationReleasabilityMark] nvarchar(500) 'cidneNS:ClassificationReleasabilityMark'
      , [ClassificationReleasabilityMarkAlias] nvarchar(500) 'cidneNS:ClassificationReleasabilityMarkAlias'
      , [CoalitionKIA] int 'cidneNS:CoalitionKIA'
      , [CoalitionWIA] int 'cidneNS:CoalitionWIA'
      , [DateDiscovered] datetime 'cidneNS:DateDiscovered'
      , [DateDiscoveredZulu] datetime 'cidneNS:DateDiscoveredZulu'
      , [DateInitialPublished] datetime 'cidneNS:DateInitialPublished'
      , [DateInitialPublishedZulu] datetime 'cidneNS:DateInitialPublishedZulu'
      , [DatePosted] datetime 'cidneNS:DatePosted'
      , [DatePostedZulu] datetime 'cidneNS:DatePostedZulu'
      , [DateUpdated] datetime 'cidneNS:DateUpdated'
      , [DateUpdatedZulu] datetime 'cidneNS:DateUpdatedZulu'
      , [DisplayDateZulu] datetime 'cidneNS:DisplayDateZulu'
      , [DisplayIcon] nvarchar(max) 'cidneNS:DisplayIcon'
      , [displaySerial] nvarchar(max) 'cidneNS:displaySerial'
      , [DisplaySummary] nvarchar(max) 'cidneNS:DisplaySummary'
      , [DisplayTitle] nvarchar(max) 'cidneNS:DisplayTitle'
      , [EODFOBDeparture] datetime 'cidneNS:EODFOBDeparture'
      , [EODFOBDepartureZulu] datetime 'cidneNS:EODFOBDepartureZulu'
      , [EODMissionComplete] datetime 'cidneNS:EODMissionComplete'
      , [EODMissionCompleteZulu] datetime 'cidneNS:EODMissionCompleteZulu'
      , [EODNotificationDTG] datetime 'cidneNS:EODNotificationDTG'
      , [EODNotificationDTGZulu] datetime 'cidneNS:EODNotificationDTGZulu'
      , [EODSceneArrival1] datetime 'cidneNS:EODSceneArrival1'
      , [EODSceneArrival1Zulu] datetime 'cidneNS:EODSceneArrival1Zulu'
      , [EODSceneArrival2] datetime 'cidneNS:EODSceneArrival2'
      , [EODSceneArrival2Zulu] datetime 'cidneNS:EODSceneArrival2Zulu'
      , [EODSceneDeparture] datetime 'cidneNS:EODSceneDeparture'
      , [EODSceneDepartureZulu] datetime 'cidneNS:EODSceneDepartureZulu'
      , [EODTeam] nvarchar(100) 'cidneNS:EODTeam'
      , [EquipmentUsedName] nvarchar(100) 'cidneNS:EquipmentUsedName'
      , [EquipmentUsedType] nvarchar(50) 'cidneNS:EquipmentUsedType'
      , [EventCategory] nvarchar(100) 'cidneNS:EventCategory'
      , [FiringPointDetails] nvarchar(max) 'cidneNS:FiringPointDetails'
      , [FiringPointDetailsMGRS] nvarchar(50) 'cidneNS:FiringPointDetailsMGRS'
      , [FPDisMGRS] bit 'cidneNS:FPDisMGRS'
      , [FPDLatitude] float(53) 'cidneNS:FPDLatitude'
      , [FPDLongitude] float(53) 'cidneNS:FPDLongitude'
      , [HostNationKIA] int 'cidneNS:HostNationKIA'
      , [HostNationWIA] int 'cidneNS:HostNationWIA'
      , [IEDEvent] nvarchar(100) 'cidneNS:IEDEvent'
      , [isDeprecated] bit 'cidneNS:isDeprecated'
      , [IsTearLine] bit 'cidneNS:IsTearLine'
      , [LanguagesKey] varchar(36) 'cidneNS:LanguagesKey'
      , [LastTimeRouteCleared] nvarchar(max) 'cidneNS:LastTimeRouteCleared'
      , [Latitude] float(53) 'cidneNS:Latitude'
      , [LightConditions] nvarchar(50) 'cidneNS:LightConditions'
      , [Longitude] float(53) 'cidneNS:Longitude'
      , [MarkingDetails] nvarchar(max) 'cidneNS:MarkingDetails'
      , [MGRS] nvarchar(20) 'cidneNS:MGRS'
      , [Narrative] nvarchar(max) 'cidneNS:Narrative'
      , [OriginatingNation] nvarchar(300) 'cidneNS:OriginatingNation'
      , [OriginatingNetwork] nvarchar(100) 'cidneNS:OriginatingNetwork'
      , [OriginatingSite] nvarchar(50) 'cidneNS:OriginatingSite'
      , [OriginatingSystem] nvarchar(50) 'cidneNS:OriginatingSystem'
      , [OriginatorGroup] nvarchar(300) 'cidneNS:OriginatorGroup'
      , [OriginatorName] nvarchar(300) 'cidneNS:OriginatorName'
      , [OriginatorUnit] nvarchar(300) 'cidneNS:OriginatorUnit'
      , [PrimarySafeAreaMGRS] nvarchar(50) 'cidneNS:PrimarySafeAreaMGRS'
      , [PSAisMGRS] bit 'cidneNS:PSAisMGRS'
      , [PSALatitude] float(53) 'cidneNS:PSALatitude'
      , [PSALongitude] float(53) 'cidneNS:PSALongitude'
      , [ReferencePoint] nvarchar(50) 'cidneNS:ReferencePoint'
      , [ReleasableDataClassification] nvarchar(50) 'cidneNS:ReleasableDataClassification'
      , [ReleasableDataClassificationReleasabilityMark] nvarchar(500) 'cidneNS:ReleasableDataClassificationReleasabilityMark'
      , [ReportingUnit] nvarchar(300) 'cidneNS:ReportingUnit'
      , [ReportKey] varchar(36) 'cidneNS:ReportKey'
      , [ReportType] nvarchar(50) 'cidneNS:ReportType'
      , [RoutePatrolInterval] nvarchar(50) 'cidneNS:RoutePatrolInterval'
      , [SafeAreaMGRS] nvarchar(50) 'cidneNS:SafeAreaMGRS'
      , [SecondarySafeAreaMGRS] nvarchar(50) 'cidneNS:SecondarySafeAreaMGRS'
      , [ServiceAffiliation] nvarchar(50) 'cidneNS:ServiceAffiliation'
      , [SigactReportKey] varchar(36) 'cidneNS:SigactReportKey'
      , [SigactTrackingNumber] nvarchar(50) 'cidneNS:SigactTrackingNumber'
      , [SiteMarked] nvarchar(50) 'cidneNS:SiteMarked'
      , [SSAisMGRS] bit 'cidneNS:SSAisMGRS'
      , [SSALatitude] float(53) 'cidneNS:SSALatitude'
      , [SSALongitude] float(53) 'cidneNS:SSALongitude'
      , [TeamLeaderComments] nvarchar(max) 'cidneNS:TeamLeaderComments'
      , [TearLineClassification] nvarchar(50) 'cidneNS:TearLineClassification'
      , [TearLineContact] nvarchar(max) 'cidneNS:TearLineContact'
      , [TearLineReleasability] nvarchar(500) 'cidneNS:TearLineReleasability'
      , [TearLineSourceReportDateUpdated] datetime 'cidneNS:TearLineSourceReportDateUpdated'
      , [TearLineSourceReportKey] varchar(36) 'cidneNS:TearLineSourceReportKey'
      , [TheaterFilter] nvarchar(50) 'cidneNS:TheaterFilter'
      , [Title] nvarchar(500) 'cidneNS:Title'
      , [TrackingNumber] nvarchar(150) 'cidneNS:TrackingNumber'
      , [UpdatedByGroup] nvarchar(300) 'cidneNS:UpdatedByGroup'
      , [UpdatedByName] nvarchar(300) 'cidneNS:UpdatedByName'
      , [UpdatedByUnit] nvarchar(300) 'cidneNS:UpdatedByUnit'
      , [Weather] nvarchar(50) 'cidneNS:Weather'
      , [WITAnalysis] nvarchar(max) 'cidneNS:WITAnalysis'
      , [WITComments] nvarchar(max) 'cidneNS:WITComments'
      , [WITDTGArrival] datetime 'cidneNS:WITDTGArrival'
      , [WITDTGArrivalZulu] datetime 'cidneNS:WITDTGArrivalZulu'
      , [WITDTGDeparture] datetime 'cidneNS:WITDTGDeparture'
      , [WITDTGFOBDeparture] datetime 'cidneNS:WITDTGFOBDeparture'
      , [WITDTGMissionComplete] datetime 'cidneNS:WITDTGMissionComplete'
      , [WITDTGNotification] datetime 'cidneNS:WITDTGNotification'
      , [WITDTGReportComplete] datetime 'cidneNS:WITDTGReportComplete'
      , [WITEvidence] nvarchar(max) 'cidneNS:WITEvidence'
      , [WITTeam] nvarchar(100) 'cidneNS:WITTeam'
    );

  • Great, emoticans in the code posted above, hope you all can decipher it.

  • Note:  your sample XML is missing a closing > on one of your Classification elements.

    Adapting the previous example code to a table containing one or more of your empty XML's  (just insert the @x variable into tables).
    This would get you all of the device level entries, with data availablility to the top level.  Hard to tell if each level has its own unqiue ID in which case we could probably generate the full detail dynamically

    WITH XMLNAMESPACES ( 'urn:cidneNS' as testalias,

    'http://www.w3.org/2001/XMLSchema-instance' as xsi)

    select a.value('*:AimingPointDetails[1]','varchar(50)') testval,

    case when b.value('testalias:Classification[1]','Varchar(100)') is null then

    a.value('testalias:Classification[1]','Varchar(100)')

    else b.value('testalias:Classification[1]','Varchar(100)') end,

    c.requestid

    from cidtemp c

    cross apply c.x.nodes('//testalias:ExploitationUXO') x(a)

    cross apply a.nodes('testalias:Device') a(b);

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

  • Matt Miller (#4) - Monday, March 20, 2017 11:50 AM

    Note:  your sample XML is missing a closing > on one of your Classification elements.

    Adapting the previous example code to a table containing one or more of your empty XML's  (just insert the @x variable into tables).
    This would get you all of the device level entries, with data availablility to the top level.  Hard to tell if each level has its own unqiue ID in which case we could probably generate the full detail dynamically

    WITH XMLNAMESPACES ( 'urn:cidneNS' as testalias,

    'http://www.w3.org/2001/XMLSchema-instance' as xsi)

    select a.value('*:AimingPointDetails[1]','varchar(50)') testval,

    case when b.value('testalias:Classification[1]','Varchar(100)') is null then

    a.value('testalias:Classification[1]','Varchar(100)')

    else b.value('testalias:Classification[1]','Varchar(100)') end,

    c.requestid

    from cidtemp c

    cross apply c.x.nodes('//testalias:ExploitationUXO') x(a)

    cross apply a.nodes('testalias:Device') a(b);

    The missing > could be due to poor cutting of data and I missed it.  I will look at this a bit later, I have work to catch up since I had to run a pet to the vets.  She got sting by something, probably a wasp.

  • Also in the "food for thought" category - you apparently have local access to the actual schema driving these messages.  The schema would have all of the fields (optional or no) and all entities, so it's kind of criminal that you have to go through a fire drill to get it imported every time someone decides to use something new.

    There are lots of widgets that can "translate" a schema file into a data model.  We happen to use the function within XMLSPY to do so, which gives you options as to whether to make the model based on EVERY field, just the mandatories, etc....  Having a known target for what to expect would seriously cut down on "fire-drilling".

    Once you've worked that out, you'd just need to be susbscribed into whenever they (where they = whoever "owns" the local schema) decide to publish an update to the XML schema.

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

  • Matt Miller (#4) - Tuesday, March 21, 2017 10:51 AM

    Also in the "food for thought" category - you apparently have local access to the actual schema driving these messages.  The schema would have all of the fields (optional or no) and all entities, so it's kind of criminal that you have to go through a fire drill to get it imported every time someone decides to use something new.

    There are lots of widgets that can "translate" a schema file into a data model.  We happen to use the function within XMLSPY to do so, which gives you options as to whether to make the model based on EVERY field, just the mandatories, etc....  Having a known target for what to expect would seriously cut down on "fire-drilling".

    Once you've worked that out, you'd just need to be susbscribed into whenever they (where they = whoever "owns" the local schema) decide to publish an update to the XML schema.

    Yes, The XSDs are local, but there are 2 caveats to it.  One, my local development VM actually doesn't have them.  Two, I am trying to get this done and working with limited knowledge and I am finding myself getting under a time constraint with limited time now to explore/learn alternatives.  Part of the time constraint is having gotten pulled of this to work other higher priority tasks.
    I would like to try and come back and revisit this and find better ways to accomplish the task.  We will see if I get some down time to do so.  I am getting close to having this done with OPENXML so I am sort of stuck on this track at the moment.

  • Understood - as is tradition, holler if you get stuck or have time to revisit it and need a hand.

    ----------------------------------------------------------------------------------
    Your lack of planning does not constitute an emergency on my part...unless you're my manager...or a director and above...or a really loud-spoken end-user..All right - what was my emergency again?

Viewing 12 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic. Login to reply