Parallel Processing

  • Comments posted to this topic are about the item Parallel Processing

  • This is an excellent article the gave an idea have to solve some problems that arise from the need of paraller processing. Thank you very very much.

    As all we want is right resource planning I show in the following article that its better to create temporary tables and use INSERT INTO instead of SELECT INTO. As the article says its a resources matter.

    The article is here:http://www.sqlservercentral.com/articles/Basic+Querying/temptablesinsqlserver/1279/

    Thank you.

  • As I am new to SQL i cannot understand why you are using the statement

    SELECT * INTO #LockHelp FROM Rainfall_Events WITH (TABLOCKX) WHERE 1=2. This statement does not fetch any rows?

    I't must be somenthing about the lock time, bit can you explain me why you are using "WHERE 1=2"?

    Thank you

  • iermis (10/12/2009)


    As I am new to SQL i cannot understand why you are using the statement

    SELECT * INTO #LockHelp FROM Rainfall_Events WITH (TABLOCKX) WHERE 1=2. This statement does not fetch any rows?

    It must be something about the lock time, bit can you explain me why you are using "WHERE 1=2"?

    Thank you

    I think Sunny is using this "WHERE 1 = 2" for creating the #LockHelp table only, having no data.

    But not sure why, because he has not refer #LockHelp in the SP.

  • Nice article. Keep it up.

  • Hi iermis

    Thanks for the comments.

    The query you mentioned is only for getting a Table Lock on the table. We dont actually need any records from the table but at the same time we want to have the table locked from the Rest_OrphanLocks SP until the records are marked as processed. The condition "1=2" does exactly that. It does not spend time retrieving records but locks the table. #LockHelp Temporary table is optional. We could as well have a SELECT query instead of SELECT INTO

    Thanks

    Satish More

  • mohd.nizamuddin (10/12/2009)

    I think Sunny is using this "WHERE 1 = 2" for creating the #LockHelp table only, having no data.

    But not sure why, because he has not refer #LockHelp in the SP.

    Thanks Nizamuddin. #LockHelp is optional. You can as well have only a SELECT query instead of SELECT INTO. The only difference is that it returns an empty result set back to the SP caller

    Thanks

    Satish More

  • Thank you all for your fast replies.

    🙂

  • I recently (within the last 2 years) created a system to monitor our SQL instances from a central instance. Without going into details about the whole process, I had a requirement to run multiple tasks to retrieve data from the monitored instances in parallel (one or more instances unavailable would cause delays in retrieving data from all the other instances).

    There's an easier way to do this than messing with lock hints, use the UPDATE ... OUTPUT format of the UPDATE query. Here's a snippet of the code that I use:

    -- ******************************************************************************************

    -- Create the local temp tables used in this proc

    CREATE TABLE #DRT_Update (

    DataRetrievalTaskIDINT)

    -- ******************************************************************************************

    -- ******************************************************************************************

    -- Clear the temp table ...

    ProcessNextServer:

    DELETE #DRT_Update

    -- ******************************************************************************************

    -- ******************************************************************************************

    -- Get the next Data Retrieval Task to be processed - set the status to 2 for the record selected.

    UPDATE DataRetrievalTask

    SET [Status] = 2

    OUTPUT inserted.DataRetrievalTaskID

    INTO #DRT_Update

    WHERE DataRetrievalTaskID = (SELECT MIN(DataRetrievalTaskID)

    FROM DataRetrievalTask

    WHERE [Status] = 1

    AND StartTime <= GETDATE() )

    -- ******************************************************************************************

    This process is easily scalable - I just have to create another job to execute the Stored Proc to add another process. This way, you dont have to worry about Locks etc - the SQL Engine does it for you.

  • Can you explain the code?

  • Simon's post about using the OUTPUT clause on the update becomes available for us to use in SQL 2005 databases. It is an awesome feature (Oracle has had it for a while).

  • Really interesting article.

    Could the clean up process use something like sp_who to find out if a process was currently running or orphaned. I'm always a bit dubious about things that depend on timings which would need to be tuned for each installation depending upon resources, data sizes etc.

  • [p]Scott was exactly right. The OUTPUT clause of the UPDATE allows us to get data back from the inserted and deleted tables (the same as we reference in DML triggers). In the case of my code snippet, I always want to process the first record in the table with Status = 1 (Ready to Process) and a StartTime < Now. I contemplated using Transactions and Setting Lock hints, but it was a lot of work compared to using the UPDATE ... OUTPUT statement. This way the record is updated to Status = 2 (In Process) and the row ID value returned to the code in a single statement - no extended or manually coded locking required. [/p]

    [p]BOL has a good article on the OUTPUT clause; but a couple of points - (1) you can get back any data from the inserted or deleted tables, not just a single column and (2) the INTO table has to pre-exist, hence the CREATE TABLE statement at the top of the code.[/p]

    [p]The Code snippet is written as part of a loop (using the forbidden GOTO statement - lets see who jumps on that, but that's a different discussion), hence the DELETE statement.[/p]

    Hope that helps.

  • Kevin.McEvoy (10/12/2009)


    Really interesting article.

    Could the clean up process use something like sp_who to find out if a process was currently running or orphaned. I'm always a bit dubious about things that depend on timings which would need to be tuned for each installation depending upon resources, data sizes etc.

    Thanks Kevin. Yes I too tried digging into any such readily available command which could tell whether the process is currently active. Hope to get more leads in this forum. The UPDATE..OUTPUT statement suggested by Simon appears to be very helpful. Thanks Simon 🙂

    Thanks

    Satish

  • Interesting article....I'm surprised at the hoops you have to jump through.

    On DB2 for i, all I need to do is set QQRYDEGREE for the system or for individual jobs to something besides *NONE (or *NBRTASKS 1).

    http://www-03.ibm.com/servers/enable/site/education/abstracts/4aea_abs.html

    From the above course:

    How SMP works

    The DB2 SMP feature provides the optimizer and database engine with additional methods and strategies for retrieving data and processing data in a parallel manner. SMP enables database parallelism on a single system or LPAR - where multiple (CPU and I/O) processors that share memory and disk resources - to work simultaneously toward achieving a single end result. This parallel processing means that the database engine can have more than one (or all) the processors working on a single query at the same time. You can significantly improve the performance of a processor-bound query with this feature on multiple-processor systems by distributing the processor load across more than one processor.

    Although using SMP does not require the presence of more than one processor, database parallelism is most effective when more than one physical processor is available to run the tasks or threads.

    Given that SMP is achieved through the use of the System i server and its i5/OS advanced architecture, table partitioning is not required for database parallelism.

    Charles Wilt

Viewing 15 posts - 1 through 15 (of 31 total)

You must be logged in to reply to this topic. Login to reply