June 2, 2011 at 4:10 pm
I have been handed the task of parsing (might be better to say "unstacking") records produced by a cash-register system. The register activity is logged as it happens - cashier logs in - record #1 created, cashier scans a bottle of water - record #2 created, cashier totals the order - record #3 created, etc., etc.
The records sent to me by the system are stacked in the order each event occurred.
I need to process these records into a normal table. For example, the UPC of the item scanned will be the first column of record #1, the second column will be column #2 for record #1, etc.
Is the best way to do this with a cursor? Any way to do this with queries?
Any suggestions appreciated.
Bill T.
June 2, 2011 at 4:39 pm
The problem and details of the input are well known to yourself, but not to those who would like to assist you.
Can you supply sample data, database table definition ... and expected or required results from the sample data. To do so quickly and easily please click on the first link in my signature block to learn how to do so.
June 2, 2011 at 4:42 pm
Based only on your description alone I am almost 100% sure you can do what is required without using a cursor.
I second what Ron said. Help us by providing DDL to create test tables, DML to create test data and your expected results and we'll help you by providing tested code that does what you need.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
June 2, 2011 at 4:55 pm
Bill Tidwell (6/2/2011)
I have been handed the task of parsing (might be better to say "unstacking") records produced by a cash-register system. The register activity is logged as it happens - cashier logs in - record #1 created, cashier scans a bottle of water - record #2 created, cashier totals the order - record #3 created, etc., etc.The records sent to me by the system are stacked in the order each event occurred.
I need to process these records into a normal table. For example, the UPC of the item scanned will be the first column of record #1, the second column will be column #2 for record #1, etc.
Is the best way to do this with a cursor? Any way to do this with queries?
Any suggestions appreciated.
Bill T.
This CAN absolutely be done without a cursor. The real question is WHY? What are you going to do when someone like me, who shops about once a month, and buys 400 different items? Why would you possibly want to list all those items in a horizontal format?
My recommendation is that the format you have the data in is probably correct already.
--Jeff Moden
Change is inevitable... Change for the better is not.
June 3, 2011 at 10:25 am
When I say, "horizontal format", I mean one single record per scan. For example, in the raw-data file I get from the POS, everything is "single stacked" - the UPC number, for example, might be on row 873. The description of the item is on row 874, the quantity scanned on row 874, the scanned price on 875, etc. I need the entire scan event (not the entire transaction) needs to be in one record, in which the UPC is the first column, the item description column 2, the quantity scanned column 3, the scanned SRP column 4, etc.
Bill
June 3, 2011 at 10:38 am
To All -
Thanks for taking an interest in my issue.
Here is a sample of the raw, POS data:
;Z60
#36657-6 07:12 Z 090
8:33AM Apr 1/11
00-0000 002 REG 2
#36658 CLERK
NO SALE
9:06AM Apr 1/11
00-0000 002 REG 2
#36659 CLERK
DROP $111.54
T O T A L $111.54
CASH $111.54
9:07AM Apr 1/11
00-0000 002 REG 2
#36660 CLERK
PICK UP -75.00
T O T A L -75.00
CHANGE $75.00
9:10AM Apr 1/11
00-0000 002 REG 2
#36661 CLERK
NO SALE
9:33AM Apr 1/11
00-0000 002 REG 2
#36662 CLERK
#5000037859
ALPO COME N G2$9.49
MDSE $9.49
TAX2 $0.93
T O T A L $10.42
CASH $15.00
CHANGE $4.58
10:53AM Apr 1/11
00-0000 002 REG 2
#36663 CLERK
#7185201360
LADY K BUNS F1$1.09
#7003862959
BC 2 RF MILKF1$1.29
#7003833241
AL SAVE PORKF1$0.55
#7003833241
AL SAVE PORKF1$0.55
#7003862953
BST CH 2 RF F1$2.29
#7003833241
AL SAVE PORKF1$0.55
#20934800000
I am not interested in parsing all the info contained. I am primarily interested in capturing the UPC, item description, the quantity scanned and the scanned SRP. In the raw data, the rows containing the UPC begin with a #, followed by a 10-digit number. The row immediately after each UPC cntains the item description, quantity scanned and price.
I haven't buit anything in SQL yet, because I am trying to determine what to build: Cursor or queries.
Again, i need to convert the data shown to a standard, database format in which all the item information (UPC, description, quantity, etc.) are distributed across columns on a single row - not broken into separate rows.
Sorry my earlier explanation was not more clear.
Bill
June 3, 2011 at 11:57 am
I second what Ron said. Help us by providing DDL to create test tables, DML to create test data and your expected results and we'll help you by providing tested code that does what you need.
Pretty please 😀
Read this article if you have any doubts about why we're asking:
http://www.sqlservercentral.com/articles/Best+Practices/61537/[/url]
Also, note the article's author and the folks that have previously posted on this thread...you'll get quality help if you help us help you 😉
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
June 3, 2011 at 12:01 pm
This is a common problem when working with older mainframes. As OPC pointed out above, being able to consume the data easily in a test structure will assist us.
In your sample data, is that raw and real, or modified?
The reason I ask is row detection is going to be a wild PITA on that. It can be done, but the parsing's going to get uuuugly.
If there's more to the structure let us see it, because we're going to need to dig deep into that.
Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.
For better assistance in answering your questions[/url] | Forum Netiquette
For index/tuning help, follow these directions.[/url] |Tally Tables[/url]
Twitter: @AnyWayDBA
June 3, 2011 at 1:21 pm
opc.three
Help me out here. I have no DDL or DML. The sample data included in my post was captured from a raw-data/ ASCII file supplied by the point-of-sale system. I have imported the text file into a 1-field table (data type varchar) SQL 2008 table and am trying to come up with the most efficient way to parse.
Is it possible to attach the text file to a post?
I guess I am not sure what you are wanting.
Sorry.
Bill
June 3, 2011 at 1:31 pm
Craig,
The data sample included in my post is raw and real.
This is coming from an older, less sophisticated POS system. My company presently extracts data from 9 or 10 different POS systems. The sample data is from a company that has maybe 1% of the POS market, nationwide.
I was afraid that parsing this file would be ugly.
I am pretty sure that I can filter out the UPC records, reliably.
When the raw file was imported into SQL, the primary key assignments reflect (precisely I hope) the ordinal position of each record in the raw file. To associate the item description with the correct UPC, I am planning to build a query that isolates the UPC rows then calculate the primary-key number for the description by adding 1 to the UPC pk (item descriptions seem to always fall in the row following the UPC). I should be able to then join this query to a query that selects only the UPC and pk and I should have each UPC associated with the proper item desctiption.
Sorry is this all sounds like Greek.
Bill T.
June 3, 2011 at 1:37 pm
Bill Tidwell (6/3/2011)
opc.threeHelp me out here. I have no DDL or DML. The sample data included in my post was captured from a raw-data/ ASCII file supplied by the point-of-sale system. I have imported the text file into a 1-field table (data type varchar) SQL 2008 table and am trying to come up with the most efficient way to parse.
Post the DDL for that 1-column table and DML to populate said table with the data in your previous post. Could we whip up a quick table on our side, copy your data and do some regex find/replace to populate the table from your previous post...sure....but we're unpaid volunteers trying to help you...so the least you could do it provide a starting point. Not being rude, just saying.
Is it possible to attach the text file to a post?
Yes, but it would be better if you provided a CREATE TABLE statement followed by a bunch of INSERTs.
I guess I am not sure what you are wanting.
Did you have a look at the article?
We also would like to see the expected results based on your sample data so we can infer the requirement from there. Note that it's best if your test data exemplifies whatever nuance is required in your request...and remember we do not know your data like you know it.
There are no special teachers of virtue, because virtue is taught by the whole community.
--Plato
June 3, 2011 at 1:37 pm
if you only need the UPC, it looks like you could search for the # sign, and either a space or the end of the row.
this example SEEMs to work on the sample data provided, or might at least provide a starting point
--Results
/*
#20934800000
#36657-6
#36658
#36659
#36660
#36661
#36662
#36663
#5000037859
#7003833241
#7003862953
#7003862959
#7185201360
*/
with mySampleData(TheData) As
(
SELECT ';Z60' UNION ALL
SELECT '#36657-6 07:12 Z 090' UNION ALL
SELECT '8:33AM Apr 1/11' UNION ALL
SELECT '00-0000 002 REG 2' UNION ALL
SELECT '#36658 CLERK' UNION ALL
SELECT 'NO SALE' UNION ALL
SELECT '9:06AM Apr 1/11' UNION ALL
SELECT '00-0000 002 REG 2' UNION ALL
SELECT '#36659 CLERK' UNION ALL
SELECT 'DROP $111.54' UNION ALL
SELECT 'T O T A L $111.54' UNION ALL
SELECT 'CASH $111.54' UNION ALL
SELECT '9:07AM Apr 1/11' UNION ALL
SELECT '00-0000 002 REG 2' UNION ALL
SELECT '#36660 CLERK' UNION ALL
SELECT 'PICK UP -75.00' UNION ALL
SELECT 'T O T A L -75.00' UNION ALL
SELECT 'CHANGE $75.00' UNION ALL
SELECT '9:10AM Apr 1/11' UNION ALL
SELECT '00-0000 002 REG 2' UNION ALL
SELECT '#36661 CLERK' UNION ALL
SELECT 'NO SALE' UNION ALL
SELECT '9:33AM Apr 1/11' UNION ALL
SELECT '00-0000 002 REG 2' UNION ALL
SELECT '#36662 CLERK' UNION ALL
SELECT '#5000037859' UNION ALL
SELECT 'ALPO COME N G2$9.49' UNION ALL
SELECT 'MDSE $9.49' UNION ALL
SELECT 'TAX2 $0.93' UNION ALL
SELECT 'T O T A L $10.42' UNION ALL
SELECT 'CASH $15.00' UNION ALL
SELECT 'CHANGE $4.58' UNION ALL
SELECT '10:53AM Apr 1/11' UNION ALL
SELECT '00-0000 002 REG 2' UNION ALL
SELECT '#36663 CLERK' UNION ALL
SELECT '#7185201360' UNION ALL
SELECT 'LADY K BUNS F1$1.09' UNION ALL
SELECT '#7003862959' UNION ALL
SELECT 'BC 2 RF MILKF1$1.29' UNION ALL
SELECT '#7003833241' UNION ALL
SELECT 'AL SAVE PORKF1$0.55' UNION ALL
SELECT '#7003833241' UNION ALL
SELECT 'AL SAVE PORKF1$0.55' UNION ALL
SELECT '#7003862953' UNION ALL
SELECT 'BST CH 2 RF F1$2.29' UNION ALL
SELECT '#7003833241' UNION ALL
SELECT 'AL SAVE PORKF1$0.55' UNION ALL
SELECT '#20934800000 '
)
SELECT
SUBSTRING(TheData,CHARINDEX('#',TheData),CHARINDEX(' ',TheData,CHARINDEX('#',TheData)))
From MySampleData
WHERE CHARINDEX('#',TheData) > 0
UNION
SELECT
SUBSTRING(TheData,CHARINDEX('#',TheData),30)
From MySampleData
WHERE CHARINDEX('#',TheData) > 0 AND CHARINDEX(' ',TheData) = 0
Lowell
June 3, 2011 at 2:03 pm
Lowell,
Thanks. I believe you are correct about isolating the UPC number using the '#'. The challenge is in relating (in a single-record layout) the row after each UPC.
Thanks to everyone for the help.
BT
June 3, 2011 at 2:26 pm
kinda fun guessing for now:
assuming you inserted the data into a table with an indeity, i'm joining on found rows to get the price;
it also looks like the descriptiuon is fixed width, so the F/G indicator is Food / Groceries?(taxiable or not, right) and a single digit quantity? is that right?
--Results
/*
ID TheUPC TheData ThePrice
26 #5000037859 ALPO COME N G2$9.49 $9.49
36 #7185201360 LADY K BUNS F1$1.09 $1.09
38 #7003862959 BC 2 RF MILKF1$1.29 $1.29
40 #7003833241 AL SAVE PORKF1$0.55 $0.55
42 #7003833241 AL SAVE PORKF1$0.55 $0.55
44 #7003862953 BST CH 2 RF F1$2.29 $2.29
46 #7003833241 AL SAVE PORKF1$0.55 $0.55
*/
with mySampleData(ID,TheData) As
(
--ASSUMING YOU INSERTED THE DATA INTO A TABLE WITH IDENTITIY
SELECT 1,';Z60' UNION ALL
SELECT 2,'#36657-6 07:12 Z 090' UNION ALL
SELECT 3,'8:33AM Apr 1/11' UNION ALL
SELECT 4,'00-0000 002 REG 2' UNION ALL
SELECT 5,'#36658 CLERK' UNION ALL
SELECT 6,'NO SALE' UNION ALL
SELECT 7,'9:06AM Apr 1/11' UNION ALL
SELECT 8,'00-0000 002 REG 2' UNION ALL
SELECT 9,'#36659 CLERK' UNION ALL
SELECT 0,'DROP $111.54' UNION ALL
SELECT 11,'T O T A L $111.54' UNION ALL
SELECT 12,'CASH $111.54' UNION ALL
SELECT 13,'9:07AM Apr 1/11' UNION ALL
SELECT 14,'00-0000 002 REG 2' UNION ALL
SELECT 15,'#36660 CLERK' UNION ALL
SELECT 16,'PICK UP -75.00' UNION ALL
SELECT 17,'T O T A L -75.00' UNION ALL
SELECT 18,'CHANGE $75.00' UNION ALL
SELECT 19,'9:10AM Apr 1/11' UNION ALL
SELECT 20,'00-0000 002 REG 2' UNION ALL
SELECT 21,'#36661 CLERK' UNION ALL
SELECT 22,'NO SALE' UNION ALL
SELECT 23,'9:33AM Apr 1/11' UNION ALL
SELECT 24,'00-0000 002 REG 2' UNION ALL
SELECT 25,'#36662 CLERK' UNION ALL
SELECT 26,'#5000037859' UNION ALL
SELECT 27,'ALPO COME N G2$9.49' UNION ALL
SELECT 28,'MDSE $9.49' UNION ALL
SELECT 29,'TAX2 $0.93' UNION ALL
SELECT 30,'T O T A L $10.42' UNION ALL
SELECT 31,'CASH $15.00' UNION ALL
SELECT 32,'CHANGE $4.58' UNION ALL
SELECT 33,'10:53AM Apr 1/11' UNION ALL
SELECT 34,'00-0000 002 REG 2' UNION ALL
SELECT 35,'#36663 CLERK' UNION ALL
SELECT 36,'#7185201360' UNION ALL
SELECT 37,'LADY K BUNS F1$1.09' UNION ALL
SELECT 38,'#7003862959' UNION ALL
SELECT 39,'BC 2 RF MILKF1$1.29' UNION ALL
SELECT 40,'#7003833241' UNION ALL
SELECT 41,'AL SAVE PORKF1$0.55' UNION ALL
SELECT 42,'#7003833241' UNION ALL
SELECT 43,'AL SAVE PORKF1$0.55' UNION ALL
SELECT 44,'#7003862953' UNION ALL
SELECT 45,'BST CH 2 RF F1$2.29' UNION ALL
SELECT 46,'#7003833241' UNION ALL
SELECT 47,'AL SAVE PORKF1$0.55' UNION ALL
SELECT 48,'#20934800000'
)
SELECT
V1.ID,
SUBSTRING(V1.TheData,CHARINDEX('#',V1.TheData),30) As TheUPC,
V2.TheData,
SUBSTRING(V2.TheData,CHARINDEX('$',V2.TheData),30) As ThePrice
From MySampleData V1
INNER JOIN MySampleData V2
ON V1.ID + 1 = V2.ID
WHERE CHARINDEX('#',V1.TheData) > 0 AND CHARINDEX(' ',V1.TheData) = 0
and a little more assumptions piled on:
/*
ID TheUPC TheData ItemDescrip ThePrice Taxindicator TheQty
26 #5000037859 ALPO COME N G2$9.49 ALPO COME N $9.49 G 2
36 #7185201360 LADY K BUNS F1$1.09 LADY K BUNS $1.09 F 1
38 #7003862959 BC 2 RF MILKF1$1.29 BC 2 RF MILK $1.29 F 1
40 #7003833241 AL SAVE PORKF1$0.55 AL SAVE PORK $0.55 F 1
42 #7003833241 AL SAVE PORKF1$0.55 AL SAVE PORK $0.55 F 1
44 #7003862953 BST CH 2 RF F1$2.29 BST CH 2 RF $2.29 F 1
46 #7003833241 AL SAVE PORKF1$0.55 AL SAVE PORK $0.55 F 1
*/
SELECT
V1.ID,
SUBSTRING(V1.TheData,CHARINDEX('#',V1.TheData),30) As TheUPC,
V2.TheData,
SubString(V2.TheData,1,12) As ItemDescrip,
SUBSTRING(V2.TheData,CHARINDEX('$',V2.TheData),30) As ThePrice,
SUBSTRING(V2.TheData,13,1)As Taxindicator,
--infering 12 char description, variable length qty
SUBSTRING(V2.TheData,14,CHARINDEX('$',V2.TheData)- 14 ) As TheQty
From MySampleData V1
INNER JOIN MySampleData V2
ON V1.ID + 1 = V2.ID
WHERE CHARINDEX('#',V1.TheData) > 0 AND CHARINDEX(' ',V1.TheData) = 0
Lowell
June 3, 2011 at 2:37 pm
To all -
Sorry it took me so long to understand what you were looking for.
Try this - it runs fine in SQL 2008:
/****** Object: Table [dbo].[tblDataSymRawImport] Script Date: 06/03/2011 15:07:37 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[tblDataSymRawImport]') AND type in (N'U'))
DROP TABLE [dbo].[tblDataSymRawImport]
GO
/****** Object: Table [dbo].[tblDataSymRawImport] Script Date: 06/03/2011 15:07:37 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_PADDING ON
GO
--=========Create the table================================================
CREATE TABLE [dbo].[tblDataSymRawImport](
[pk] [int] IDENTITY(1,1) NOT NULL,
[RawImport] [varchar](25) NULL
) ON [PRIMARY]
GO
SETDATEFORMAT DMY
GO
SET ANSI_PADDING OFF
GO
--Insert the data into the table ==========================================
SET IDENTITY_INSERT dbo.tblDataSymRawImport ON
INSERT INTO dbo.tblDataSymRawImport(pk, RawImport)
Select '1',';Z60' UNION ALL
Select '2','#53670-6 07:14 Z 090' UNION ALL
Select '3',' 7:14AM Apr 1/11' UNION ALL
Select '4','00-0000 001 REG 1 ' UNION ALL
Select '5','#53671 CLERK ' UNION ALL
Select '6','CARD # ' UNION ALL
Select '7',' ************6056' UNION ALL
Select '8','MANUALLY ENTERED ' UNION ALL
Select '9','REF# 0000' UNION ALL
Select '10',' 7:17AM Apr 1/11' UNION ALL
Select '11','00-0000 001 REG 1 ' UNION ALL
Select '12','#53672 CLERK ' UNION ALL
Select '13','#3600019586 ' UNION ALL
Select '14','POISE PADS X-2$7.39 ' UNION ALL
Select '15','CARD # ' UNION ALL
Select '16',' ************3019' UNION ALL
Select '17','REF# 0001' UNION ALL
Select '18','MDSE $7.39' UNION ALL
Select '19','TAX2 $0.72' UNION ALL
Select '20',' T O T A L $8.11' UNION ALL
Select '21','DEBIT CARD $8.11' UNION ALL
Select '22',' 7:18AM Apr 1/11' UNION ALL
Select '23','00-0000 001 REG 1 ' UNION ALL
Select '24','#53673 CLERK ' UNION ALL
Select '25','GROCERY F1$3.99 ' UNION ALL
Select '26','GROCERY F1$11.00 ' UNION ALL
Select '27','FOOD COUPON F-1.00 ' UNION ALL
Select '28','#2430004120 ' UNION ALL
Select '29','LD NUTTY BARF1$1.25 ' UNION ALL
Select '30','#2430004130 ' UNION ALL
Select '31','LD SWISS CK F1$1.59 ' UNION ALL
Select '32','#2430004101 ' UNION ALL
Select '33','LD OATMEAL CF1$1.25 ' UNION ALL
Select '34','#2430004120 ' UNION ALL
Select '35','LD NUTTY BARF1$1.25 ' UNION ALL
Select '36','MDSE $19.33' UNION ALL
Select '37','TAX1 $1.68' UNION ALL
Select '38',' T O T A L $21.01' UNION ALL
Select '39','CASH $25.01' UNION ALL
Select '40','CHANGE $4.00' UNION ALL
Select '41',' 7:19AM Apr 1/11' UNION ALL
Select '42','00-0000 001 REG 1 ' UNION ALL
Select '43','#53674 CLERK ' UNION ALL
Select '44','#5000041250 ' UNION ALL
Select '45','CARN COFMAT F1$2.59 ' UNION ALL
Select '46','MDSE $2.59' UNION ALL
Select '47','TAX1 $0.21' UNION ALL
Select '48',' T O T A L $2.80' UNION ALL
Select '49','CASH $10.00' UNION ALL
Select '50','CHANGE $7.20' UNION ALL
Select '51',' 7:20AM Apr 1/11' UNION ALL
Select '52','00-0000 001 REG 1 ' UNION ALL
Select '53','#53675 CLERK ' UNION ALL
Select '54','#4369505632 ' UNION ALL
Select '55','CRSSNT PKT HF1$2.50 ' UNION ALL
Select '56','#1200000134 ' UNION ALL
Select '57','D M DEW F1$1.59 ' UNION ALL
Select '58','#2200000667 ' UNION ALL
Select '59','JUICY FRUIT F2$1.09 ' UNION ALL
Select '60','MDSE $5.18' UNION ALL
Select '61','TAX1 $0.34' UNION ALL
Select '62','TAX2 $0.11' UNION ALL
Select '63',' T O T A L $5.63' UNION ALL
Select '64','CASH $20.00' UNION ALL
Select '65','CHANGE $14.37' UNION ALL
Select '66',' 7:26AM Apr 1/11' UNION ALL
Select '67','00-0000 001 REG 1 ' UNION ALL
Select '68','#53676 CLERK ' UNION ALL
Select '69','#3400000241 ' UNION ALL
Select '70','HRSH ALMOND F2$0.79 ' UNION ALL
Select '71','MDSE $0.79' UNION ALL
Select '72','TAX2 $0.08' UNION ALL
Select '73',' T O T A L $0.87' UNION ALL
Select '74','CASH $5.00' UNION ALL
Select '75','CHANGE $4.13' UNION ALL
Select '76',' 7:31AM Apr 1/11' UNION ALL
Select '77','00-0000 001 REG 1 ' UNION ALL
Select '78','#53677 CLERK ' UNION ALL
Select '79','#1128400301 ' UNION ALL
Select '80','JJS APPLE PIF1$0.50 ' UNION ALL
Select '81','#1128400301 ' UNION ALL
Select '82','JJS APPLE PIF1$0.50 ' UNION ALL
Select '83','#2430004325 ' UNION ALL
Select '84','LD BP BROWNIF1$3.00 ' UNION ALL
Select '85','#2500005603 ' UNION ALL
Select '86','MM OJ F1$1.59 ' UNION ALL
Select '87','#1200010092 ' UNION ALL
Select '88','FRAPP MOCHA F1$6.69 ' UNION ALL
Select '89','#3700018506 ' UNION ALL
Select '90','PRINGLES SOUF1$0.60 ' UNION ALL
Select '91','#5410740100 ' UNION ALL
Select '92','KIWI FRUIT 5F1$1.99 ' UNION ALL
Select '93','CARD # ' UNION ALL
Select '94',' ************3874' UNION ALL
Select '95','REF# 0002' UNION ALL
Select '96',' T O T A L $14.87' UNION ALL
Select '97','FS ST $14.87' UNION ALL
Select '98','EBT FSTEND $14.87' UNION ALL
Select '99',' 7:32AM Apr 1/11' UNION ALL
Select '100','00-0000 001 REG 1 '
SET IDENTITY_INSERT dbo.tblDataSymRawImport OFF
Viewing 15 posts - 1 through 15 (of 31 total)
You must be logged in to reply to this topic. Login to reply