June 23, 2008 at 7:03 am
Hi,
I have a table ThreePointEstimate. I have a stored procedure that pulls records from ThreePointEstimate where a field is equal to an input parameter. This is the stored procedure:
ALTER PROCEDURE [dbo].[usp_ThreePointEstimate]
@ProjectID int
AS
set nocount on
SELECTThreePointEstimateID, WBSID, WBSSubsetID, SectionID, case when MinimumPercentage = 0 then null else ((MostLikelyCost / 100) * MinimumPercentage) end as MinCost, --MinimumCost,
MostLikelyCost, case when MaximumPercentage = 0 then null else ((MostLikelyCost / 100) * MaximumPercentage) end as MaxCost, --MaximumCost,
CurrencySymbol, ProjectID, MinimumPercentage, MaximumPercentage, ChangeID
FROMdbo.ThreePointEstimate
WHEREProjectID = @ProjectID
ORDER BY WBSID, WBSSubsetID, SectionID
set nocount off
The first three records in ThreePointEstimate are (the main fields):
MostLikelyCost MinimumPercentage MaximumPercentage
19864 95 105
458467 100 100
314408 95 105
So, you would expect that given the percentage, MinCost and MaxCost would return these values (rounded):
MostLikelyCost MinCost Maxcost
19864 18871 20857
458467 458467 458467
314408 298688 330128
But these values are returned:
MostLikelyCost MinCost Maxcost
19864 18810 20790
458467 458400 458400
314408 298680 330120
What I find the most interesting is that the record which has 100% as percentages (458467) changes :crazy:. Anyone know what could be going on here? I thought maybe its to do with rounding, but the first record isn't doing that.
Thanks,
Andrew
June 23, 2008 at 7:22 am
It is integer operations issue. Write as follows
(MostLikelyCost / 100.0) * MinimumPercentage
June 23, 2008 at 8:19 am
Thanks, that solved it.
Andrew
Viewing 3 posts - 1 through 2 (of 2 total)
You must be logged in to reply to this topic. Login to reply