Home : Products : OPS SQL : Handling Precision Rules for New York DMR
Q10086 - HOWTO: Handling Precision Rules for New York DMR

Handling Precision rules for New York DMR


The State of New York has some unique requirements for handling precision in calculations when filling out the DMR.  These rules are outlined in New York State Department of Environmental Conservation’s “DMR MANUAL - For Completing the Discharge Monitoring Report for the State Pollutant Discharge Elimination System (SPDES)”.  This document will take you through the steps to properly handle these rules when creating your SPDES.


Rules for Precision (from DMR Manual)

  1. For addition and subtraction, the answer can contain no more decimal places than the least precise measurement.
  2. For multiplication and division, the least Number of significant digits in a measurement determines the number of significant digits in the answer.
  3. Numbers such as conversion factors or number of days, are counted numbers and are not considered when determining the number of significant digits or decimal places in the calculation.
  4. If both addition/subtraction and multiplication/division are used in a calculation, follow the rules for multiplication/division.



We will run through an example of reporting your Effluent BOD (concentration and pounds)



Step 1: Setup Variables

The Effluent BOD variable must be setup to allow the user to enter (or import from LIMS) a different number of decimal places for each day.  Typically, an OPS variable is setup to allow a fixed number of decimal places when entering data.  In order to allow a “Not Fixed” number of decimal places the variable must be setup to allow data qualifiers and set the Decimal Places to not fixed. (Note: you must first check the Allow the use of ‘<’,’>’, and ‘ND’ Entries before the “Not Fixed” option is available for the variable.)







Setup your Effluent BOD Loading variable as follows:






  1. Enter the data:




  1. Report the data using Spread Reports.


When you choose a variable, the Decimal Places field gets set to the default set in Variable Edit.  In this case, sets it to “Not Fixed”.  When locating data with a setting of Not Fixed the data will be located into the report as entered (i.e. if 2.1234 is entered 2.1234 will be displayed, if 2.1 is entered 2.1 will be displayed).  This locates the VT function VT(4011,1,9999) where the 9999 is the setting for “Not Fixed” Decimal Places. 


Place Monthly Average of Effluent BOD.  According to Rule #2 from the DMR Manual (Least Number of significant digits in a measurement determines the number of significant digits in the answer)




Locates the formula GAVGZ(4011,1,"R",1022,1).  Notice the Decimal Places is set to 1022.  When Decimal Places is set to 1000 or above each digit has a meaning:


2nd Digit sets number of Significant digits to round to.  A setting of 0 here means round to the minimum of the significant digits of the inputs.  A setting of 3 means round to 3 significant digits.


3rd digit sets the Maximum number of Decimal Places to display even if more are required to display the Significant digits set in the 2nd digit.


4th digit sets the rounding used if required:  1-scientific, 2 Normal, 3 Trunc


Therefore, 1022 means round to the minimum number of Significant Digits found in the inputs, to a max of 2 decimal places and use normal rounding.


Now let’s place the Maximum. 



With Maximum (any stat that finds a single value like min, max, first, last…) any Decimal Places set above 999 means to simple report the value as it was entered.





Locating the Total Pounds:


In order to locate the Total, Precision Rule #1 applies (For addition and subtraction, the answer can contain no more decimal places than the least precise measurement).


In this case, you will have to locate the Total to 4 decimal places (max number it could ever be).  Then use the Round function and the GDP function to round the number to the minimum number of decimal places in the input.


The formula becomes:



GSUM returns the total (1,967.70)


GDP returns the minimum number of decimal places for each value in the report date range for variable 4012.  (return 0)


Round – Rounds the value to the number of decimal places.  Returns 1968 – 1967.70 rounded to 0 decimal places.






SPDES report:

On your SPDES report, you report the summaries on the report.  Therefore you will have to use formulas like GAVGZ(4011,1,"R",1022,1), round(GSUM(4012,1,"R"),GDP(4012,1,"R","MIN"))…




  In order to handle New York Precison Rules for rounding we need the following functions:

SFMIN(Min,Vx,Vy,....) Math toolbox function - returns the minimum Precision (Significant Figures) for Vx, Vy...  Min is a constant that is returned if it is lower than the minimum Precision returned.

V1 = 3.75
V2 = 5.6

SFMIN(4,V1,V2)  = 2 (because 5.6 is 2 Sig Figs)
SFMIN(1,V1,V2) = 1 (because min of 1 is less than 2)

SDIGITS(V1*V2*8.34,SFMIN(2,V1,V2)) = 180


Spread Function add to GAVGZ type to ignore >'s and cascade >.  IE >4, <3 is >3.  The <4 is ignored, average is 3 and the > is cascaded in. 

Spread Function GMAXZ needs to have an MDL rule that returns >4 in the following series of numbers:
>4, 8 , 6. 

See Q10106 Request.
Related Articles
No Related Articles Available.

Article Attachments
No Attachments Available.

Related External Links
No Related Links Available.
Help us improve this article...
What did you think of this article?


Tell us why you rated the content this way. (optional)
Approved Comments...
Google preved rodnoy! apcservicder Approved: 9/21/2006 12:47 AM
WOW. Thats really impressive you all can handle this. Approved: 2/3/2005 3:26 PM
Created on 2/3/2005 11:41 AM.
Last Modified on 3/23/2005 2:19 PM.
Last Modified by Scott Dorner.
Article has been viewed 9535 times.
Rated 7 out of 10 based on 13 votes.
Print Article
Email Article