Developers want to have test meter data for OED development that is consistent, stable and does not require having access to actual meters. This describes how to get and load such meter test data.
The comparison graphic in OED uses the current time to compare usage over a given period to the same period shifted by that same period in the past. Since the test data has static dates, they will not include dates near the current time. This means that you cannot see comparison data. We will be posting directions to import test data that will allow comparison graphs.
For using the following data with maps, please see the test map page.
What follows assumes you have a properly installed and working OED but have not yet loaded in the reading data described here. If they have already been loaded then the loading step for that meter will be skipped. At a minimum you should be able to see the main OED page in a web browser (normally URL of localhost:3000). Normally the database is automatically created and works fine. If you get errors about the database then you need to address them before you can do this or get the test data in (if you get errors during this process).
The import of this data into OED uses the CSV import feature that is described in the CSV import help and pages linked from that page, esp. about readings. You do not need to know all those details but can get further information as needed.
The simple and mathematical test data is created in OED by doing the following (they are described next):
npm run testData
. This may take a few minutes but when it
completes there should be new meters with data, new groups, the DB structures are all updated and the CSV
files used
will be removed unless there was an error. This generates all
the standard test data except the one where readings are every minute since that is a large dataset that is
less frequently used. See the next step before you
can use the new OED information within the OED web page.If you only want to load some of the data (not the common usage), you can look at package.json for the individual commands to insert the mathematical data. The others are so small/fast that they are done as above.
The simple/basic test data is not representative of real meter data nor mathematical but it is easy to use. Each meter has readings for five days from June 15, 2021. All the line graphs should be a straight line where value go 15 unless otherwise noted in the name. The meters and units are based on the simplified example in the Resource Generalization document with the addition of kW (flow data) and Fahrenheit/Celsius (raw data). The following table gives the expected values for the line graphs for all units for these simple meters where NA indicates that unit is not allowed. The item for each meter/row that is marked with a ^{*} indicated the default graphic unit if one is not already selected. Note that the temperature data will not be a straight line unless viewed as raw data when graphed. It will vary modestly from that with daily points. Also, a meter without a unit cannot be graphed and OED switches the displayable to false in this case. The meter "Electric Utility kWh not displayable" can only be seen if logged in as an admin. Note kW and kWh are not impacted by a rate change. The Gallon and Gallon flow are so available for testing the rate choices.
Meter name  kWh  BTU  MJ  M^{3} Gas  100 W bulb  US Dollar  Euro  kg  Metric ton  kg of CO2  Metric ton of CO2  Fahrenheit  Celsius  kW  Gallon  Liter  Gallon per minute  Liter per hour 

Electric Utility kWh  1  5^{*}  3412.08  17060.4  3.6  18  9.36e2  0.468  15  0.115  0.575  0.1012  0.506  NA  NA  0.709  3.545  7.09e4  3.545e3  NA  NA  NA  NA  NA  NA  NA 
Electric Utility kWh not displayable  1  5^{*}  3412.08  17060.4  3.6  18  9.36e2  0.468  15  0.115  0.575  0.1012  0.506  NA  NA  0.709  3.545  7.09e4  3.545e3  NA  NA  NA  NA  NA  NA  NA 
Electric Utility kWh 26  2  6^{*}  6824.16  20472.48  7.2  21.6  0.1872  0.5616  2  6  0.23  0.69  0.2  0.6  NA  NA  1.418  4.254  1.418e3  4.254e3  NA  NA  NA  NA  NA  NA  NA 
Electric Utility kWh in BTU  1  5  3412.08  17060.4^{*}  3.6  18  9.36e2  0.468  15  0.115  0.575  0.1012  0.506  NA  NA  0.709  3.545  7.09e4  3.545e3  NA  NA  NA  NA  NA  NA  NA 
Electric Utility kWh in MTon CO2  1  5  3412.08  17060.4^{*}  3.6  18  9.36e2  0.468  15  0.115  0.575  0.1012  0.506  NA  NA  0.709  3.545  7.09e4  3.545e3^{*}  NA  NA  NA  NA  NA  NA  NA 
Electric Utility no unit  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA 
Natural Gas BTU  2.93e4  1.465e3  1  5^{*}  1.06e3  5.27e3  2.74e5  1.37e4  2.93e4  1.465e3  2.95e6  1.48e5  2.6e6  1.3e5  NA  NA  5.28e5  2.64e4  5.28e8  2.64e7  NA  NA  NA  NA  NA  NA  NA 
Natural Gas BTU in Dollar  2.93e4  1.465e3  1  5  1.06e3  5.27e3  2.74e5  1.37e4  2.93e4  1.465e3  2.95e6  1.48e5^{*}  2.6e6  1.3e5  NA  NA  5.28e5  2.64e4  5.28e8  2.64e7  NA  NA  NA  NA  NA  NA  NA 
Natural Gas Dollar  NA  NA  NA  NA  NA  1  5^{*}  0.884.4  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA 
Natural Gas Cubic Meters  10.7  53.4  3.64e4  1.82e5  38.5  192.3  1  5^{*}  10.7  53.4  0.11  0.55  0.97  4.84  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA 
Trash Kg  NA  NA  NA  NA  NA  NA  NA  1  5^{*}  1e3  5e3  NA  NA  NA  NA  NA  NA  NA  NA  NA 
Temp Fahrenheit 0212  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  0  212^{*}  17.78  100  NA  NA  NA  NA  NA 
Temp Fahrenheit in Celsius  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  0  212  17.78  100^{*}  NA  NA  NA  NA  NA 
Electric kW  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  1  5^{*}  NA  NA  NA  NA 
Electric kW 26  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  2  6^{*}  NA  NA  NA  NA 
Water Gallon  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  15^{*}  3.7918.93  NA  NA 
Water Gallon flow 15 per minute  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  15^{*}  227.121135.6 
For bar graphs, you can easily check both day and 4week bars. The 1week bar is a little strange because the 5 days cross a week boundary so the first day (June 1) is the only day in the May 26June 1, 2021 bar and the other four days are in the June 28, 2021. The values are correct when this is considered. Back to the 1day bar graph. The value can be found by taking the day value on the line graph (shown in the table above) and multiplying by 24. For example, Electric Utility kWh as kWh has line values of 15 and bar values of 24120. In US Dollars it is 0.115  0.575 on line and 2.7613.8 on bar. This holds for both quantity and flow units. For raw units, it does not make sense to do a bar graph. For example, what is the total temperature over time (not a good question). For the 4week bar, it is the sum of the 5 daily bars. For example, the Electric Utility kWh as kWh has 1day bar values of 24, 24, 72, 96, 120 which sums to 360 and is the 4week bar value.
The following table gives groups that are created from the meters above as well as the mathematical meter data described below.
Group name  Default graphic unit  Values for default unit  Included meters  Included groups  All meters included  Compatible units  Note 

Electric Utility 15 + 26 kWh  kWh  3  11  Electric Utility kWh, Electric Utility kWh 26  none  Electric Utility kWh, Electric Utility kWh 26  see Electric Utility kWh  
Electric Utility 15 + 26 Dollar  US Dollar  0.345  1.265  Electric Utility kWh, Electric Utility kWh 26  none  Electric Utility kWh, Electric Utility kWh 26  see Electric Utility kWh  
Natural Gas Dollar Euro  Euro  0.88  4.4  Natural Gas Dollar  none  Natural Gas Dollar  see Natural Gas Dollar  
Electric Utility 15 + Natural Gas Dollar Euro  Euro  0.9812  4.906  Electric Utility kWh, Natural Gas Dollar  none  Electric Utility kWh, Natural Gas Dollar  see Natural Gas Dollar  
Electric Utility 15 kWh not displayable  kWh  1  5  Electric Utility kWh  none  Electric Utility kWh  see Electric Utility kWh  
Electric kW + 26 kW  kW  3  11  Electric kW, Electric kW 26  none  Electric kW, Electric kW 26  see Electric Utility kWh  This should only been seen and graphable by an admin 
SqSin + SqCos kWh  kWh  constant 2.5  testSqSin kWh, testSqCos kWh  none  testSqSin kWh, testSqCos kWh  see Electric Utility kWh  
SqSin + SqCos no unit  none  cannot graph  testSqSin kWh, testSqCos kWh  none  testSqSin kWh, testSqCos kWh  see Electric Utility kWh  Will graph once a unit compatible with kWh is chosen 
Amp 1 + 5 kWh  kWh  sine from 06  testAmp1Sin kWh, testAmp5Sin kWh  none  testAmp1Sin kWh, testAmp5Sin kWh  see Electric Utility kWh  
Amp 2 + 6 kWh  kWh  sine from 08  testAmp2Sin kWh, testAmp6Sin kWh  none  testAmp2Sin kWh, testAmp6Sin kWh  see Electric Utility kWh  
Amp 3 + 4 kWh  kWh  sine from 07  testAmp3Sin kWh, testAmp4Sin kWh  none  testAmp3Sin kWh, testAmp4Sin kWh  see Electric Utility kWh  
Amp 2 + (1 + 5) kWh  kWh  sine from 08  testAmp2Sin kWh  Amp 1 + 5 kWh  testAmp1Sin kWh, testAmp2Sin kWh, testAmp5Sin kWh  see Electric Utility kWh  
Amp 3 + 6 + (2 + (1 + 5)) + (3 + 4) kWh  kWh  sine from 021  testAmp3Sin kWh, testAmp6Sin kWh  Amp 2 + (1 + 5) kWh, Amp 3 + 4 kWh  testAmp1Sin kWh, testAmp2Sin kWh, testAmp3Sin kWh, testAmp4Sin kWh, testAmp5Sin kWh, testAmp6Sin kWh  see Electric Utility kWh  testAmp3Sin included twice but not double counted by OED 
Amp 6 + 7 + (1 + 5) + (2 + 6) + (3 + 4) kWh'  kWh  sine from 028  testAmp6Sin kWh, testAmp7Sin kWh  Amp 1 + 5 kWh, Amp 2 + 6 kWh, Amp 3 + 4 kWh  testAmp1Sin kWh, testAmp2Sin kWh, testAmp3Sin kWh, testAmp4Sin kWh, testAmp5Sin kWh, testAmp6Sin kWh, testAmp7Sin kWh  see Electric Utility kWh  testAmp6Sin included twice but not double counted by OED 
The OED test data generation code can generate sine and cosine based data. It has several advantages:
Note that the values are shifted so the minimum is 0 since most meter data is like that. It is easy to use this data by following these steps
The following table gives a list of the available data and information about it. The meter name assumes you use
the provided script to load the data. (If you want to generate the test data outside the usual script, the
<command name> is what you use to create this dataset manually with
npm run <command name>
in a web/vsc Docker container terminal. You will then need to
load
the reading data into OED as described on the reading import
page. The file will have the name indicated and is located in the
src/server/test/db/data/automatedTests/
directory.)
Description  Meter name  # points/file size 

File produced 

Sine function with a period of 45 days and amplitude 3 for 1 year with points every 4 days  test4DaySin  92 points/5kB  generateFourDayTestingData  fourDayFreqTestData.csv 
Sine function with a period of 45 days and amplitude 3 for 1 year with points every 4 hours  test4HourSin  2196 points/129kB  generateFourHourTestingData  fourHourFreqTestData.csv 
Sine function with a period of 45 days and amplitude 3 for 1 year with points every 23 minutes  test23MinSin  22915 points/1.4MB  generateTwentyThreeMinuteTestingData  twentyThreeMinuteFreqTestData.csv. 
Sine function with a period of 45 days and amplitude 3 for 1 year with points every 15 minutes  test15MinSin  35136 points/2.1MB  generateFifteenMinuteTestingData  fifteenMinuteFreqTestData.csv 
Cosine function with a period of 45 days and amplitude 3 for 1 year with points every 23 minutes  test23MinCos  22915 points/1.4MB  generateCosineTestingData  23FreqCosineTestData.csv 
Sine squared function with a period of 45 days and amplitude 2.5 for 1 year with points every 1 day  testSqSin  366 points/21kB  generateSineSquaredTestingData  2.5AmpSineSquaredTestData.csv 
Cosine squared function with a period of 45 days and amplitude 2.5 for 1 year with points every 1 day  testSqCos  366 points/21kB  generateCosineSquaredTestingData  2.5AmpCosineSquaredTestData.csv 
Sine function with a period of 45 days and amplitudes 17 for 2 years with points every 15 minutes  testAmp1Sin, testAmp2Sin, ...  70176 points/4.2MB each; 29.4MB total  generateVariableAmplitudeTestingData  15Freq1AmpSineTestData.csv, 15Freq2AmpSineTestData.csv, ... 
Sine function with a period of 45 days and amplitude 3 for 1 year with points every 1 minute. This is normally used for specialized testing of a large number of readings and not automatically generated by the script.  test1MinSin  527040 points/32MB  generateOneMinuteTestingData  oneMinuteFreqTestData.csv 
The following figure shows the kWh meters and groups as a line graph:
The following figure shows the kWh meters and groups as a 4 week bar graph:
Same as above but as a 1 week bar graph:
Same as above but as a 1 day bar graph:
Same as the line graphic above but in kg of CO2 to show changing the unit:
Same as above but as a 1 day bar graph:
The following figure shows the water meter as a line graph where the rate is Hour to show changing the rate and a different type of unit. Note KWh must be shown per hour.
The following figure shows the kW meters and groups as a line graph:
The following figure shows the kW meters and groups as a 4 week bar graph:
The following figure shows the water flow meter as a line graph where the rate is Hour to show changing the rate and a different type of unit. Note kW must be shown per hour.
The following figure shows a temperature meter that collects in Fahrenheit but has a default graphic unit of Celsius as a line graph:
he following picture shows the meters test4DaySin, test4HourSin,
test23MinSin & test15MinSine. It is designed to
show how the number of points impact the line quality in OED and verify that varied reading rates do not impact
the OED code. What you see is that the 4 day line is stair step because each group of 4 day points have the same
value. The other meters are very similar in value. This shows that the daily averaging in OED is accurate. Note
that in this and all line graphics, you will see the same value as you zoom in to see hourly or even minute
level points. This is because the data is kW. The only change you will see is in the quality of the graphic if
you zoom in so tight that you have limited points to graph.
The following picture shows the same meters as above but as a bar graph with 4 week bars. In a perfect world, all
the bars would have the same value for a given date range. As can be seen, there are small differences where the
value varies more as the frequency of the readings goes down. This variation is due to the inaccuracy of low
frequency readings as is normal. Note the last set of bars varies more due to a known issue in how the bar
lengths are created.
The following picture shows a line graphic of the sin values for the 7 different amplitudes (17). As expected,
they have the same period with a different amplitude. Plus, it looks nice!
The following picture is the same meters as above but for a bar graphic of 1 day that is stacked. It shows some
stair step features due to only having one bar per day but has the same shape as the line graphic. Plus, it also
looks nice!
The following picture shows a line graphic of a group containing all the sine amplitudes above but with 6
included twice. OED does not duplicate the values so the sum is a maximum of 28 or 1 + 2 + ... + 7 = 7 * 8 /
2.
The following picture shows a sin and cos with the same amplitude. It should be noted that normally sin and cos
and phase shifted so the high of one is the low of the other. You do not see that because the values were
shifted to all be positive.
The following picture shows the meters testSqSin & testSqCos along with a group containing both meters (c2s2). If
OED is working properly then the group should have a constant value since sin^2 + cos^1 = 1 or the value of the
amplitude in this case. This is a good test of whether groups work (for simple groups of only two meters). If
you want to see how accurate OED is then just graph this group and you will see that the value is always 2.5.
Note it graphs the group without a unit and that was okay because it was chosen after the meters so it uses the
unit of the meters.
The following image shows sin with readings every minute. It is a very high quality version of the first picture
with points that are less frequent. It is zoomed in to a couple of days and the hover shows it is then every
minute.This data is not automatically generated and is normally only used to test
OED's speed when there is lots of reading points.
This data can show what happens with actual data. However, it is harder to see when things are off. It is provided as an alternative to the mathematical data but will probably be used less now that that data exists.
The table below lists the types of test data we have. The first four all have 3.5 years of electric data but for different meters so the values vary. The last one is a sample file that you can edit to create your own data.
Description  Readings CSV  Size of Readings File 

Contains actual electric data on an hourly basis for about 3.5 years (Oct. 2016Apr. 2020)  threeYearA.csv  1.3MB 
Contains actual electric data on an hourly basis for about 3.5 years (Oct. 2016Apr. 2020)  threeYearB.csv  1.3MB 
Contains actual electric data on an hourly basis for about 3.5 years (Oct. 2016Apr. 2020)  threeYearC.csv  1.3MB 
Contains actual electric data on an hourly basis for about 3.5 years (Oct. 2016Apr. 2020)  threeYearD.csv  1.3MB 
Sample file to edit for your own data  sampleReadings.csv  0 MB 
Choose the file you want where clicking on the link in a web browser will download that file. Note you can load as many test data files as you want into a single OED instance.
You will want to have OED running during this process. Start it with Docker as you should have previously done. Log in as an admin user. Go to the CSV upload page using the "CSV" button in the top, right of the OED web page. You should default to the "Readings" tab which is what you want. Now do the following:
npm run updateCikAndViews
in a terminal in the web container. Then you need to refresh the web
page with OED.
Below is a line graphic with the four 3.5 year meters. If all went well then this is what you should see.
If you don't see this then something is wrong and you should not continue this process and instead ask us for help. (See link at bottom of page)
When you do a compare graph, you choose how far from current time you compare. As a result, you need current readings and ones immediately before that time to get data for the comparison graph. If you are using meter data that does not have this, you will need to create it. Here is how to do it:
date +%Z
. For what
follows the timezone is assumed to be CDT but you should change if your timezone differs.name,ipaddress,enabled,displayable,meter_type,default_timezone_meter,gps,identifier compare 1,,FALSE,TRUE,other,,,compare 1 compare 2,,FALSE,TRUE,other,,,compare 2You can then import this CSV meter file (assuming you have OED running and have the default admin user) with:
curl localhost:3000/api/csv/meters X POST F 'headerRow=true' F 'gzip=false' F 'email=test@example.com' F 'password=password' F 'csvfile=@compareMeters.csv'
.
psql U oed
to access the database terminal. You can then do the following steps within this
terminal.
select clock_timestamp() at time zone 'cdt';
insert into readings (meter_id, reading, start_timestamp, end_timestamp) (select M.id, R.reading, R.start_timestamp, R.end_timestamp from meters as M, readings as R where M.name = 'compare 1' and R.meter_id = (select id from meters where name = 'Meter 1') and R.end_timestamp > '20191231' and R.start_timestamp < '20200401');
select date_trunc('hour', clock_timestamp() at time zone 'cdt')  max(end_timestamp) as shift from readings where meter_id = (select id from meters where name = 'compare 1');
where you can replace cdt
with the appropriate timezone you used above. This will give
the time
shift needed so the date of the last end_timestamp is nearest to current hour.
update readings set start_timestamp = start_timestamp + interval '509 days 10:00:00', end_timestamp = end_timestamp + interval '509 days 10:00:00' where meter_id in (select id from meters where name in ('compare 1'));
.
You can repeat this step, properly edited, for other meters where the time shift will be the same.
You can
also add comma separated meters to the "in" list in the SQL command.\q
npm run refreshAllReadingViews
. You will need to visit the meters
page or refresh the web page as described here
in order the see the new meters. delete from readings where meter_id in (select id from meters where name in ('compare 1'));
and then start again from above to add values and then do the shift (do not need to recreate the meter).
Note you can edit the list of meters in the "in" list to the actual compare meters you have (comma
separated) if you want to delete multiple meters' readings. You can download the sample data from the table above. You can edit the sampleReadings.csv file by opening it in you preferred editor where a spreadsheet program is fine as long as you make sure to save it as a CSV file. Remove the two sample readings on lines 2 and 3 of the file and create new lines with the data you want. Note that normally the end time of the previous point is equal to the start time of the next point in real meter data. You can put in as many lines as you want with readings. Please note that if you don't put in readings for a meter then you may get unexpected results when you graph that meter in OED. This file has a header row so it is easier to see what goes in each column. Once you are done editing and save the file in CSV format, import it the same way as the provided data above except you need to check the "Header Row" box since the file has a header row. Deal with any messages to correct problems and see the readings import page for more info on dealing with issues. Note if there are lots of errors then you need to look at the log or console where OED is running to see them all.
If you cannot load the data into the database or it disappears after shutting down OED, then you likely have a database issue. To test you can do the following:
You should now be done and have the test data in your OED development system. If something did not work then you can seek help from the project by contacting us.