Configuring

Configuration for the load processor is specified in a JSON file.

Option	Description	Required	Default
inputPath	The path to the directory with meter energy readings CSV files.	Yes	-
processedPath	The path to the directory where successfully processed reading files are moved.	Yes	-
temp	The path to the directory where reading files that were unable to be processed are moved.	Yes	-
ewbDataPath	The path to the ewb data directory.	Yes	-
inputTimeZone	The timezone of the input load readings.	Yes	-
outputTimeZone	The desired timezone of the processed load readings.	Yes	-
daysToSearchForNetwork	The number of days to search (forwards and backwards) from a load reading to find the network model containing meter mappings	No	0
syslogFacility	The facility code to specify the type of program logging the message. Valid Values : SYSLOG, LOCAL0, LOCAL1, LOCAL2, LOCAL3, LOCAL4, LOCAL5, LOCAL6, LOCAL7	No	Disabled
syslogTag	The tag for the log message. Must be provided if you provide a syslogFacility.	No	None
maxCacheSize	The maximum number of records to cache in memory. Increasing this improves performance but requires more RAM.	No	5,000,000
preloadCacheThreshold	The number of records for a single date in a file before caching data for that whole date into memory.	No	100,000
meterFilePattern	The regex pattern used to match CSV meter reading files. Requires named capture groups (?<company>...) and (?<date>...).	No	^.*_?\d{2}M_CONS_(?<company>CITI|PCOR)_(?<date>\d{8})_(\d{8})[.]csv$
meterJsonPattern	The regex pattern used to match JSON meter reading files.	No	^.*[.]json$
companyMapping	Reference to a file providing a mapping between the company values in the network (from meter mappings) and the meter readings csvs (from file name) if they differ. See company mapping config for details.	No	None

caution

If any paths in the configuration are given as relative paths, they will be relative to the working directory of the JVM.

Sample Configuration File

{
  "inputPath": "/Users/zepben/loads/readings",
  "processedPath": "/Users/zepben/processed",
  "failedPath": "/Users/zepben/failed",
  "ewbDataPath": "/Users/zepben/ewb",
  "inputTimeZone": "+10:00",
  "outputTimeZone": "Australia/Melbourne",
  "daysToSearchForNetwork": 365,
  "syslogFacility": "SYSLOG",
  "syslogTag": "Load Processor",
  "meterFilePattern": "(?<company>.*)_(?<date>.*)[.]csv",
  "meterJsonPattern": ".*[.]json",
  "companyMapping": "/Users/zepben/company-mapping.json"
}

Company Mapping Config

The company mapping file should contain a single JSON object with the following contents:

Key	Description	Required	Default
mappings	A list of mappings between the company in the network and the readings file. See	No	None

Company Mapping Entry

Each entry in the mappings list should be a JSON object with the following contents:

Key	Description	Required	Default
network	: A string containig the company name as found in the network model, sourced from the meter mapping file.	Yes
reading	: A string containig the company name as extracted from the meter reading file name.	Yes

Sample Company Mapping File

{
  "mappings": [
    {
      "network": "network company 1",
      "reading": "reading company 1"
    },
    {
      "network": "network company 2",
      "reading": "reading company 2"
    }
  ]
}