Skip to main content
Version: Next

Timeseries Data Ingestion

This section describes the file formats and requirements for ingesting customer meter reading data (AMI/NMI data) into the Zepben timeseries database for use in Hosting Capacity Module work packages.

Sign Convention Reference

For information about how these channel types (B, E, K, Q) relate to power sign conventions in HCM, see Power Sign Conventions.

Reading Types and Channel Codes​

Zepben supports ingestion of meter reading data from multiple Australian Distribution Network Service Providers (DNSPs), each with their own file format specifications. All formats represent interval meter data (typically 30-minute intervals) for active and reactive power consumption and generation at customer connection points.

All formats use variations of the AEMO standard channel codes to indicate the direction and type of power flow:

Channel CodeMeaningDirection
EActive energy importCustomer consumes power (grid → customer)
BActive energy exportCustomer generates power (grid ← customer)
QReactive energy importCustomer consumes reactive power (grid → customer)
KReactive energy exportCustomer generates reactive power (grid ← customer)
Customer Perspective

Import/Export terminology is from the customer's perspective. Import means the customer draws power from the grid; Export means the customer delivers power to the grid. See Power Sign Conventions for detailed explanation.

File Format Specifications​

Essential Energy (EE)​

Essential Energy File Format Details ▼

Endeavour Energy​

Endeavour Energy File Format Details ▼

Citipower/Powercor​

Citipower/Powercor File Format Details ▼

United Energy (UE)​

United Energy File Format Details ▼

General Requirements and Rules​

CSV Format Types​

Narrow form (Essential Energy, Endeavour): One row = one reading

Wide form (Citipower/Powercor, United Energy): One row = 24-hour period (48 intervals)

CSV Formatting Rules​

  • Unknown columns are ignored by the ingestor
  • No spaces around commas: use a,b,c not a, b, c
  • Quote handling: Varies by DNSP (Citipower/Powercor and UE treat quotes as literal; Essential Energy and Endeavour parse quotes). Best practice is to avoid commas and quotes in string values.

Identifier Uniqueness​

  • Meter identifiers: Must be unique within a company
  • Channel identifiers: Must be unique only within the same meter

Duplicate Reading Handling​

Duplicate Readings Must Be Avoided

Two meter readings are considered duplicates if they share the same:

  • Meter ID
  • Channel ID
  • Reading type (e.g., reactive generation)
  • Timestamp

When duplicate readings are provided, the behavior is not well-defined. The system cannot guarantee which reading will be ingested (earliest/latest in file, or based on alphabetical file order).

Action required: Ensure duplicate meter readings are removed before ingestion.

Data Quality Recommendations​

Timestamp Consistency​

  • Use consistent timezone representation (UTC recommended)
  • Ensure timestamps align to expected interval boundaries:
    • 30-minute intervals: :00:00 or :30:00
    • 15-minute intervals: :00:00, :15:00, :30:00, or :45:00

Data Validation Checklist​

Before providing data for ingestion:

  1. Check for and remove duplicate readings
  2. Validate timestamp formats match the specification
  3. Ensure required columns are present and non-blank where specified
  4. Verify unit codes and channel types match the allowed values
  5. Check that reading values are appropriate for the unit (e.g., kWh vs Wh)