Requesting Models

danger

The API for consuming data from the Evolve data server is currently in alpha and very likely to experience breaking changes in the future. Please provide any feedback about this API to Zepben.

The SDK provides a client to request models to a remote data server via gRPC. The service and proto definitions for this API can be found here. An implementation of the consumer server is provided with the Evolve platform data services.

When working with models, it is often impractical to load a whole model to a client due to the size of the data. This is generally not a problem however, as most use cases only operate on a small subsection of the model at a time, such as a Feeder. So, the consumer API provides the ability to request smaller portions of the model quickly and easily. The other benefit to this is you can set up many clients in parallel operating on different chunks of the model to reduce the amount of time to run any analytics you may wish to perform across the whole model.

Connecting to a server

There are many helper functions to connect to a Energy Workbench server (EWB). All authenticated connections are based on OAuth2 flows, and you will need to use the correct function for the configuration of the server, and for which credentials you plan to use. This will typically be one of the following:

1. connectInsecure(), for unauthenticated, insecure connections. Cannot be used against a secured (HTTPS) EWB.
2. connectTls(), for unauthenticated, but https connections. Can only be used if EWB is configured without auth.
3. connectWithPassword(), for authenticating with a username, password, and client ID.
4. connectWithSecret(), for authenticating with a client ID and secret, typically used for machine-to-machine services.1
5. connectWithIdentity(), for authenticating with an Azure managed identity.

Java

// Insecure
try (GrpcChannel channel = Connect.connectInsecure("example.zepben.com", 443)) {
    NetworkConsumerClient client = new NetworkConsumerClient(channel);
    GrpcResult result = client.getEquipmentContainer("container_mrid").throwOnError();
    NetworkService ns = client.getService();
}

// With SSL
try (GrpcChannel channel = Connect.connectTls("example.zepben.com", 443, "ca.cert")) {
    NetworkConsumerClient client = new NetworkConsumerClient(channel);
    GrpcResult result = client.getEquipmentContainer("container_mrid").throwOnError();
    NetworkService ns = client.getService();
}

// With SSL and OAuth2
try (GrpcChannel channel = Connect.connectWithSecret(
    "client_id",
    "client_secret",
    "example.zepben.com", // rpc hostname
    443,                  // rpc port
    "config_address",     // set to null to fetch configuration from "https://{rpc hostname}/ewb/auth"
    "config_CA_filename", // set to null to use system CAs for fetching configuration
    "auth_CA_filename",   // set to null to use system CAs for fetching access tokens
    "path/to/ca.cert"     // set to null to use system CAs for server cert verification
    )
) {
    NetworkConsumerClient client = new NetworkConsumerClient(channel);
    GrpcResult result = client.getEquipmentContainer("container_mrid").throwOnError();
    NetworkService ns = client.getService();
}

// With an Azure managed identity
try (GrpcChannel channel = Connect.connectWithIdentity(
    "client_id",
    "example.zepben.com", // rpc hostname
    443,                  // rpc port
    "path/to/ca.cert"     // set to null to use system CAs for server cert verification
    )
) {
    NetworkConsumerClient client = new NetworkConsumerClient(channel);
    GrpcResult result = client.getEquipmentContainer("container_mrid").throwOnError();
    NetworkService ns = client.getService();
}

// Or specify the authentication configuration directly:
try (GrpcChannel channel = Connect.connectWithSecret(
    "client_id",
    "client_secret",
    "https://evolve-ewb/",
    "zepben.au.auth0.com",
    "example.zepben.com", // rpc hostname
    443,                  // rpc port
    AuthMethod.Auth0,
    "auth_CA_filename",   // set to null to use system CAs for fetching access tokens
    "path/to/ca.cert"     // set to null to use system CAs for server cert verification
    )
) {
    NetworkConsumerClient client = new NetworkConsumerClient(channel);
    GrpcResult result = client.getEquipmentContainer("container_mrid").throwOnError();
    NetworkService ns = client.getService();
}

// With SSL and client authentication
try (GrpcChannel channel = new GrpcChannelBuilder().forAddress("example.zepben.com", 443)
    .makeSecure("ca.cert", "certchain.pem", "private_key").build()) {
    NetworkConsumerClient client = new NetworkConsumerClient(channel);
    GrpcResult result = client.getEquipmentContainer("container_mrid").throwOnError();
    NetworkService ns = client.getService();
}

Kotlin

// Insecure
Connect.connectInsecure("example.zepben.com", 443).use { channel ->
    val client = NetworkConsumerClient(channel)
    val result = client.getEquipmentContainer("container_mrid").throwOnError()
    val ns = client.service
}

// With just SSL
Connect.connectTls("example.zepben.com", 443, "path/to/ca.cert").use { channel ->
    val client = NetworkConsumerClient(channel)
    val result = client.getEquipmentContainer("container_mrid").throwOnError()
    val ns = client.service
}

// With SSL and OAuth2
Connect.connectWithSecret(
    "client_id",
    "client_secret",
    "example.zepben.com", // rpc hostname
    443,                  // rpc port
    "config_address",     // set to null to fetch configuration from "https://{rpc hostname}/ewb/auth"
    "config_CA_filename", // Do not pass if you intend to use system CAs for fetching EWB configuration
    "auth_CA_filename",   // Do not pass if you intend to use system CAs for fetching access tokens
    "path/to/ca.cert"     // Do not pass if you intend to use system CAs for server cert verification
).use { channel ->
    val client = NetworkConsumerClient(channel)
    val result = client.getEquipmentContainer("container_mrid").throwOnError()
    val ns = client.service
}

// With an Azure managed identity
Connect.connectWithIdentity(
    "client_id",
    "example.zepben.com", // rpc hostname
    443,                  // rpc port
    "path/to/ca.cert"     // Do not pass if you intend to use system CAs for server cert verification
    )
).use { channel ->
     val client = NetworkConsumerClient(channel)
     val result = client.getEquipmentContainer("container_mrid").throwOnError()
     val ns = client.service
}

// Or specify the authentication configuration directly:
Connect.connectWithSecret(
    "client_id",
    "client_secret",
    "https://evolve-ewb/",
    "zepben.au.auth0.com",
    "example.zepben.com", // rpc hostname
    443,                  // rpc port
    AuthMethod.Auth0,
    "auth_CA_filename",   // Do not pass if you intend to use system CAs for fetching access tokens
    "path/to/ca.cert"     // Do not pass if you intend to use system CAs for server cert verification
).use { channel ->
    val client = NetworkConsumerClient(channel)
    val result = client.getEquipmentContainer("container_mrid").throwOnError()
    val ns = client.service
}

// With SSL and client authentication
GrpcChannelBuilder().forAddress("example.zepben.com", 443).makeSecure("ca.cert", "certchain.pem", "private_key").build().use { channel ->
    val client = NetworkConsumerClient(channel)
    val result = client.getEquipmentContainer("container_mrid").throwOnError()
    val ns = client.service
}

Network Hierarchy

The network can be built with a hierarchy as discussed earlier here. This allows you to easily identify and request smaller chunks of the network, so you can focus on areas of concern. Here is an example of how to request the network hierarchy and print it out as a tree to the console.

Java

void printNetworkHierarchy(NetworkConsumerClient client) {
    NetworkHierarchy hierarchy = client.getNetworkHierarchy().getResult();
    if (hierarchy == null)
        return;

    hierarchy.getGeographicalRegions().values().forEach(region -> {
        System.out.println(String.format("- %s [%s]", region.getName(), region.getMRID()));
        region.getSubGeographicalRegions().values().forEach(subRegion -> {
            System.out.println(String.format("  |- %s [%s]", subRegion.getName(), subRegion.getMRID()));
            subRegion.getSubstations().values().forEach(substation -> {
                System.out.println(String.format("    |- %s [%s]", substation.getName(), substation.getMRID()));
                substation.getFeeders().values().forEach(feeder -> {}
                    System.out.println(String.format("      |- %s [%s]", feeder.getName(), feeder.getMRID()));
                });
            });
        });
    });
}

Kotlin

fun printNetworkHierarchy(client: NetworkConsumerClient) {
    val hierarchy: NetworkHierarchy = client.getNetworkHierarchy().result ?: return
    hierarchy.geographicalRegions.values.forEach { region ->
        println("- ${region.name} [${region.mRID}]")
        region.subGeographicalRegions.values.forEach { subRegion ->
            println("  |- ${subRegion.name} [${subRegion.mRID}]")
            subRegion.substations.values.forEach { substation ->
                println("    |- ${substation.name} [${substation.mRID}]")
                substation.feeders.values.forEach { feeder ->
                    println("      |- ${feeder.name} [${feeder.mRID}]")
                }
            }
        }
    }
}

Each item from the hierarchy result contains an identified object mRID and its name. This simplified data structure enables you to do things like easily build a suitable UI component allowing a user to select a portion of the network they wish to use, without needing to pull back large amounts of full object data.

Requesting Identified Objects

danger

The *ConsumerClient APIs will take care of this for you, and you typically only need these functions if you're developing the consumer client APIs themselves. Make sure what you want to achieve isn't already covered by the API before delving into this code.

Identified objects can be requested to build a model client side. When identified objects are loaded, any referenced objects that have not been previously requested need to be requested explicitly.

To find the mRIDs of any references that need to be requested you can use the deferred reference functions on the service provided when requesting identified objects.

Java

void getWithBaseVoltage(NetworkService service, NetworkConsumerClient client, String mrid) {
    IdentifiedObject equipment = client.getIdentifiedObject(service, mrid).getResult();
    if (equipment == null || !(equipment instanceof ConductingEquipment)) {
        return;
    }
    
    Set<String> mrids = service.getUnresolvedReferenceMrids(Resolvers.baseVoltage(equipment));
    if (!mrids.isEmpty()) {
        client.getIdentifiedObject(service, mrids.iterator().next());
    }
}

Kotlin

fun getWithBaseVoltage(service: NetworkService, client: NetworkConsumerClient, mrid: String) {
    val equipment = client.getIdentifiedObject(service, mrid).result
    if (equipment == null || equipment !is ConductingEquipment) {
        return
    }

    service.getUnresolvedReferenceMrids(Resolvers.baseVoltage(equipment)).firstOrNull()
        ?.let { client.getIdentifiedObject(service, it) }
}

You can also query the services UnresolvedReferences in the following ways:

Java

String mrid = "feeder1";

// To get unresolved references pointing from `equipment` to other objects
List<UnresolvedReferences> references = service.getUnresolvedReferencesFrom(mrid);

for (UnresolvedReference ref: references) {
    client.getIdentifiedObject(service, ref.toMrid)
}

// To get unresolved references pointing to `equipment`
references = service.getUnresolvedReferencesFrom(mrid);

for (UnresolvedReference ref: references) {
    client.getIdentifiedObject(service, ref.from.mRID)
}

Kotlin

val mrid = "feeder1"

service.getUnresolvedReferencesFrom(mrid).forEach {
    client.getIdentifiedObject(service, ref.toMrid)
}

service.getUnresolvedReferencesTo(mrid).forEach {
    client.getIdentifiedObject(service, ref.from.mRID)
}

Service metadata

Metadata about the servers title, version, and data sources can be retrieved via the getMetadata() call.

Java

ServiceInfo serviceInfo = client.getMetadata();

System.out.println(serviceInfo.title);

System.out.println("Data sources:");
for (DataSource ds : serviceInfo.dataSources) {
    System.out.println("  " + ds.title);
    System.out.println("  " + ds.version);
    System.out.println("  " + ds.timestamp);
}

Kotlin

val serviceInfo = client.getMetadata()
println(serviceInfo.title)

println("Data sources:")
serviceInfo.dataSources.forEach {
    println("  ${ds.title}")
    println("  ${ds.version}")
    println("  ${ds.timestamp}")
}