What Are Rules?

Rules define what data (i.e. device events) to capture, how to transform those events, and what to do with transformed events. Events not matching any rule are discarded. Without rules, your data pipeline ends prematurely, with all of your device data gushing into the abyss, instead of into your app.

Rules are written in a custom domain specific language called rule syntax.
All rules have four parts:

  1. Rule – Identifies rule

  2. Using – Transforms events

  3. Where – Selects events from data stream

  4. With – Specifies action

You can see the four clauses in the example "catch-all" rule below.

AllEvents Rule

rule AllEvents using my_map = { body = body, header = header } where true with amqp(body = json(bindings.my_map))

Rule

The rule clause identifies and names the rule.

The two elements of the rule clause are listed below.

rule <ruleName>

The rule name is any string of your choice.

Using

The using clause allows you to transform selected event data into a new structure and enhance the data using built-in functions (listed below).

In order to properly transform an event's structure, you must know the event's incoming structure. For this, you'll need to refer to our event catalog.

  • Using Clause Syntax

using <my_map> = <Array, Object, LiteralValue, StringWithInsertedFields, EventField,...> 

• The map(s) name can be any identifier of your choice (e.g. "my_map", "message", etc.)
• The map itself is any number of recursive expressions, including arrays, objects, literal values, strings with inserted fields, and event fields.

Arrays

Arrays are an ordered group of elements. This expression is most useful when the incoming event contains data that you want to convert into an ordered list. To create an array, fill two square brackets with the fields from the event you want.

The example below shows a transformation of the latitude and longitude fields in the Danlaw UDP Header into an array.

  • Example Clause with Array

using gps_coordinates = [ body.header.longitude, body.header.latitude ]
  • Example Output

[
  -83.72955975,
  42.2793934
]

Objects

Objects contain data. If you want to group related data into an object, this is easily achieved by filling curly braces with the fields from the event you want.

In the example below, we've selected only the fields we want from the message object of the Danlaw UDP Connect Event and created our own object.

  • Example Clause with Object

using my_map = { eventType = body.message.eventType, timestamp = body.message.eventTime }
  • Example Output

{
  "eventType": "Connect",
  "timestamp": "2017-06-01T11:42:13-05:00"
}

Literal Values

Literal values are values written exactly as they're meant to be interpreted. In some cases, you may want to add static fields to your event.

Let's take the example above. Maybe instead of getting an object that says, eventType = Connect inside an object, you want something easier to display, so you insert a literal.

  • Example Clause with Literal Value

using my_map = { message = 'Device connected', timestamp = body.message.eventTime }
  • Example Output

{
  "message": "Device connected",
  "timestamp": "2017-06-01T11:42:13-05:00"
}

String With Inserted Fields

You can also define whole strings and insert fields from your event into the string. A string is contained in single quote marks, preceded by a lowercase letter s, while the inserted fields are surrounded by curly braces, preceded by the $ sign.

We'll continue with the example above. Maybe you want a sentence that you can display directly on your UI, so instead of defining your map as a JSON object, you make it a string and insert the fields that provide the information you need.

  • Example Clause with String & Inserted Fields

using message = s'Device ${header.deviceId} was plugged in at ${body.message.eventTime}.'
  • Example Output

Device 6020981915 was plugged in at 2017-06-01T11:42:13-05:00.

Where

The where clause is the filter that grabs the events you're interested in from the stream of your data.

Each incoming event is evaluated against the where clause of every rule you've created. If the conditions of the event match the where clause, it will be processed by the rule according to the using and with clauses.

  • Rule Syntax

where <BooleanLiteral, FunctionCall(<FunctionArguments>)>

Boolean Values

  • Equals: eq

  • Greater Than: gt

  • Greater Than or Equals: gte

  • Less Than: lt

  • Less Than or Equals: lte

  • Conjunction: and

  • Disjunction: or

  • Contains: contains

  • Not: not

  • Starts With: startsWith

  • Ends With: endsWith

These boolean values can be combined, but order is important.

Equals

The simplest and most common boolean usage is the eq value, which looks for an exact match in a field. Provide the map of the field that you're filtering on (and for this you'll need to look at the structure of the incoming event in the event catalog) and follow it with the string value you're interested in in single quotes.

Let's say you want to capture all the idling events sent by all of your devices. You know that your devices send information over the Danlaw UDP protocol, so you look at the Danlaw UDP Idling Message and see that the type field inside the message object inside the body object is what you're interested in.

  • Example Clause with Equals

where eq( body.message.type, 'IdlingMessage' )

Greater Than

The gt boolean allows you to ignore values below a threshold. Provide the map of the field that you're filtering on (and for this you'll need to look at the structure of the incoming event in the event catalog) and the follow it with your threshold value.

In the example below, the rule will look for Malfunction Indicator Lamp alerts where the driver has driven for more than 100 miles (160 km) with the light on. The incoming event structure can be seen here.

  • Example Clause with Greater Than

where gt( body.message.milDistance, 160 )

Less Than

The lt boolean allows you to ignore values above a threshold. Provide the map of the field that you're filtering on (and for this you'll need to look at the structure of the incoming event in the event catalog) and the follow it with your threshold value in single quotes.

Maybe you're doing some analysis on short trips. You filter out Danlaw UDP Trip End Events that show a trip time of less than 10 minutes (600 seconds).

  • Example Clause with Less Than

where lt( body.message.tripTime, 600 )

Conjunction

The and boolean is a powerful tool that allows you to combine the other boolean values listed here to better filter out unwanted events. To combine two boolean values, nest them inside parentheses with and in front.

Let's say you're interested in battery alerts. In order to filter out the OBD-II events that are lumped into a single event type in the UDP protocol, you would look for an exact match on two fields instead of one, using the conjunction.

  • Example Clause with And

where and( eq( body.message.type, `OBDPidAndVehicleBatteryEvent` ), eq( body.message.eventType.type, 'Battery Event` ) )

Disjunction

The or boolean is similar to and in that it combines two nested boolean values. Only one must evaluate to true to trigger the rule, instead of both. To combine two boolean values, nest them inside parentheses with or in front.

For example, you may be interested in both current and pending DTC events. Instead of writing two rules to capture these events, you could write one using the or boolean.

  • Example Clause with Or

where or( eq( body.message.type, 'VehicleCurrentDTCInfo' ), eq( body.message.type, 'VehiclePendingDTCInfo' ) )

Contains

The contains boolean allows you to filter events based on objects or fields that may not always be present in a certain event. This clause is especially useful when extracting PID data that is logged at varying frequencies in the TCP Relative Time Trip Data Events. In the parentheses after contains, provide the map that will contain the field or object that you're filtering on, followed by the string of the field or object name in single quotes.

For example, let's say your Dataloggers are collecting PID 12 (Engine RPM) every 2 seconds and PID 13 (Vehicle Speed) every second. Data from both of these PIDs are present in every other Trip Data event, but you want to capture them separately.

  • Example Clause with Contains

where contains( body.pidData, `EngineRpm` )

*Note: In order to only pass through the PID data that you're filtering on, instead of the entire set of PIDs present in the object, you'd need to make use of the transformations in the using clause).

Not

Not allows you to exclude certain parameters from a rule that would otherwise collect events meeting its criteria. While not can be used alone, it's most useful when combined with another boolean value.

For example, let's say you're interested in capturing Danlaw UDP GPS events; however, you need to exclude Danlaw UDP Heartbeat events, which are a subset of GPS events. You can use the not boolean to do this.

  • Example Clause with Not

and( eq( body.message.type, `GpsMessage` ) not( body.message.messageTriggerReason, `KeyOff` ))

With

The with clause defines what to do with events that are matched by the where clause and transformed by the using clause.

Currently, there are 3 actions supported:

  1. AMQP (amqp) – Most common (see more below).

  2. AWS DynamoDB (Destination)

  3. Webhook (Destination)

Multiple actions can be defined for a single rule. Use a comma to separate actions in the with clause.

  • Rule Syntax

with <Action(<ActionArugments>)>(, ...)

AMQP

The most commonly used action is AMQP. AMQP sends transformed events to an exchange (named actions) in your vhost on RabbitMQ. You can then use standard RabbitMQ clients to consume events from managed or unmanaged queues.

The AMQP action supports three arguments: key, headers, and body.

  1. key – (optional, string) The routing key for this event.

  2. headers – (optional, object) An object that defines miscellaneous headers for routing in RabbitMQ. Example: { misc = 'some text' }

  3. body – (required, string) Represents the message to publish to the actions topic in RabbitMQ. Example: json(bindings.json_body). The json(<args>) function converts the object as its argument to a JSON string.

  • Example Clause

with amqp(body = json(bindings.<my_map>))

Currently, the only supported encoding of data sent across the platform is JSON, so the action argument that body takes is a transformation of the map you defined in the using clause into a JSON object.

AWS DynamoDB

Send your data to cloud storage directly with the DynamoDB destination.

  • Example Clause

with Destination(item = json(bindings.my_map), destinationId = 'b3be6575-36d2-44b2-98b3-0db7ea1dfb74')

Webhook (Supported method)

Send your data directly to a configured Webhook destination.

  • Example Clause

with Destination(item = json(bindings.myMap), destinationId = '4fb6bcb8-de62-41a0-bf51-8dc700f03845')

Webhook (Deprecated method, support ending June 1, 2020)

A webhook is an HTTP message that is POSTed to a configured URL.

The webhook action supports two arguments: uri and body.

  1. uri – (required, string) Defines the HTTP endpoint to post the message.

  2. body – (required, string) Represents the message to POST to the HTTP endpoint.

  • Example Clause

with post(uri = 'https://requestb.in/12jd31ias32', body = json(bindings.post_body))
Did this answer your question?