Advanced webhook usage with WunderAutomation and JSONPath

This post is a bit more technical than our usual tutorials and one that also provides a bit of background to describe how WunderAutomation deals with things behind the curtain. Regardless if you think you will ever need to use JSONPath in your project, we hope you find this an interesting read.

So you never heard of JSONPath? Well, don’t worry because it’s no where near as well known as it’s older and much more popular relative XPath. So if this is the first time you hear about it, you are probably not alone. But it’s a very practical tool so in this post we’ll talk a bit about what JSONPath is and how you can use it in WunderAutomation workflows.

What is JSONPath?

The short answer. JSONPath is to JSON documents what XPath is to XML documents. Or if you prefer, what CSS Selectors is to HTML.

So to understand JSONPath, it helps to know what XPath is. XPath is a query language for selecting nodes from an XML document and was originally designed by the World Wide Web consortium (W3C) back in 1999. In XPath, you can use a query like:

/Wikimedia/projects/project

to query an XML document for all nodes with tag name “project” that sits under a parent tag named “projects” which in turn sits under a root tag named “Wikimedia”. So in the imaginary case that you query an API that responds with an XML document, you can use XPath to pick out the one or two tags that you are the most interested in.

If this reminds you a little bit about CSS selectors it’s probably because CSS Selectors are actually also a query language for documents, but highly specialized for dealing with HTML documents and with the specific purpose of applying style rules to the node / tag that matches the query.

If you think about it, a CSS rule is a query combined with a style rule:

div.foobar {
    background-color: blue;
}

The above rule tells the browser to go find any div that has “foobar” in its class attribute and go apply a blue background color to them. A query and a rule.

But I have JSON!

Both XPath and CSS Selectors are designed to work with XML and HTML documents respectively. So what if you are like most modern developers and are mostly using APIs that return JSON documents?

Well, Stefan Goessner, professor in mechanics in Dortmund in Germany, invented JSONPath back in 2007 to solve that precise problem. Imagine you are using If you are using an API that returns country data in JSON format like:

{
	"name": "Mexico",
	"topLevelDomain": [".mx"],
	"alpha2Code": "MX",
	"callingCodes": ["52"],
	"capital": "Mexico City",
	"currencies": [{
		"code": "MXN",
		"name": "Mexican peso",
		"symbol": "$"
	}]
}

and now you want to extract the name of the main currency. You can use JSONPath to get that string using this expression:

$.currencies[0].name

This expression roughly translates to:

Start from the root
Find a member named “currencies”
Treat it like an array and extract the first element
Treat that array element as an object and return the “name” property

XPath vs JSONPath

An overview of the differences between XPath and JSONPath reveals how closely related they are:

XPath	JSONPath	Description
/	$	the root object/element
.	@	the current object/element
/	. or []	child operator
..	n/a	parent operator
//	..	recursive descent. JSONPath borrows this syntax from E4X.
*	*	wildcard. All objects/elements regardless their names.
@	n/a	attribute access. JSON structures don’t have attributes.
[]	[]	subscript operator. XPath uses it to iterate over element collections and for predicates. In Javascript and JSON it is the native array operator.
\|	[,]	Union operator in XPath results in a combination of node sets. JSONPath allows alternate names or array indices as a set.
n/a	[start:end:step]	array slice operator borrowed from ES4.
[]	?()	applies a filter (script) expression.
n/a	()	script expression, using the underlying script engine.
()	n/a	grouping in Xpath

Clearly, Stefan was well versed in XPath when defining that JSONPath syntax and picked up a lot of the ideas from there. Thanks Stefan!

How does WunderAutomation support JSONPath?

Ok, to answer that we first need to back out a bit for some perspective. WunderAutomation supports working with remote APIs in two main ways:

You can use an incoming webhook to trigger a workflow. In some cases the webhook sending us data may actually provide (POST) a JSON document and no doubt it would be very practical to extract pieces of information from that JSON document in filters and parameters.
You can create one or more REST API or webhook actions in a workflow. If one of those external webhooks returns data in JSON format and you may want to access an individual field from that response as a parameter in a subsequent action.

WunderAutomation do support both these usage scenarios via JSONPath. Let’s look at how that’s done.

Context objects

More perspective needed, before we dive into how to use JSONPath in WunderAutomation, let’s just briefly talk about objects and context. Any trigger that starts a workflow will introduce one or more objects into the current context. You will see this quite clearly when you select your workflow trigger:

In the image above we see that the “Post published” trigger will add a post object and a user object into the context. Additionally (almost) all triggers also introduces the currentuser object into the context, so when the workflow using the “Post published” trigger starts there will be a total of three objets in the context:

The Post object that represent the post that just got published and makes it possible to use Post related filters and parameters like post title, publish date etc.
The User object that represents the author / owner of the post object. This makes it possible to use parameters like user email or user first name etc.
The currentuser object that represents the currently logged in user when the trigger fired.

The difference between the user and currentuser may be confusing but imagine a case where user Anna has written a blog post but her manager Bella is the one actively clicking on the publish button when the post is reviewed. In that case the user object will still be Anna, but currentuser will be Bella.

For the most part objects is a 1 to 1 representation of a WordPress object. But look what happens when we use a webhook trigger:

We also get an additional object aptly named webhook into the object context. This little object is a way to gain access to any data what was sent along with the webhook so that we can use it in filters and parameters. And that ability brings us to the subject of this entire post, JSONPath.

JSONPath from trigger data in filters

The first thing that happens when a trigger fires is that the workflow filters are evaluated. At this point the only objets that exists in the context are the ones that was provided by the trigger.

As you may or may not know, the trigger you select to start your workflow will enable or disable individual filters in the filter selector. Once you select the webhook trigger you should see that the Webhook parameter filter becomes available:

A side note: It’s easy to think that the Webhook parameter filter is becomes available because we selected the Webhook trigger specifically. But the technical explanation is actually that the filter becomes available for any trigger that provides the webhook object. As it happens, there are triggers available in some premium add-ons that also provides that same object.

When you select the webhook parameter filter, you will be presented with a few extra input options:

Let’s look at what these two extra fields are used for:

Field	Description
Parameter name	Name of a parameter provided via the webhook request
Object path	Optional. If the parameter defined above is a complex type (object or array), Object path is a JSONPath expression to pick out any sub field in that complex type.

At last! About 1200 words into this post we finally get to see some JSONPath entered into an actual workflow.

Let’s assume that we fill out the parameter name and object path like this:

….then we pass the following JSON document to the workflow :

{
	"foobar": {
		"fieldA": "value",
		"fieldB": "other value",
		"list": [{
				"name": "foobar"
			},
			{
				"name": "Matt"
			},
			{
				"name": "Dave"
			},
		]
	}
}

…then the filter would pass.

JSONPath from trigger data in parameters

We saw above that selecting the webhook trigger to start a workflow will make the webhook filter available for selection. What we didn’t mention is that there is also a webhook parameter that becomes available:

Using the webhook parameter we can get access to the webhook data when setting up an action. Let’s assume that we have a webhook that provides the same JSON document as above, we could fire of an email like this:

As usual, you don’t have to remember the syntax for that expression, you can just use the parameter editor, click the webhook “pill” in the parameters box to bring up the editor:

You can use any JSONPath expression to pick out the interesting piece of the JSON document you need to.

JSONPath in HTTP request actions

The third place we can use JSONPath to improve our workflows is when executing the actions. There are two built in actions in WunderAutomation that fires of HTTP requests to remote services:

The REST API Call action
The Webhook action

The main difference between these two actions is the user interface when composing the HTTP request. The Webhook action is probably easier to use for most people because it provide more guidance in adding parameters. The REST API Call action is much more “free form” and requires more knowledge about how to write a fully working request payload.

The interesting thing from a JSONPath perspective is what happens when the HTTP request is fired and there’s (often) a response containing data. At this point, WunderAutomation adds a new parameter that can be used in subsequent actions.

Note: At the time of writing this, there’s no UI to reflect this (obviously something that will be addressed is future releases).

The new parameter name depends on the type of action you used:

Action	Parameter name
REST API Call	rest.response
Webhook	webhook.response

Knowing about this parameter makes it possible to use response data. In a workflow with multiple actions you can first run an action that fetches data from an external API and then use the response from that to update a post custom field.