About Webhooks

Webhooks are ways of integrating Jiwa with other 3rd party API's.  When an event in Jiwa occurs (such as when a product is created), a 3rd party API can be notified instantly using webhooks.

Webhooks eliminate the need for polling, which can be inefficient and introduce latency between the event occurrence and when an action in response to that event occurs.

Open

Implementation

Webhooks in Jiwa are implemented by augmenting the existing REST API with webhook functionality via a plugin. It is based on a subscriber / subscription model.

Subscribers are defined in Jiwa, and a subscriber once given their SubscriberID can register a subscription to the possible webhooks available in Jiwa.  When events occur in Jiwa, a message is generated and sent to the webhook subscriptions.  The result of that message is then stored in a Message response.

Messages which fail to be sent to a subscriber are queued for retry based on some system settings.

Subscribers can also remove their own subscriptions and inspect what messages have been sent or attempted to be sent and the current status of the message.

All webhook messages are sent as a POST operation with any relevant document DTO as the body.

Enabling Webhooks

Webhooks are enabled when the REST API plugin is enabled, and either the Self Hosted service or IIS are configured and running.

An additional step is to configure the WebhooksHostURL system setting of the REST API Plugin.  This can be done via the System Configuration form, on the REST API tab.

The value should be the URL of your REST API reachable by internal Jiwa users.  For webhooks to function, each Jiwa client will POST events to this internal URL and the service will then forward those messages to subscribers.

Open

NOTE: In most circumstances the value for the WebhooksHostURL should not be http(s)://localhost or http(s)://127.0.0.1 - this must be the address your Jiwa users can reach via a HTTP POST.  If all your users are inside the firewall this can be the machine DNS name or local IP Address.  Check if the WebhooksHostURL is correct by putting the value of WebhooksHostURL in a browser address bar on all the Jiwa client machines - the REST API metadata page should appear.

Operations for use by Subscribers

The following is a list of operations subscribers can use to manage their subscriptions and related messages and message responses.  Note no authentication is required.

List Webhook Events

var client = new ServiceStack.JsonServiceClient("https://api.jiwa.com.au");

var eventsListRequest = new JiwaFinancials.Jiwa.JiwaServiceModel.WebhooksEventsGETRequest() {  };            
List<WebHookEvent> eventsListResponse = client.Get(eventsListRequest);

using (var webClient = new System.Net.WebClient())
{ 
    responsebody = webClient.DownloadString("https://api.jiwa.com.au/Webhooks/Events");
}

 curl -H 'Accept: application/json' -H 'Content-Type: application/json' -X GET https://api.jiwa.com.au/Webhooks/Events

https://api.jiwa.com.au/Webhooks/Events?format=json

[
	{
	 "Name":"debtor.created",
	 "Description":"Occurs when a new debtor (customer) is created"
	},
	{
	 "Name":"debtor.deleted",
	 "Description":"Occurs when a debtor (customer) is deleted"
	}
]

The possible events that are published by default are listed in the following table. A plugin can add more events as required.

Add a new Subscriber

Subscribers are not intended to be added by external source, and as such require authentication - see Authenticating under Consuming the REST API

Once a subscriber is added, the SubscriberID (RecID returned by the POST to /Webhooks/Subscribers) is the unique identifier you would perhaps provide to external to allow them to manage their own subscriptions.

var client = new ServiceStack.JsonServiceClient("https://api.jiwa.com.au");
var authResponse = client.Get(new ServiceStack.Authenticate() { UserName = "admin", Password = "password" });

var webhooksSubscriptionsPOSTRequest= new JiwaFinancials.Jiwa.JiwaServiceModel.WebhooksSubscribersPOSTRequest { Name = "My Test Subscriber", IsEnabled = true };
JiwaFinancials.Jiwa.JiwaServiceModel.SY_WebhookSubscription WebhooksSubscribersPOSTResponse = client.Post(WebhooksSubscribersPOSTRequest );

using (var webClient = new System.Net.WebClient())
{
	string json = Newtonsoft.Json.JsonConvert.SerializeObject(new
    	{
	        Name = "My Test Subscriber",
    	    IsEnabled = true
        });
    responsebody = webClient.UploadString("https://api.jiwa.com.au/Webhooks/Subscribers/", "POST", json);
}

 curl -H 'Accept: application/json' -H 'Content-Type: application/json' -X POST https://api.jiwa.com.au/Webhooks/Subscribers/?Name="My Test Subscriber"&IsEnabled=true

 curl -H 'Accept: application/json' -H 'Content-Type: application/json' -X POST https://api.jiwa.com.au/Webhooks/Subscribers/ -d '{"Name":"My Test Subscriber","IsEnabled":"true"}'

Add a new Subscription (Subscribe to a webhook event)

List all subscriptions for a Subscriber

Delete a Subscribers Subscription

Messages

This is a queryable request, meaning filtering, pagination, ordering and limiting what fields are returned is possible through either URL parameters or DTO property values

List all Messages for a subscriber

This is a queryable request, meaning filtering, pagination, ordering and limiting what fields are returned is possible through either URL parameters or DTO property values.

Filtered, Curated List of Messages for a subscriber

Message Statuses

Delete a Message

List all Message Responses for a Subscriber

This is a queryable request, meaning filtering, pagination, ordering and limiting what fields are returned is possible through either URL parameters or DTO property values.

Message Retries & Resilience

Webhook messages are sent to subscribers as they occur in Jiwa immediately and asynchronously - meaning it happens in the background and the time taken to send the message does not delay or impact users of Jiwa.

If a message should fail, then it is retried based on a time schedule.  All messages are sent by the REST API service, not the Jiwa clients themselves - so the Jiwa client that originally generated the webhook event does not need to remain powered on. 

Messages are persisted to a SQL Table SY_WebhookMessage, and that table is read when the REST API Service starts and unsent messages are queued for delivery.  By default messages are retried after 1 second, then 10 seconds, 100 seconds, and so on until after the 6th retry the message if marked as failed (Status 3) and no longer retried.

System settings under the "REST API Webhooks" tab of the system configuration form control how long the retry interval is, and the maximum number of retries to attempt.

This strategy of persisting the messages to an SQL table and retrying delivery of failed messages at growing intervals provides the resilience required to integrate with other API's

Creating a Designated Webhook Message Handler

By default, the API Service handles REST API requests and also manages webhook messaging and retries. It is possible to separate these concerns, which can be useful in spreading the load across computing resources. The topology is simple: two REST API Services are configured - one handles the incoming REST API requests, and the other handles webhook messaging including retries. There are 3 system settings involved that must be set appropriately:

• WebhooksHostURL - This is the URL of the main REST API service. This URL should be accessible by all clients. It is this URL to which all webhook messages are initially sent by clients.

• WebhooksHostName - This setting defines the NetBIOS name of the machine (i.e. the windows machine name) that is running the instance of the REST API service that will be responsible for sending webhook messages to their target, as well as handling retries if the target is not responding. This is the same machine as that defined in the "WebhooksHostRetrierURL" system setting.

• WebhooksHostRetrierURL - This is the URL of the machine that is running the instance of the REST API service that will be responsible for sending webhook messages to their target, as well as handling retries if the target is not responding. This URL need only be accessible to the machine defined in the WebhooksHostURL. In fact, good security practices would dictate that the URL / port (i.e. port 80) should only be accessible to the WebhooksHostURL machine. This is the same machine as that defined in the "WebhooksHostName" system setting.

Below is a diagram illustrating the setup:

Open

Here is a step-by-step tutorial that demonstrates setting up a designated webhook message handler.

Step 1 - Involved Machines

Decide which machine is going to run the REST API Service that handles requests, and which machine will run the REST API Service that acts as the webhook message handler. In this tutorial there is also a third machine defined - the SQL Server hosting the Jiwa database:

• Machine 1:

  • NetBIOS (windows) name: API1

  • DNS name: api1.company.com

  • Private IP Address: 10.0.0.1

• Machine 2:

  • NetBIOS (windows) name: API2

  • DNS name: api.company.com

  • Private IP Address: 10.0.0.2

• Machine 3:

  • NetBIOS (windows) name: SQL

  • DNS name: sql.company.com

  • Private IP Address: 10.0.0.3

Note that all machines are on the same subnet (10.0.0.0/24), and DNS names are already configured (configuring DNS names is outside the scope of this document).

Step 2 - Machine 3 Configuration (SQL)

We start with this machine as this is where the Jiwa databases will be hosted. You likely already have this machine configured and running. In this tutorial the machine is Windows Server 2022. On this, we will be installing and configuring Microsoft SQL Server 2022, and then in subsequent steps we shall create a Jiwa demonstration database for use during this tutorial.

• Install SQL Server 2022 trial. In our testing we used the SQL Server On-premises "Evaluation" edition. 

  Open

  

• Add a firewall rule to allow sqlsvr.exe through.

Open

Open

Open

Open

Open

Step 3 - Machine 1 Configuration (API1)

This machine will handle REST API requests. It also receives webhook messages but will forward those over to API2 for processing.

• Install Jiwa 7.2.1 SR19 or later.

• Create a Jiwa demonstration database on the SQL machine using the Jiwa client installed in the previous step (remember to use the machine name, "SQL", or the machine's private IP address, 10.0.0.3, to connect).

• Import the latest REST API plugin into the database. The REST API plugin must be v7.2.1.62 or later. This is the version that ships with Jiwa 7.2.1 SR19. Follow the instructions given here to extract the REST API plugin from the installed service release files.

• Configure the REST API system settings as below and then save the plugin.

  • WebhooksHostURL = http://api1.company.com

  • WebhooksHostName = API2

  • WebhooksHostRetrierURL = http://api2.company.com

• Configure the Jiwa 7 API windows service by editing JiwaAPISelfHostedService.exe.config (by default found at C:\Program Files (x86)\Jiwa Financials\Jiwa 7) with these values:

  • ServerName = SQL

• In services.msc, change the Jiwa 7 API service to have a "Startup type" of automatic, then start the service. You can use the windows event viewer to troubleshoot any startup errors.

• Check that the Jiwa 7 API service is listening by opening a web browser and visiting http://localhost on this machine (API1). You should see something similar to the following:

  Open

  

• To allow other machines to also reach the API listening on this machine, you must open TCP port 80 on the windows firewall:

Open

Open

Open

Open

Open

Open

• Test that other machines can access the API service running on API1 by logging into the SQL machine and using a web browser to visit http://api1.company.com.

• Edit the hosts file on API1 so that operations between internal machines use the internal network and do not try to go out onto the Internet and then back into the internal network.

Step 4 - Machine 2 Configuration (API2)

This machine (API2) will handle webhook messages. The API1 machine will forward any webhook messages it receives via the REST API plugin system setting "WebhooksHostURL" to this machine (API2). API2 then sends the webhook message off to its target, and handles retries if required. API2 records results in the SY_WebhookMessage and SY_WebhookMessageResponse tables in the Jiwa database - you should look here to troubleshoot non-sent messages.

• Install Jiwa 7.2.1 SR19 or later.

• Configure the Jiwa 7 API windows service by editing JiwaAPISelfHostedService.exe.config (by default found at C:\Program Files (x86)\Jiwa Financials\Jiwa 7) with these values:

  • ServerName = SQL

• In services.msc, change the Jiwa 7 API service to have a "Startup type" of automatic, then start the service. You can use the windows event viewer to troubleshoot any startup errors.

• Check that the Jiwa 7 API service is listening by opening a web browser and visiting http://localhost on this machine (API2). You should see something similar to the following:

  Open

  

• We need to allow port 80 TCP through the firewall, but it is only API1 that requires access (to forward on to this machine the webhook messages it receives). To this end, we shall configure a firewall rule that opens TCP port 80 only to the API1 machine.

Open

Open

Open

Open

Open

Open

Open

Open

• Test that the API1 can access the API service running on API2 by logging into the API1 machine and using a web browser to visit http://api2.company.com. Test that other machines can NOT access the API service running on API2 by logging into the SQL machine and using a web browser to visit http://api2.company.com.

•  

  • Edit the hosts file on API1 so that operations between internal machines use the internal network and do not try to go out onto the Internet and then back into the internal network.

Step 5 - Test Webhooks

We can test webhooks by creating a webhook subscriber, adding a webhook subscription, and using a target provided to us by https://webhook.site. https://webhook.site is a website that can be used for testing webhooks. It tells us when a webhook message is received, as well as the IP address it came from. We want to make sure that our webhook messages always come from the API2 machine, and not API1.

• Go to https://webhook.site, clear any requests if some exist, then copy the URL provided under the "Your unique URL" heading. Leave this website open in your web browser.

• Follow the tutorial here to add a webhook subscriber and a webhook subscription, however use the URL copied from the step above for the subscription "URL" value.

• Log in to Jiwa on API1.

• Perform a stock transfer to cause the webhook created previously to fire.

• Go back to the https://webhook.site page you left open in the web browser. Check that you have received a request (this is the webhook that we fired), and make sure that the "Host" IP address of the request is the public IP address of the API2 machine (the machine we designated for handling webhooks). The "Host" IP address can be found in the "Request Details" section of the https://webhook.site page when a request is selected on the left-hand side.

Tutorial - Using SwaggerUI to add a subscription

In this step-by-step example, we show how to add a subscription to the inventory.stocklevel webhook, so that a http endpoint https://example.com/api/dosomething is invoked whenever the stock level for a product changes.

Step 1 - Visit the SwaggerUI page

Visit the /swagger-ui route of your api in a web browser.  For example, for our demo Jiwa api it is https://api.jiwa.com.au/swagger-ui/

Open

Step 2 - Authenticate

Locate and expand the auth section and then expand the section for GET /auth.  Enter the UserName and password fields.

Open

Press the Try it out! button

Open

Step 3 - Create a Subscriber

Locate and expand the Webhooks section and then expand the section for POST /Webhooks/Subscribers.

Change the Parameter content type: to be application/json.

Click the json fragment in the Example Value area to pre-populate the body with the example json.