Remote Event

Introduction

"Remove Events are a setup of SharePoint. We use this name for the general ability of kenaflow to receive data from other systems and then execute workflows. - Workflows always consist of PowerShell scripts. This means that the data from other systems - the "remote events" - will be passed on to the scripts. Based on this data, actions can then be performed by the scripts. The workflow developer is free to use the data.

There are the following variants of "Remove Events":

  • SharePoint Remove Events: when adding or updating an SharePoint list item
  • custom data
  • unstructured data
  • link remote events

Here is a table which Workflow Types can receive which types of remote events.

SharePoint Remote Event custom data unstructured data
List Workflow yes yes -
State Machine Workflow yes yes -
Site Workflow - - yes
PowerShell Workflow - - yes
Email Workflow - - -

Remote event endpoint URLs

When remote events are active, kenaflow provides HTTP(s) endpoints to which data can be sent.

In the case of SharePoint, the remote event endpoint is registered in SharePoint. SharePoint sends requests to this endpoint when list items are edited or created.

In the case of “unstructured data”, any data can be sent to kenaflow.

The remote event endpoints are set up via the Global Configuration. Various parameters play a role. For example, SSL can be set and a host name or an IP and an HTTP(S) port.

To enable multiple instances of kenaflow to run in parallel on a server, the instance ID is also built into the URL by default.

Alternatively, an alias can be configured that is built into the URL.

The instance ID URL and the alias URL can be used in parallel.

Here is an example of a configuration:

  <section name="rer">
    <entry key="useInstanceIdForSeparation" type="System.Boolean" default="true">true</entry>
    <entry key="useAliasForSeparation" type="System.Boolean" default="False">true</entry>
    <entry key="rerAlias" type="System.String" default="kenaflow">kenaflowalias</entry>
    <entry key="rerHost" type="System.String" comment="dns of kenaflow" default="+">kenaflow.local</entry>
    <entry key="rerHostUsesSSL" type="System.Boolean" default="false">true</entry>
    <entry key="sslCertThumbprint" type="System.String" default="">0123456789ABCDEF0123456789ABCDEF</entry>
  </section>

This will enable the following endpoint URLs. These are examples with instance ID 33e5c54e-da78-44d0-a022-217572f4c8d0 and "Unstructured data" endpoint = "/uda/!!

https://kenafllow.local/33e5c54e-da78-44d0-a022-217572f4c8d0/uda/check https://kenafllow.local/kenaflowalias/uda/check

The Execution of Remote Events

Remote Events most likely need to be executed really soon after they are raised.

Besides the normal time/queue based workflow execution kenaflow has a separate event exection machanism that we call "Remote Event Runner". (Our naming "RER" means both "Remote Event Receiver" and "Remote Event Runner"!)

Remove Events always come by "http(s)". From SharePoint of from third party systems.

There is one kenaflow process that hosts all http(s) endpoints of the kenaflow instance. It is started by the "main mode" (kenaflow.exe --run). It's mode parameter is --rer ("Remote Event Receiver"). It receives all events and stores them in small files in the "queue folder".

These files are encrypted. But you can disable encryption in the global configuration, Section "RER".

By default there is at least one process per SharePoint version (2013, 2016, 2019, Online) and one for pure PowerShell workflows. This processes monitor the queue directory. They are started by the "main mode" process of kenaflow with the mode parameters:

Each one of them monitors the "queue folder" and executes the events for all workflows that belong to the runner's SharePoint version or "platform".

The runner process decrypts the events and executes it.

It knows where to find the SharePoint web and list (in case of an event coming for a SharePoint workflow). It creates a PowerShell runtime universe in a nutshell and executes the workflow script in it. Than the PowerShell universe ist destroyed before the next event is processed.

You can have multiple runners per platform - or none. You can configure it in the global configuration.

E.g. if you have an on-premises SharePoint 2016 and no other SharePoint environment than you can disable all runners except the SharePoint 2016 runner. But for SharEPoint 2016 you can specify 5 runners to process events faster.

    <entry key="rerRunnerSP2013" type="System.Int32" default="1">0</entry>
    <entry key="rerRunnerSP2016" type="System.Int32" default="1">5</entry>
    <entry key="rerRunnerSP2019" type="System.Int32" default="1">0</entry>
    <entry key="rerRunnerSPO" type="System.Int32" default="1">0</entry>
    <entry key="rerRunnerPOSH" type="System.Int32" default="1">0</entry>

Debugging of Remote Events

Debugging of remote is easy. Please read the article Debugging.

SharePoint Remove Events

External systems such as kenaflow can subscribe event notifications from SharePoint.

kenaflow registers - if configurated - subscripts for "Item Added" and "Item Updated" events. This is only possible in context of list items.

SharePoint - 2013, 2016, 2019 and ONLINE - sends http(s) requests to the subscriber. This contains XML.

You can only use http and port 80 or https and port 443. This is a limitation of SharePoint.

SharePoint remote events are send in SOAP envelops. Here is a sample:

<s:Envelope xmlns:s=""http://schemas.xmlsoap.org/soap/envelope/"">
    <s:Body>
        <ProcessOneWayEvent xmlns=""http://schemas.microsoft.com/sharepoint/remoteapp/"">
            <properties xmlns:i=""http://www.w3.org/2001/XMLSchema-instance"">
                <AppEventProperties i:nil=""true""/>
                <ContextToken/>
                <CorrelationId>00000000-0000-0000-0000-000000000000</CorrelationId>
                <CultureLCID>1031</CultureLCID>
                <EntityInstanceEventProperties i:nil=""true""/>
                <ErrorCode/>
                <ErrorMessage/>
                <EventType>ItemAdded</EventType>
                <ItemEventProperties>
                    <AfterProperties xmlns:a=""http://schemas.microsoft.com/2003/10/Serialization/Arrays"">";
                        <a:KeyValueOfstringanyType>
                            <a:Key>Title</a:Key>
                            <a:Value i:type=""b:string"" xmlns:b=""http://www.w3.org/2001/XMLSchema"">Hello kenaflow!</a:Value>
                        </a:KeyValueOfstringanyType>";
                    </AfterProperties>
                    <AfterUrl i:nil=""true""/>
                    <BeforeProperties xmlns:a=""http://schemas.microsoft.com/2003/10/Serialization/Arrays""/>
                    <BeforeUrl/>
                    <CurrentUserId>1</CurrentUserId>
                    <ExternalNotificationMessage i:nil=""true""/>
                    <IsBackgroundSave>false</IsBackgroundSave>
                    <ListId>1</ListId>
                    <ListItemId>77c52d58-bbdb-41ae-975f-eef13fd42a21</ListItemId>
                    <ListTitle>Custom List</ListTitle>
                    <UserDisplayName>Ingo</UserDisplayName>
                    <UserLoginName>sharepoint\ingo</UserLoginName>
                    <Versionless>false</Versionless>
                    <WebUrl>https://intranet19.sharepoint.farm</WebUrl>
                </ItemEventProperties>
                <ListEventProperties i:nil=""true""/>
                <SecurityEventProperties i:nil=""true""/>
                <UICultureLCID>1031</UICultureLCID>
                <WebEventProperties i:nil=""true""/>
            </properties>
        </ProcessOneWayEvent>
    </s:Body>
</s:Envelope>

Custom Data

This remote event type is also related to sharepoint list item.

It is used to trigger the workflow to execute a script on the given item.

The incoming data can be send with http method "GET" by using the query string URL portion for the data. Or method "POST" where the data is in the http request stream. If a query string portion is specified this will be uses instead of the request stream!!!

The endpoint URL looks like this:

https://incoming.kenaflow.com/<instance_guid>/cd<X>/<workflow_guid>/

The <instance_guid> is the guid of the kenflow instance. This URL part can be disabled in the global configuration.

The <X> stands for one of these alternatives:

  • A for "anonymous"
  • N from NTLM authentication
  • B for basic authentication

In case of the autentication alternatives the sender of data need to authenticate itself. kenaflow does verify the identity against Active Directory, but it does not authorize a request. It just sends the information about the user to the workflow script. The script need to decide wether to process the "remove event" or not.

The workflow script has a global parameter $eventData that contains the remove event object in case of workflow execution in context of an received remote event. Otherwise this parameter is $null

This is an object. In case of "custom data" it has the following properties:

  • __user : Login name of the user or null
  • __userIsAuthenticated : bool that indicates the authentication state. This property does not exist if __user is null.
  • __userAuthType : the used authentication type. This property does not exist if __user is null.

The data send to the workflow need to be in JSON format:

{ "__wfid" : "<workflow_guid>", "__itemid" : <item_id>, <custom_data> }

As you can see here only to parameters are required: __wfid and __itemid. They need to be specified.

When sending data to a SharePoint list or state machine workflow you need to send the ID of the related SharePoint list item to the workflow using querystring parameter __itemid.

When sending data to a SharePoint list or state machine workflow that has more than 1 connected list you need to specify the corresponding list using querystring parameter __list. It names the list by its title.

When sending data to a SharePoint list or state machine workflow that has more than 1 connected web you need to specify the corresponding web using querystring parameter __web. It names the web by its URL.

{ "__wfid" : "<workflow_guid>", "__itemid" : <item_id>, 
  "__list": "Custom List", "__web": "https://intranet.sharepoint.farm/sites/kenaflow", 
  <custom_data> }

Unstructured Data

This belongs to all types of SharePoint workflows and to PowerShell workflows.

Unlike "Custom Data" remote events, "Unstructured Data" remote events allow you to send any data to the workflow. They don't have to be sent in JSON format and don't have to contain item IDs or similar.

The entire http request is given to the workflow in the form of an object that then contains, for example, the property "__body" in which the entire request stream is stored as string. All other request data that is accessible for kenaflow is passed on to the workflow.

The endpoint URL looks like this:

https://incoming.kenaflow.com/<instance_guid>/ud<X>/<workflow_guid>/

Data have to be sent with http method "POST"! The query string is ignored but passed to the workflow script.

The <instance_guid> is the guid of the kenflow instance. This URL part can be disabled in the global configuration.

The <X> stands for one of these alternatives:

  • A for "anonymous"
  • N from NTLM authentication
  • B for basic authentication

In case of the autentication alternatives the sender of data need to authenticate itself. kenaflow does verify the identity against Active Directory, but it does not authorize a request. It just sends the information about the user to the workflow script. The script need to decide wether to process the "remove event" or not.

The workflow script has a global parameter $eventData that contains the remove event object in case of workflow execution in context of an received remote event. Otherwise this parameter is $null

This is an object. In case of "custom data" it has the following properties:

  • __user : Login name of the user or null
  • __userIsAuthenticated : bool that indicates the authentication state. This property does not exist if __user is null.
  • __userAuthType : the used authentication type. This property does not exist if __user is null.

You get an PowerShell Hashtable object of the following structure:

Name                            Value                                                              
----                            -----                                                              
__body                          test                                                               
__user                          sharepoint\ingo                                                    
__userAuthType                  Basic                                                              
__userIsAuthenticated           True                                                               
__wfid                          E7A0A1BA-5293-4B2F-9F8A-37B052733935                               
AcceptTypes                                                                                        
ContentEncoding                 System.Text.SBCSCodePageEncoding                                   
ContentLength                   4                                                                  
ContentType                                                                                        
Headers                         {Connection, Host, Expect, Content-Length}                         
HttpMethod                      POST                                                               
IsAuthenticated                 False                                                              
IsLocal                         True                                                               
IsSecureConnection              False                                                              
KeepAlive                       True                                                               
LocalEndPoint                   [::1]:40969                                                        
ProtocolVersion                 1.1                                                                
QueryString                     {qs}                                                               
RawUrl                          /kenaflow/?qs=test1                                                
Referrer                                                                                           
RemoteEndPoint                  [::1]:40970                                                        
RemoteEndPointIP                ::1                                                                
RemoteEndPointPort              40970                                                              
Url                             http://localhost/udb/E7A0A1BA-5293-4B2F-9F8A-...
UserAgent                                                                                          
UserHostAddress                 [::1]:40969                                                        
UserHostName                    localhost:40969                                                    
UserLanguages                                                                                      

When sending data to a SharePoint list or state machine workflow you need to send the ID of the related SharePoint list item to the workflow using querystring parameter __itemid.

When sending data to a SharePoint list or state machine workflow that has more than 1 connected list you need to specify the corresponding list using querystring parameter __list. It names the list by its title.

When sending data to a SharePoint list or state machine workflow that has more than 1 connected web you need to specify the corresponding web using querystring parameter __web. It names the web by its URL.

Receiving Data from a Workflow

You can receive data from a workflow that you call by using a web request.

This is currently only valid for "Unstructured Data" remove events, send to SharePoint site workflows or PowerShell workflows.

To receive data from a workflow you use /w (for PowerShell CliXML serialized return data) or /j for JSON serialized return data or /b for binary return data in the URL between /uda and the workflow id.

Example: kenaflow is running locally. There is a workflow with this config (_wfconfig.ps1):

@{
    Version = "4.0";
    Enabled = $true; 
    Type = "SHAREPOINT";
    SubType = "SITE";
    Name = "unstructured-rer-test-with-wait"; 
    Platform = "sp2013"; 
    Id = "E7A0A1BA-5293-4B2F-9F8A-37B052733935"; 
    TBE = -1; 
    ErrorHandlingScript = "errorHandling.ps1";
    spUser = "env\kenaflow";
    spPwd = "kr@ftwerk";
    Web = "https://intranet13.sharepoint.farm";
    RER = $true; 
    Script = "script.ps1";
}

if(!$kenaflow){import-module "C:\Program Files\kenaflow\kenaflow.runtime.dll";Test-KFConfig;exit}

With this workflow script:

param($wf, $web, $config, $eventData)

if($wf-eq$null){import-module "C:\Program Files\kenaflow\kenaflow.runtime.dll";Invoke-Kenaflow;exit}

return @{Test="kenaflow"; CurrentDate = [DateTime]::Now; Evt = $eventData}

From a fresh powershell console I execute the following web request:

$result = (Invoke-WebRequest "http://localhost:80/uda/j/E7A0A1BA-5293-4B2F-9F8A-37B052733935" -Body "test" -Method "POST")

See the "j/" in it?

I deserialize the response body with this and look at the two result objects

$jsonResult = ConvertFrom-Json ([System.text.encoding]::utf8.getstring($result.Content))
$jsonResult.CurrentDate
$jsonResult.Test
$jsonResult.Evt

Thats the result:

(1) is $jsonResult.CurrentDate
(2) is $jsonResult.Test
(3) is $jsonResult.Evt - the actual event object that is passed back to the calling client.

For a PowerShell serialized request it looks like this:

$result = (Invoke-WebRequest "http://localhost:80/uda/w/E7A0A1BA-5293-4B2F-9F8A-37B052733935" -Body "test" -Method "POST")

See the "w/" in it?

I deserialize the response body with this and look at the two result objects

$psResult = [System.Management.Automation.PSSerializer]::Deserialize([System.text.encoding]::utf8.getstring($result.Content))
$psResult.CurrentDate
$psResult.Test
$psResult.Evt

Thats the result:

(1) is $psResult.CurrentDate
(2) is $psResult.Test
(3) is $psResult.Evt - the actual event object that is passed back to the calling client. Here it is deserialzed not as an PSObject but as Hashtable. (JSON/the serializer/the deserializer does not know that it is a Hashtable instead of an object.)

You can use this functionality to communicate with a workflow on a website through AJAX requests. If you want to test such an request from JavaScript you could create a simple HTML page with an jQuery based AJAX request:

<html>
<script src="jquery.js"></script>
<script type="text/javascript">
function send(){
	$.post('http://localhost:80/uda/j/E7A0A1BA-5293-4B2F-9F8A-37B052733935/',
	   JSON.stringify({ "test":"kenaflow is cool!"})
    ).done(function(data, State) { 
		$('#response').html('Response data: ');
		$('#data').html(data);
	});
}
</script>
<body>
<input type="button" onclick="javascript:send();return false" value="send"/>
<div id="response"></div>
<pre id="data"></pre>
</body>
</html>

This is the result in the browser:

When you use /b to return binary data the first value in the Accept http request header will be used as Content Type in the http response header.

The response content type can also be specified by a query string parameter called __as with the content type as value. (See example below.) If this is available it is used instead of the Accept http request header.

The workflow script should return one or more byte arrays. If there are more than 1 the will be concatinated. Also possible is Base64 a return value. It is concatinated with the other results to a byte array. - Returned strings from the workflow script are tested to be Base64 and get concatinated if they are. If they are not Base64 they are converted to Byte array using encoding UTF-8. After that they are concatinated to the Byte array.

With that it would be possible to create a dynamic PDF in the browser:

https://kenaflow.local/<instance-id>/uda/b/<powershell-workflow-id>/?__as=application/pdf

The workflow script could look like this:

_wfconfig.ps1

@{
    Version = "4.0";
    Enabled = $true; 
    Type = "POWERSHELL";
    Name = "get-pdf"; 
    Id = "7c24e165-21c2-418e-ab3e-c6702837b533"; 
    TBE = -1; 
    ConfigListDefaults = @{
        "kenaflow" = "cool"
    }; 
    AlertAddresses = @(); 
    MailFrom = ""; 
    MailReply = ""; 
    Script = "script.ps1";
    RER = $true;     
}

if(!$kenaflow){import-module "D:\kenaflow\engine\kenaflow.runtime.dll";Test-KFConfig;exit}

script.ps1

param($wf, $eventData, $config)
if($wf-eq$null){import-module "D:\kenaflow\engine\kenaflow.runtime.dll";Invoke-Kenaflow;exit}

if($eventData) {
	return (New-KFPDF -html "<html><body><h1>$([Datetime]::Now)</h1></body></html>")
}

test.bat

"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" "http://kenaflow20.sharepoint.farm/6070b29c-bfd3-4f58-9d21-3a444fa0f69b/uda/b/7c24e165-21c2-418e-ab3e-c6702837b533?__as=application/pdf"

This type of remote event is described here: Links

Behaviour in Case of Errors

If an error / exception occurs during remote event execution the remote event will be postponed.

The amount of seconds is configurable in _wfconfig.ps1. If there is no value configured the setting from global configuration will be used. There is a default of 5 seconds.

Postponing will happen again and again if the remote evens fails.

There is a configurable maximum lifetime for a remote event. The setting is taken from _wfconfig.ps1. If there is no value for that the global configuration setting will be used. The default there is 120 seconds.