Aluma Integration

JavaScript sample code for Script Task

In this article, we show how a script task can be used to integrate with Aluma. The script will extract the information of an invoice such as document type, Invoice number, Invoice date, different types of amounts (Net Total, Tax Total and Gross Total) etc. 

Preparations to set up a Aluma account and set up an API client to interact with aluma (via REST APIs of aluma).

1. If you don't have already a Aluma account, go here to create one https://www.aluma.io/home (choose anyone: Free Trial, Sign Up or Sign In).

All the above options will give you an option to set up/log in with the following methods:                                                                                    

1. User can choose any option to set up an account with Aluma.

After authentication from the selected service provider (among the above 3 options), the dashboard of aluma appears.

                                       






















Click on 'API Clients' menu option from the top header of the dashboard.

1. Following page gives you an option to create your API Client:                                                                            
   
2. 
   
   
   
   
   
   
   A popup will appear after click on 'Create API Client' button to choose your API Client name:
                                       
3. User can enter any name into the above text box to create an API Client for Aluma.
4. After click on 'Create Client' button, your Client ID and Client Secret keys are generated.                                                                  
5. User can copy the keys via copy buttons (located at the right of each text box). 

Create a workflow to extract the invoice data:

We have created a workflow to extract the invoice data from an invoice. We've also implemented the following best practices for Aluma integration:

1. Generate the access token when an existing token is about to expire or already expired. Used workflow data to store workflow variables.

2. Implemented a retry mechanism to handle permanent errors and its notifications.

3. Delete the uploaded document after the extraction process (even in case of an exception).

Script Task to extract invoice data via Aluma APIs

In order to extract the invoice data via Aluma APIs, first, we need to generate an access token and store the access token and its expiration time to workflow data. 

In the workflow designer, open the settings of the 'Extract Invoice Data via Aluma'. This will open an empty Javascript editor.

Enter the following code to get/generate the access token and its functionality:

//Method to get the valid access token from workflow data
//if expire time is lesser than 2 minutes, then generate a new access token and set it as workflow data
function getValidAccessToken() {
    var currentDateTime = moment();
    var expireDateTme = straatos.getWorkflowData('expireDateTime');
    var token = straatos.getWorkflowData('access_token');
    
    if(token == null || expireDateTme == null) {
        console.log('generating new access token, previous token is not stored at workflow data');
        token = generateAccessToken();
    } else {
        var expireDateTimeObj = moment(expireDateTme);
        var diffInSeconds = expireDateTimeObj.diff(currentDateTime, 'seconds');
        if(diffInSeconds < 120) {
            console.log('generating new token, as previous token is near about to expire or already expired...');
            token = generateAccessToken();
        }
    }
    return token;
}

//Method to generate the new Access Token via Aluma REST api
function generateAccessToken() {
    var token = '';
    try {
        straatos.ajax({
            url: 'https://api.aluma.io/oauth/token',
            method: 'POST',
            headers: {'Content-Type': 'application/json'},
            data: JSON.stringify({'client_id': '<client ID>', 'client_secret': '<client secret>'})

        }).done(function (output) {
            errorCode = 0;
            ErrorInAccessToken = 'false';
            var response = JSON.parse(output);
            token = response.access_token;
            setAccessTokenInfo(token, response.expires_in);
        }).fail(function (jqXHR, error) {
            errorCode = getHTTPStatusCode(error);
            if((errorCode >= 500 || errorCode == 408)) {
                ErrorInAccessToken = 'true';
                straatos.setError('Error in fetching access token: ' + error);
            } 
            console.log('Aluma error in fetching access token: ' + error);
        });

    } catch(err) {
        console.log('Error in fetching access token: ' + err);   
    }
    return token;
}

//Method to set the workflow data
function setAccessTokenInfo(accessToken, seconds) {
    var secondsNum = Number(seconds);
    var today = moment();
    var expirationTime = moment().add(secondsNum, 'seconds');
     
    straatos.setWorkflowData('access_token', accessToken);
    straatos.setWorkflowData('expireDateTime', expirationTime.valueOf());
}

Next, we are uploading a document (an invoice) at Aluma, in PDF format to extract the invoice data. Following are the script:

//Upload document at Aluma
function uploadDocument() {
    
    var documentInfo = straatos.adapter.getDocumentInfo();
    var fileName = documentInfo.originalURL;
    fileName = fileName.substring(fileName.lastIndexOf('/') + 1);
    var attachmentURL = documentInfo.originalURL + '?w=' + straatos.webServiceKey;
    
    console.log('attachment URL: ' + attachmentURL);
    if(attachmentURL) {
        UploadedDocumentID = uploadFile(attachmentURL, fileName);
    }
}
//Method to create a document at Aluma
function uploadFile(fileUrl, fileName) {
    var docId;
    try {
        var fileContents = straatos.decodeBase64(getFileContent(fileUrl));
        var boundaryName = 'Straatos-' + straatos.uuid();
        var boundary = '--' + boundaryName + 'rn';

        var multipart = straatos.newFormData();
        multipart.append(boundary);

        multipart.append('Content-Disposition: form-data; name="' + 'file' + '"; filename="' + fileName + '"rn');
        multipart.append('Content-Type: ' + 'text/plain' + 'rn');
        multipart.append('rn');
        multipart.append(fileContents);
        multipart.append('rn');
        multipart.append('--' + boundaryName + '--rn');

        straatos.ajax({
            url: 'https://api.aluma.io/documents',
            method: 'POST',
            data: multipart,
            contentType: 'multipart/form-data; boundary="' + boundaryName + '"',
            headers: {
                'Authorization': 'Bearer ' + getValidAccessToken(),
                'Content-Type': 'application/pdf',
            }
        }).done(function (dataString) {
            var response = JSON.parse(dataString);
            console.log('response of create document: ' + JSON.stringify(response));
            docId = response.id;
            errorCode = 0;
            ErrorInUpload = 'false';
        }).fail(function (jqXHR, error) {
            errorCode = getHTTPStatusCode(error);
            if((errorCode >= 500 || errorCode == 408)) {
                ErrorInUpload = 'true';
                straatos.setError('Error in aluma upload: ' + error);
            } 
            console.log('Error in aluma upload: ' + error);
        });
    } catch(err) {
        console.log('Error in aluma upload: ' + err);   
    }
    return docId;
}

function getFileContent(fileUrl) {
    var base64String;

    straatos.ajax({
        url: fileUrl,
        dataType: 'binary'
    }).done(function (pdf) {
        base64String = straatos.encodeBase64(pdf);
    }).fail(function (jqXHR, error) {
        straatos.setError('Get attachment error: ' + error);
    });    

    return base64String;
}

Next, we are extracting the data from an uploaded document (an invoice) such as Document Type, Invoice number, Net Total, Gross Total etc.

Some index fields such as 'DocumentType', 'InvoiceNumber', 'InvoiceDate' etc. are created to store the extracted values of an invoice.

//Method to extract invoice data using in-built extractor 'aluma.invoices.gb'
function extractInvoiceData(docID) {
    straatos.ajax({
        url: 'https://api.aluma.io/documents/' + docID + '/extract/aluma.invoices.gb',
        method: 'POST',
        headers: {'Authorization': 'Bearer ' + getValidAccessToken()}

    }).done(function (output) {
        var response = JSON.parse(output);
        errorCode = 0;
        ErrorInExtraction = 'false';
        setIndexFields(response);
    }).fail(function (jqXHR, error) {
        errorCode = getHTTPStatusCode(error);
        if((errorCode >= 500 || errorCode == 408)) {
            ErrorInExtraction = 'true';
            straatos.setError('Aluma extraction error: ' + error);
        } 
        console.log('Aluma extraction error: ' + error);
    });
}
//Method to set the index fields
function setIndexFields(jsonResponse) {
    if(jsonResponse) {
        var jsonArr = jsonResponse.field_results;
        if(jsonArr) {
            for(var i=0; i < jsonArr.length; i++) {

                switch (jsonArr[i].field_name) {

                    case 'Document Type':
                        DocumentType = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Invoice Number':
                        InvoiceNumber = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Invoice Date':
                        InvoiceDate = (jsonArr[i].result) ? jsonArr[i].result.value : null;
                        break;
                    case 'Currency':
                        Currency = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Net Total':
                        NetTotal = (jsonArr[i].result) ? parseFloat(jsonArr[i].result.text).toFixed(2) : null;
                        break;
                    case 'Tax Total':
                        TaxTotal = (jsonArr[i].result) ? parseFloat(jsonArr[i].result.text).toFixed(2) : null;
                        break;
                    case 'Gross Total':
                        GrossTotal =  (jsonArr[i].result) ? parseFloat(jsonArr[i].result.text).toFixed(2) : null;
                        break;
                    case 'PO Number':
                        PONumber = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Supplier Tax ID':
                        SupplierTaxID = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Supplier Company Number':
                        SupplierCompanyNumber = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Supplier Bank Account Number':
                        SupplierBankAccountNumber = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Supplier Bank Code':
                        SupplierBankCode = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    case 'Supplier IBAN':
                        SupplierIBAN = (jsonArr[i].result) ? jsonArr[i].result.text : null;
                        break;
                    default:
                        break;
                }
            }
        }
    }
}

Next, we are going to delete the uploaded document from Aluma which are already processed (even in case of an exception too).

//Delete the document from aluma for the given input documentID
function deleteDocument(documentID) {
    straatos.ajax({
        url: 'https://api.aluma.io/documents/' + documentID,
        method: 'DELETE',
        headers: {'Authorization': 'Bearer ' + getValidAccessToken()}

    }).done(function (output) {
        errorCode = 0;
        ErrorInDeleteDocument = 'false';
        console.log('Document has been deleted successfully for document Id: ' + documentID);
    }).fail(function (jqXHR, error) {
        errorCode = getHTTPStatusCode(error);
        if((errorCode >= 500 || errorCode == 408)) {
            ErrorInDeleteDocument = 'true';
            straatos.setError('Error in document delete: ' + error);
        } 
        console.log('Aluma document delete  error: ' + error);
    });
    
}

Script to consolidate the extraction process, retry mechanism and handle the permanent errors 

var errorCode = 0;
NumberOfRetries = 10;
var uploadedDocId = UploadedDocumentID;
AccessToken = getValidAccessToken();
if(AccessToken) {
    ErrorInAccessToken = null;
    if(!uploadedDocId) {
        uploadDocument(); 
        console.log('document uploaded with docID: ' + UploadedDocumentID);
    }
    if(UploadedDocumentID) {
        ErrorInUpload = null;
        if((ErrorInExtraction == null && ErrorInDeleteDocument == null) || (ErrorInExtraction == 'true')) {
            extractInvoiceData(UploadedDocumentID);
            console.log('document Info extracted');
            deleteDocument(UploadedDocumentID);
            console.log('document deleted successfully');
            resetErrorIndexFields();
        } else if(ErrorInDeleteDocument == 'true') {
            deleteDocument(UploadedDocumentID);
            console.log('document deleted successfully...');
            resetErrorIndexFields();
        }
    }
}

function resetErrorIndexFields() {
    ErrorInAccessToken = null;
    ErrorInUpload = null;
    ErrorInExtraction = null;
    ErrorInDeleteDocument = null;
}

Following are the index fields to use in the above script:

Script code at 'Retry Handling':

if(ErrorInAccessToken == 'true') {
    RetryCountForToken += 1; 
    if(RetryCountForToken > NumberOfRetries) {
        console.log('Access Token API is giving a permanent error.');
        straatos.setError('Access Token API is giving a permanent error.');
    }
} else {
    RetryCountForToken = 0;   
}

if(ErrorInUpload == 'true') {
    RetryCountForUpload += 1;
    if(RetryCountForUpload > NumberOfRetries) {
        console.log('Document Upload API is giving a permanent error.');
        straatos.setError('Document Upload API is giving a permanent error.');
    }
} else {
    RetryCountForUpload = 0;   
}

if(ErrorInExtraction == 'true') {
    RetryCountForExtract += 1;
    if(RetryCountForExtract > NumberOfRetries) {
        console.log('Extract Information API is giving a permanent error.');
        straatos.setError('Extract Information API is giving a permanent error.');
    }
} else {
    RetryCountForExtract = 0;   
}

if(ErrorInDeleteDocument == 'true') {
    RetryCountForDeleteDoc += 1;
    if(RetryCountForDeleteDoc > NumberOfRetries) {
        console.log('Delete Document API is giving a permanent error.');
        straatos.setError('Delete Document API is giving a permanent error.');
    }
} else {
    RetryCountForDeleteDoc = 0;   
}

console.log('++ Retry Token count: ' + RetryCountForToken + ', Upload retry count: ' + RetryCountForUpload + ', Extract retry count: ' + RetryCountForExtract + ', Del retry count: ' + RetryCountForDeleteDoc);

Configuration of Timer Boundary Event i.e. 'Delay 10 min. before retry' is:

Now, when a document hits an error, the document is retried every 10 minutes for a maximum of 'N' number of times. That means the Document is retried for 'N' times before it is marked as a 'Permanent Error', where 'N' is configurable by an index field 'NumberOfRetries'.

Permanent Error Handling

The document will be routed to a user to take action on a permanent error. The user can view the document and then decide if a document is retried or no longer be processed.

1. Add a User Task which a document is routed for by the User. In the User Task Settings, enable the Buttons "Complete' and 'Reject'. Rename the labels to 

   'Terminate' for the Reject Button and 'Retry' for the Complete Button.

2. Add an Exclusive Gateway (decision) after the Permanent Error.

3. Have one decision to route to an end Event. Click on the line of the terminate route and click settings.

As the condition Field, select _status and in 'Equals' type in 'reject'. This has the effect that when the user in web validation clicks on the reject button, the document is routed along the reject path.

4. Draw a line from the Decision to the 'Reset Counter' and then to 'Extract Invoice Data via Aluma'. No additional condition needs to be set. This will be the default routing.

Email Notification

So far, the error documents are automatically retried and if the retry fails for more than 'N' number of retries, they are assigned to a user which can then take appropriate action and retry or terminate a document.

The next step is to add an email notification so that the user is responsible for the Permanent Error task will get the information that documents require his/her attention.

Adding an email notification step just before the User Task 'Permanent Error' would notify the user when a document enters the Permanent Error. However, the notification would be triggered for each document. Hence if 5,000 documents fail within 5 minutes, then 5,000 email notifications will be sent out.

We will add a notification process that sends a new notification maximum every 30 minutes, and only if a new document has entered the 'permanent error' step in the workflow.

Set a workflow Flag if there is a permanent error

Firstly, we add a 'Script Task' step just before the 'Permanent Error' Step.

Enter the settings for 'Set Error Flag' and enter the following script:

//Use the workflow parameter 'PermError'.
var permErrorCount = straatos.getWorkflowData('PermError');

if(permErrorCount !== null || permErrorCount !== '')
{
   //If the permErrorCount is not null or not empty, then increase the PermError by one.
    straatos.setWorkflowData('PermError',(Number(permErrorCount) + 1).toString());
            
}
else{
   //if the permErrorCount is empty or null, set the PermError to 1
    straatos.setWorkflowData('PermError','1');
}

With this step, we now create a workflow variable that is larger than 0 if a document has passed into the 'Permanent Error' step.

Check for Errors every 30 Minutes

The target is now to check every  30 minutes if at least one new document has been sent to the 'Permanent Error' step.

1. Add a timer start event.
2. Change the settings of the timer start event to check every hour at 0 and 30 minutes past the hour.

               3. Route the new task to a script step 'Check Permanent Error'

The Timer Start Event will be triggered by Straatos at the time configured. In this example every 30 minutes (specifically on the hour and 30 minutes past the hour). This task is routed to the 'Check Permanent Error'

In the Permanent Error task, we want now to check if the variable 'PermError' is larger than 0. We also want to set a workflow index field to indicate if there have been new documents in 'Permanent Error'.

1. Create a new Workflow Index field 'DocsInPermanentError'
2. Use the following script in the 'Check Permanent Error' script task:

if(Number(straatos.getWorkflowData('PermError')) > 0){
    //If the PermError is larger than 0, means documents have been added to the Permanent Error stage. In this case
    //set the PermErro variable to 0 (resetting the counter) as we going to send an email notification
    //and set the workflow 'DocsInPermanentError' to 'yes'. This will be used for routing.
    straatos.setWorkflowData('PermError', '0');
    DocsInPermanentError = 'yes';
}
else{
    //If the 'PermError' is 0, then no documents have been added to the 'Permanent Error' stage since the last reset and hence
    //no notification should be sent.
    DocsInPermanentError = 'no';
}

Routing the 'Check Error' Task

Now that the script above as determined if an email notification should be sent, the task can be routed to either send an error email notification or to terminate if now notification should be sent.

1. Add an Exclusive Gateway (decision) after the 'Check Permanent Error'
2. Add a routing path to an End Event 'No Errors found' when no email notification should be send
3. Add a routing path to a 'Service Task' and name the service task to 'Send Email Notification', then route from 'Send Email Notification' to an End Event. Don't forget to configure the Email Notification Task. (use SendGrid and send the appropriate information).
4. Click on the routing path to the 'Send Email Notification' task and click on Settings. Select 'DocsInPermanentError' as 'Condition Field' and 'Equals' as 'yes'

A final complete workflow for Aluma integration with error handling mechanism