Multi URL API Load Processor

Modified on Tue, 30 Nov 2021 at 11:02 AM

Overview

Loads data from REST APIs that can be targeted with GET requests (Access to the network where the REST API resides is necessary, for distributed computation also the cluster nodes need that). A path through the retrieved JSON can be provided to further specify the content of the created rows.


Configuration



WARNING: Note that THE URL THAT SHOULD BE USED AS A QUERY is the only mandatory field. All the other fields can be configured as per requirement.


Input

The Multi URL API Load Processor operates on any input data containing a valid JSON URL.


Output

The processor has two output nodes.

  • The first output node returns the result JSON Object.
  • The second output node returns the failing URLs along with the corresponding error messages.


Example

Workflow

Example Input

In this example we're using an input table containing two valid JSON URLS, a non valid URL (here Google.de) and an empty row.    

url
http://api.plos.org/search?q=title:DNA
http://api.plos.org/search?q=title:%22Drosophila%22%20and%20body:%22RNA%22&fl=id,abstract
https://www.google.de/


Example Configuration


Result

Result JSON

Error result table



Note that the result JSON is just one column since we didn't specify any JSONPath in our configuration. In the next example we will add $.response as JSONPath.


Using JSONPATH

JsonPath uses special notation to represent nodes and their connections to adjacent nodes in a JsonPath path. There are two styles of notation, namely dot and bracket. 

Result JSON



Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select atleast one of the reasons

Feedback sent

We appreciate your effort and will try to fix the article