This processors has been deleted, please refer to the processor Data Table Load instead.

Overview

This processor allows to search for data and load it into a workflow.

Introduction

One of the most interesting features about ONE DATA is Workflows, as OD gives the user the opportunity to configure and build complex workflows in a very interactive way.

Workflows heavily depend on Data, and to provide them with input Data there is the Dataset Load Processor.

As the name suggests, this processor is used to upload an existing Dataset into the workflow.

Configuration

First, the user has to create new workflow.

Then open the processor of interest (i.e: Dataset Load Processor).

When opening the Processor, you will see this interface:

The user has two options of selecting the Dataset that will be uploaded:

• Provide the Dataset name: the processor will search for an existing dataset with the input name, and if no dataset is found, an information warning shows-up

• Select from drop-down menu: the processor will display the Datasets that the user can read. Then the user can chose the Dataset by direct selection or by filtering from the list.

if both configuration options are filled then the configuration of "match Dataset by name" is ignored

The user can have a look at the selected Dataset via "OPEN DATASET" button.

Advanced Configuration

There are three advanced options that are filled by default:

1. Sanitize Loaded Data: specify whether the loaded data should be sanitized with default values (in order to protect from malicious Data).
2. Parser Mode: this is a mandatory field so keeping it empty or providing wrong configuration will lead to failure of loading process and thus will lead to failure of the Workflow.The user chooses one out of three options:
   1. Drop Manifold lines
   2. Create error on first manifold line
   3. Try to parse all lines
3. Warn on invalid rows: will add a warning if the processor encountered invalid rows

Output

This processor has two output nodes:

• Left Node: contains the successively loaded rows from the selected Dataset
• Right Node: contains the set of invalid rows from the selected Dataset

Example

In this example we are going to create a simple Dataset with some invalid rows to visualize the functionality of the Dataset Load processor

Related articles

Custom Input Table

Microservice Input 

Random Number Generator