StandardizeMetadata

The StandardizeMetadata processor performs field standardization.

Field standardization modifies documents so that they have a consistent structure and consistent field names. You can use field standardization so that documents which originated from different connectors use the same fields to store the same type of information.

For example, documents created by the File System Connector can have a field named FILEOWNER. Documents created by the Documentum Connector can have a field named owner_name. Both of these fields store the name of the person who owns a file. Field standardization renames the fields so that they have the same name.

Properties

Name Description
IDOL License Service

An IdolLicenseServiceImpl that provides a way to communicate with a Knowledge Discovery License Server.

Component name Identifies the field standardization rules to use. For example, specify the component name of the connector that created the documents.
Standardizer XML The path of the XML file that contains the field standardization rules. This property is optional and if not set, a default set of rules is used.
Collate Metadata Collate document metadata to provide a summary of the fields that have been seen. (You can view the list of metadata fields in the advanced UI). The metadata is stored in memory for the lifetime of the processor, unless you clear it using the advanced UI.
Collated Metadata Max Examples The maximum number of example values to store for each metadata field that is seen, when you set Collate Metadata to true.

Relationships

Name Description
success Successfully processed FlowFiles.
failure FlowFiles for which there were parsing errors.

Customize Field Standardization

The StandardizeMetadata processor has an advanced configuration interface. This includes an editor so that you can modify the field standardization rules. To open the advanced configuration interface, right-click the processor and click Advanced.

For more information about editing the standardizer XML and using the advanced UI, see Customize Field Standardization.