 | OrchestratorConfigRecordInitiator Property |
Introduces records into the processing pipeline of Data Conveyer.
This function is called once per record immediately after the record is read from intake and parsed, before being forwarded to the clustering process. There are 2 roles of this function: trace bin setup and trigger the start of the Transformation phase. Any data collected from the current record can be stored in a trace bin for access during processing of subsequent records (as well as during subsequent phases of processing). The start of the Transformation phase can be triggered by this function only in case of DeferTransformation set to UntilRecordInitiation.
The function accepts 2 parameters: current record and a trace bin object (dictionary). The function returns boolean value to trigger the start of the Transformation phase in case of DeferTransformation set to UntilRecordInitiation. Note that records are processed sequentially on Intake; after returning the first true (or if DeferTransformation setting is other than UntilRecordInitiation), the values returned by this function are inconsequential. In case RecordInitiator returns false for all records and the DeferTransformation is UntilRecordInitiation, the transformation starts after all input records have been read.
Deferral of Transformation may be useful in cases where transformation of initial clusters (e.g. head cluster) requires data that is read from some records during Intake and saved in global cache. Note though that such deferral affects performance as well as memory footprint; and in case of limited buffer sizes may lead to deadlocks.
If not supplied, Data Conveyer assumes default function that does not make any updates and returns true, i.e.: Any exception thrown by this function will cause the process shutdown with a completion status of Failed.Namespace: Mavidian.DataConveyer.Orchestrators
Assembly: DataConveyer (in DataConveyer.dll) Version: 3.4.6+1324144ff7
SyntaxSee Also