extract.incremental.transform
IncrementalTransform Objects
class IncrementalTransform()
deduplication_disabled
@property
def deduplication_disabled() -> bool
Skip deduplication when length of the key is 0
JsonIncremental Objects
class JsonIncremental(IncrementalTransform)
find_cursor_value
def find_cursor_value(row: TDataItem) -> Any
Finds value in row at cursor defined by self.cursor_path.
Will use compiled JSONPath if present, otherwise it reverts to column search if row is dict
__call__
def __call__(row: TDataItem) -> Tuple[Optional[TDataItem], bool, bool]
Returns:
Tuple (row, start_out_of_range, end_out_of_range) where row is either the data item or None
if it is completely filtered out