What is used to determine what data to ingest during an incremental run of a dataflow with a cloud-storage source?

Determining what data to ingest during an incremental run of a dataflow with a cloud-storage source primarily relies on the last-modified timestamp of the file. This timestamp indicates the most recent time the file was changed or updated. By using this information, the system can effectively identify which files are new or have been altered since the last data ingestion process. This focus on modified timestamps allows for efficient data processing by ensuring that only the relevant, changed data is moved into the target system, saving time and resources.

Using the last-modified timestamp helps avoid redundancy in data ingestion; if a file's content hasn't changed since the last run, there's no need to process it again. This method enhances performance and ensures that the data within the platform is current and only includes necessary updates.

The other options, while related to file properties, do not serve the specific purpose of identifying changes for incremental loading as effectively as the last-modified timestamp does.