Skip to main content

Time Travel - Query an earlier version of the table

Time travel in Databricks

Delta Lake time travel allows you to query an older snapshot of a Delta Lake table.
For timestamp_string, only date or timestamp strings are accepted. For example, "2019-01-01" and "2019-01-01'T'00:00:00.000Z".
To query an older version of a table, specify a version or timestamp in a SELECT statement. For example, to query version 0 from the history above, use:
Copy to clipboardCopy
SELECT * FROM events VERSION AS OF 0
or
Copy to clipboardCopy
SELECT * FROM events TIMESTAMP AS OF '2019-01-29 00:37:58'
Note
Because version 1 is at timestamp '2019-01-29 00:38:10', to query version 0 you can use any timestamp in the range '2019-01-29 00:37:58' to '2019-01-29 00:38:09' inclusive.
DataFrameReader options allow you to create a DataFrame from a Delta Lake table that is fixed to a specific version of the table.
Copy to clipboardCopy
df1 = spark.read.format("delta").option("timestampAsOf", timestamp_string).load("/delta/events")
df2 = spark.read.format("delta").option("versionAsOf", version).load("/delta/events")

Comments

Popular posts from this blog

Install pip on Windows

If you haven't downloaded get-pip.py yet: You need to download this file from the official pip website or use curl in your command prompt. Using curl (recommended): Open Command Prompt and type: DOS curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py This will download the file to your current directory in the Command Prompt. Manual Download: Go to https://bootstrap.pypa.io/get-pip.py in your web browser, right-click on the page, select "Save as...", and save the file as get-pip.py to a location you can easily navigate to (e.g., your Downloads folder, or a dedicated Python projects folder). The file is in a different directory: You might have downloaded get-pip.py to your Downloads folder, your Desktop, or another location, but you're trying to run the command from C:\Users\gparn . Solution: Navigate to the directory where you saved get-pip.py in your Command Prompt using the cd (change directory) command. Example (if in Downloads): DOS cd C:\User...