Access historical data
BigQuery lets you query and restore data stored in BigQuery that has been changed or deleted within your time travel window.
Query data at a point in time
You can query a table's historical data from any point in time within the
time travel window by using a
FOR SYSTEM_TIME AS OF
clause. This clause takes a constant timestamp expression and references the
version of the table that was current at that timestamp. The table must be
stored in BigQuery; it cannot be an external table. There is no
limit on table size when using SYSTEM_TIME AS OF.
For example, the following query returns a historical version of the table from one hour ago:
SELECT *
FROM `mydataset.mytable`
FOR SYSTEM_TIME AS OF TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR);
If the timestamp specifies a time from prior to the time travel window or from before the table was created, then the query fails and returns an error like the following:
Invalid snapshot time 1601168925462 for table myproject:mydataset.table1@1601168925462. Cannot read before 1601573410026.
After you replace an existing table by using the CREATE OR REPLACE TABLE
statement, you can use FOR SYSTEM_TIME AS OF to query the previous version of
the table.
If the table was deleted, then the query fails and returns an error like the following:
Not found: Table myproject:mydataset.table was not found in location LOCATION
Restore a table from a point in time
You can restore a table from historical data by copying the historical data into a new table. Copying historical data works even if the table was deleted or has expired, as long as you restore the table within the duration of the time travel window.
When you restore a table from historical data, tags from the source table aren't copied to the destination table. Table partitioning information also isn't copied to the destination table. To recreate the partitioning scheme of the original table, you can view the initial table creation request in Cloud Logging and use that information to partition the restored table.
You can restore a table that was deleted but is still within the time travel
window by copying the table to a new table, using the @<time> time decorator.
You can't query a deleted table, even if you use a time decorator. You must
restore it first.
Use the following syntax with the @<time> time decorator:
tableid@TIMEwhereTIMEis the number of milliseconds since the Unix epoch.tableid@-TIME_OFFSETwhereTIME_OFFSETis the relative offset from the current time, in milliseconds.tableid@0: Specifies the oldest available historical data.
To restore a table, select one of the following options:
Console
You can't undelete a table by using the Cloud de Confiance console.
bq
-
In the Cloud de Confiance console, activate Cloud Shell.
At the bottom of the Cloud de Confiance console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
To restore a table, first determine a UNIX timestamp of when the table existed (in milliseconds). You can use the Linux
datecommand to generate the Unix timestamp from a regular timestamp value:date -d '2023-08-04 16:00:34.456789Z' +%s000
Then, use the
bq copycommand with the@<time>time travel decorator to perform the table copy operation.For example, enter the following command to copy the
mydataset.mytabletable at the time1418864998000into a new tablemydataset.newtable.bq cp mydataset.mytable@1418864998000 mydataset.newtable
(Optional) Supply the
--locationflag and set the value to your location.You can also specify a relative offset. The following example copies the version of a table from one hour ago:
bq cp mydataset.mytable@-3600000 mydataset.newtable
Go
Before trying this sample, follow the Go setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Go API reference documentation.
To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries.
Before running code samples, set the GOOGLE_CLOUD_UNIVERSE_DOMAIN environment
variable to s3nsapis.fr.
Java
Before trying this sample, follow the Java setup instructions in the
BigQuery quickstart using
client libraries.
For more information, see the
BigQuery Java API
reference documentation.
To authenticate to BigQuery, set up Application Default Credentials.
For more information, see
Set up authentication for client libraries.
Before running code samples, set the GOOGLE_CLOUD_UNIVERSE_DOMAIN environment
variable to s3nsapis.fr.
Node.js
Before trying this sample, follow the Node.js setup instructions in the BigQuery quickstart using client libraries. For more information, see the BigQuery Node.js API reference documentation.
To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries.
Before running code samples, set the GOOGLE_CLOUD_UNIVERSE_DOMAIN environment
variable to s3nsapis.fr.
Python
Before trying this sample, follow the Python setup instructions in the
BigQuery quickstart using
client libraries.
For more information, see the
BigQuery Python API
reference documentation.
To authenticate to BigQuery, set up Application Default Credentials.
For more information, see
Set up authentication for client libraries.
Before running code samples, set the GOOGLE_CLOUD_UNIVERSE_DOMAIN environment
variable to s3nsapis.fr.
If you anticipate that you might want to restore a table later than what is allowed by the time travel window, then create a table snapshot of the table. For more information, see Introduction to table snapshots.
You cannot restore a logical view directly. For more information, see Restore a view.
What's next
- Learn more about table snapshots.
- Learn more about Data retention with time travel and fail-safe.
- Learn more about managing tables.