Load and query data with the bq tool
Learn how to create a dataset, load sample data, and query tables with the bq command-line tool.
Before you begin
-
In the Trusted Cloud console, on the project selector page, select or create a Trusted Cloud project.
-
Verify that billing is enabled for your Trusted Cloud project.
Ensure that the BigQuery API is enabled.
If you created a new project, the BigQuery API is automatically enabled.
-
In the Trusted Cloud console, activate Cloud Shell.
At the bottom of the Trusted Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
If you don't enable billing for the Trusted Cloud project that you use in this tutorial, then you load and query the data in the BigQuery sandbox. The BigQuery sandbox lets you learn BigQuery with a limited set of BigQuery features at no charge.
Download the file that contains the source data
The file that you're downloading contains approximately 7 MB of data about popular baby names. It's provided by the US Social Security Administration.
For more information about the data, see the Social Security Administration's Background information for popular names.
Download the US Social Security Administration's data by opening the following URL in a new browser tab:
https://www.ssa.gov/OACT/babynames/names.zip
Extract the file.
For more information about the dataset schema, see the
NationalReadMe.pdf
file you extracted.To see what the data looks like, open the
yob2024.txt
file. This file contains comma-separated values for name, assigned sex at birth, and number of children with that name. The file has no header row.Move the file to your working directory.
If you're working in Cloud Shell, click
yob2024.txt
file, and then click Upload.If you're working in a local shell, copy or move the file
yob2024.txt
into the directory where you're running the bq tool.
Create a dataset
If you launched Cloud Shell from the documentation, enter the following command to set your project ID. This prevents you from having to specify the project ID in each CLI command.
gcloud config set project PROJECT_ID
Replace PROJECT_ID with your project ID.
Enter the following command to create a dataset named
babynames
:bq mk --dataset babynames
The output is similar to the following:
Dataset 'babynames' successfully created.
Confirm that the dataset
babynames
now appears in your project:bq ls --datasets=true
The output is similar to the following:
datasetId ------------- babynames
Load data into a table
In the
babynames
dataset, load the source fileyob2024.txt
into a new table namednames2024
:bq load babynames.names2024 yob2024.txt name:string,assigned_sex_at_birth:string,count:integer
The output is similar to the following:
Upload complete. Waiting on bqjob_r3c045d7cbe5ca6d2_0000018292f0815f_1 ... (1s) Current status: DONE
Confirm that the table
names2024
now appears in thebabynames
dataset:bq ls --format=pretty babynames
The output is similar to the following. Some columns are omitted to simplify the output.
+-----------+-------+ | tableId | Type | +-----------+-------+ | names2024 | TABLE | +-----------+-------+
Confirm that the table schema of your new
names2024
table isname: string
,assigned_sex_at_birth: string
, andcount: integer
:bq show babynames.names2024
The output is similar to the following. Some columns are omitted to simplify the output.
Last modified Schema Total Rows Total Bytes ----------------- ------------------------------- ------------ ------------ 14 Mar 17:16:45 |- name: string 31904 607494 |- assigned_sex_at_birth: string |- count: integer
Query table data
Determine the most popular girls' names in the data:
bq query \ 'SELECT name, count FROM babynames.names2024 WHERE assigned_sex_at_birth = "F" ORDER BY count DESC LIMIT 5'
The output is similar to the following:
+-----------+-------+ | name | count | +-----------+-------+ | Olivia | 14718 | | Emma | 13485 | | Amelia | 12740 | | Charlotte | 12552 | | Mia | 12113 | +-----------+-------+
Determine the least popular boys' names in the data:
bq query \ 'SELECT name, count FROM babynames.names2024 WHERE assigned_sex_at_birth = "M" ORDER BY count ASC LIMIT 5'
The output is similar to the following:
+---------+-------+ | name | count | +---------+-------+ | Aaran | 5 | | Aadiv | 5 | | Aadarsh | 5 | | Aarash | 5 | | Aadrik | 5 | +---------+-------+
The minimum count is 5 because the source data omits names with fewer than 5 occurrences.
Clean up
To avoid incurring charges to your Trusted Cloud account for the resources used on this page, delete the Trusted Cloud project with the resources.
Delete the project
If you used the BigQuery sandbox to query the public dataset, then billing is not enabled for your project, and you don't need to delete the project.The easiest way to eliminate billing is to delete the project that you created for the tutorial.
To delete the project:
- In the Trusted Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down to delete the project.
Delete the resources
If you used an existing project, delete the resources that you created:
Delete the
babynames
dataset:bq rm --recursive=true babynames
The
--recursive
flag deletes all tables in the dataset, including thenames2024
table.The output is similar to the following:
rm: remove dataset 'myproject:babynames'? (y/N)
To confirm the delete command, enter
y
.
What's next
- Learn more about using the bq tool.
- Learn about the BigQuery sandbox.
- Learn more about loading data into BigQuery.
- Learn more about querying data in BigQuery.