TopolearnKnowing where data lives and what you pay for prevents both permission confusion and surprise bills.
BigQuery is serverless: there is no cluster to size or start. You submit a query as a job, Google's infrastructure runs it on as much compute as it needs, and you pay either for the data scanned or for reserved capacity.
Resources form a hierarchy:
- A Google Cloud project owns billing, jobs and IAM settings.
- A dataset inside a project groups tables, views, routines and models. It has a fixed location (
US,EU,europe-west3...) chosen at creation, and a query can only read datasets in one location. - Tables live in datasets and are addressed as
project.dataset.table.
Storage and compute are separate. Tables are stored in a compressed columnar format on Google's distributed file system, and queries run on slots, units of CPU, memory and network, drawn from a huge shared pool or from your reservations. Because storage is columnar, a query reads only the columns it mentions. That single fact drives most cost advice.
You work with BigQuery through the Google Cloud console's BigQuery Studio (SQL editor, notebooks), the bq command-line tool, client libraries (Python's google-cloud-bigquery, Java, Go, Node.js) and tools like dbt, Looker and Looker Studio. Hundreds of public datasets in the bigquery-public-data project are free to explore; you pay only for the queries.
Example
-- Fully qualified: project.dataset.table (backticks needed with hyphens)
SELECT name, SUM(number) AS babies
FROM `bigquery-public-data.usa_names.usa_1910_2013`
WHERE gender = 'F'
GROUP BY name
ORDER BY babies DESC
LIMIT 3;
-- Mary 3728041
-- Patricia 1567660
-- Elizabeth 1528917 (values from the public dataset; may vary)
-- What tables does a dataset contain?
SELECT table_name, table_type
FROM `bigquery-public-data.usa_names.INFORMATION_SCHEMA.TABLES`;Remember
- Project → dataset → table; datasets have a fixed location
- Serverless: queries run as jobs on slots, nothing to provision
- Columnar storage means only referenced columns are read
Watch out for
- Creating datasets in different locations and then being unable to join them.