Skip to content
Toolforge

2. The project

A dbt project is a directory with a particular shape. Learn the shape once and every dbt project you ever open is navigable.

The two files that matter

Section titled “The two files that matter”
   my_project/
   │
   ├── dbt_project.yml   ← in git. describes the PROJECT.
   │                       "what is this thing, and how
   │                        should its models behave?"
   │
   └── (elsewhere)
       profiles.yml      ← NOT in git. describes the CONNECTION.
                           "which warehouse, which
                            credentials, which schema?"

The split is the important part. Project configuration is shared and versioned; connection configuration is personal and secret. Your teammate checks out the same dbt_project.yml and gets the same models, but their profiles.yml points at their own development schema with their own credentials.

Get this distinction and half of dbt’s setup confusion disappears.

dbt_project.yml

Section titled “dbt_project.yml”

The one file that must exist. Its presence is what makes a directory a dbt project — dbt walks up from your working directory looking for it.

name: 'jaffle_shop'
version: '1.0.0'
config-version: 2


# which profile in profiles.yml to use
profile: 'jaffle_shop'


# where dbt looks for each kind of file
model-paths:    ["models"]
seed-paths:     ["seeds"]
test-paths:     ["tests"]
macro-paths:    ["macros"]
snapshot-paths: ["snapshots"]


# where dbt writes its own output
target-path:  "target"
clean-targets: ["target", "dbt_packages"]


# default configuration for models, by folder
models:
  jaffle_shop:
    +materialized: view
    staging:
      +materialized: view
      +schema: staging
    marts:
      +materialized: table
      +schema: marts

Three things to notice.

profile: is a pointer, not a connection. It names a key that dbt will look up in profiles.yml. The project says “I need the jaffle_shop profile”; the profile says what that means on this machine.

The models: block is defaults by folder. The nesting mirrors your directory tree under models/. Anything under models/staging/ materialises as a view; anything under models/marts/ as a table. Individual models override it. Chapter 11 covers the full precedence rules, which are worth knowing because they surprise people.

The + prefix means “this is a config, not a folder”. Without it, dbt would think materialized was a subdirectory name. It is an ugly bit of syntax and you will forget it at least once.

profiles.yml

Section titled “profiles.yml”

Where the credentials live. By default at ~/.dbt/profiles.yml, outside the project, so it never lands in git.

jaffle_shop:
  target: dev
  outputs:


    dev:
      type: snowflake
      account: xy12345.eu-west-1
      user: sarah@example.com
      password: "{{ env_var('DBT_PASSWORD') }}"
      role: transformer
      warehouse: transforming_wh
      database: analytics
      schema: dbt_sarah        # ← personal dev schema
      threads: 8


    prod:
      type: snowflake
      account: xy12345.eu-west-1
      user: dbt_service
      password: "{{ env_var('DBT_PROD_PASSWORD') }}"
      role: transformer
      warehouse: transforming_wh
      database: analytics
      schema: analytics        # ← the real one
      threads: 16

target: dev is the default output. dbt run uses it. dbt run --target prod uses the other. That one flag is the entire mechanism for environments — the same code, a different destination.

schema: under a dev target should be personal. The near-universal convention is dbt_<yourname>. It means every developer builds into their own sandbox and nobody overwrites anybody. Chapter 11 explains how a model’s final schema is actually computed, because it is not simply this value.

Never put a real password in this file. env_var() reads from the environment; dbt fails loudly if the variable is missing, which is the behaviour you want.

Check what dbt is actually going to do before you run anything:

Terminal window
dbt debug              # test the connection, print resolved config
dbt debug --config-dir # print where dbt is looking for profiles.yml

dbt debug is the first command to reach for when something will not connect. It tells you which profile it found, which target, and whether the credentials work.

Adapters

Section titled “Adapters”

dbt Core on its own cannot talk to any database. The connection logic lives in an adapter — a separate Python package per warehouse.

   ┌──────────────────────────────────────┐
   │  dbt Core                            │
   │  parsing, Jinja, the DAG, the CLI    │
   └──────────────┬───────────────────────┘
                  │  dbt-adapters interface
       ┌──────────┼──────────┬──────────┐
       ▼          ▼          ▼          ▼
   dbt-snowflake  dbt-       dbt-       dbt-
                  bigquery   databricks postgres
       │          │          │          │
       ▼          ▼          ▼          ▼
   Snowflake   BigQuery   Databricks  Postgres

You install the adapter and it pulls dbt Core in with it:

Terminal window
pip install dbt-snowflake      # brings dbt-core along

The type: key in profiles.yml selects which adapter handles the connection.

Adapters are why dbt SQL is not quite portable. The adapter translates dbt’s concepts — create this table, merge these rows, describe this relation — into the dialect of one warehouse. It does not translate your SQL. A model using Snowflake’s qualify will not run on Postgres. dbt gives you portable structure, not portable SQL. Chapter 8 covers adapter.dispatch, which is how packages write macros that do work everywhere.

Adapters are maintained at different levels. Snowflake, BigQuery, Databricks, Redshift and Postgres are the well-trodden ones; there are community adapters for Trino, DuckDB, Athena, ClickHouse, Spark and dozens more, with correspondingly varied polish.

The directory layout

Section titled “The directory layout”

The layout dbt creates, and what actually goes where:

   jaffle_shop/
   ├── dbt_project.yml
   ├── packages.yml          third-party packages
   ├── package-lock.yml      resolved package versions
   │
   ├── models/               ← the bulk of the project
   │   ├── staging/
   │   │   ├── _sources.yml      raw table declarations
   │   │   ├── _staging.yml      tests + docs for staging
   │   │   ├── stg_customers.sql
   │   │   └── stg_orders.sql
   │   ├── intermediate/
   │   │   └── int_orders_joined.sql
   │   └── marts/
   │       ├── _marts.yml
   │       ├── customers.sql
   │       └── orders.sql
   │
   ├── seeds/                small CSVs, version controlled
   │   └── country_codes.csv
   │
   ├── snapshots/            slowly changing dimensions
   │   └── orders_snapshot.sql
   │
   ├── macros/               reusable Jinja
   │   └── cents_to_dollars.sql
   │
   ├── tests/                singular tests (bespoke SQL)
   │   └── assert_positive_revenue.sql
   │
   ├── analyses/             SQL that compiles but never runs
   │
   ├── dbt_packages/         installed packages (gitignored)
   └── target/               compiled SQL + artifacts
                             (gitignored)

target/ and dbt_packages/ never go in git. They are build output and vendored dependencies. The generated .gitignore handles this; check it if you cloned a project from somewhere.

target/ is the most useful directory for debugging. After any command it holds the fully-rendered SQL for every model, with all Jinja resolved. When a model behaves oddly, read target/compiled/<project>/models/.../<model>.sql and you are looking at exactly what the warehouse was sent. Chapter 7 traces how it gets there.

The staging / intermediate / marts convention

Section titled “The staging / intermediate / marts convention”

The three-layer split under models/ is not enforced by dbt. It is a convention dbt Labs published, and near enough everybody follows it.

   SOURCES        raw tables loaded by Fivetran etc.
      │           you do not own these
      ▼
   STAGING        one model per source table
      │           rename, cast, clean. no joins.
      │           1:1 with the source. always a view.
      ▼
   INTERMEDIATE   joins, pivots, the messy middle
      │           not for consumption
      ▼
   MARTS          what people actually query
                  one model per business concept
                  tables, tested, documented

The rules that make it work:

  • Staging models are the only place source() appears. Everything downstream refs a staging model. That means one file to change when a source table gets renamed.
  • Staging does not join. One source table in, one model out, renamed and cast. Resist the urge.
  • Marts are named for the business, not the source. customers, not stg_salesforce__accounts_joined_v2.

Naming follows the layers — stg_, int_, and marts unprefixed. stg_stripe__payments uses a double underscore to separate source from table.

This is worth adopting on day one. Projects that skip it end up with two hundred models in one folder and no way to tell which are safe to change. Chapter 13 comes back to it at scale.

dbt Core and dbt Cloud

Section titled “dbt Core and dbt Cloud”

Two things with one name, which causes confusion in job adverts and architecture discussions alike.

   dbt CORE                     dbt CLOUD
   ─────────────────────────    ──────────────────────────
   open source, Apache 2.0      commercial SaaS
   a CLI: `dbt run`             hosted IDE in a browser
   you run it wherever          scheduler + job runs
   you handle scheduling        logging, alerting, RBAC
   you host the docs            hosted docs, lineage UI
   free                         per-seat pricing
                                Semantic Layer serving

Everything in chapters 3 to 12 of this guide is dbt Core. The models, the graph, the tests, the artifacts, the compilation — identical either way. dbt Cloud is orchestration, hosting and collaboration wrapped around the same engine.

The realistic choice is: dbt Cloud, or dbt Core plus something that schedules it. That something is usually Airflow or Dagster, and Airflow’s chapter 15 covers the pairing.

One live caveat, as in chapter 1: the Fusion engine and the Fivetran merger are both reshaping what sits on which side of that line. Check current documentation rather than any write-up before committing.

Packages

Section titled “Packages”

Third-party dbt code — macros and sometimes models — installed from a registry, git or a local path.

packages.yml
packages:
  - package: dbt-labs/dbt_utils
    version: [">=1.1.0", "<2.0.0"]


  - package: calogica/dbt_expectations
    version: ["0.10.0"]


  - git: "https://github.com/my-org/our-internal-macros.git"
    revision: v2.1.0
Terminal window
dbt deps    # install into dbt_packages/

dbt_utils is close to mandatory. It provides surrogate_key, date_spine, union_relations, pivot, star, and a set of extra generic tests that fill obvious gaps. Most projects install it in their first week.

dbt deps writes package-lock.yml pinning exact resolved versions. Commit that file — it is what makes CI reproducible.

A package’s models become part of your DAG and build into your warehouse. Read what you are installing.

What to take from this chapter

Section titled “What to take from this chapter”
  • dbt_project.yml describes the project and is in git; profiles.yml describes the connection and is not. That split is the core of dbt’s setup.
  • profile: in the project is a pointer to a key in profiles.yml. dbt debug resolves and tests the whole chain when something will not connect.
  • --target is the entire environment mechanism. Same code, different output, one flag.
  • Use env_var() for secrets and a personal dbt_<name> schema for development.
  • Adapters are separate packages and give portable structure, not portable SQL. Your qualify clause is still Snowflake-only.
  • The layout matters: models/, seeds/, snapshots/, macros/, tests/. target/ and dbt_packages/ stay out of git.
  • target/compiled/ holds the fully-rendered SQL and is the first place to look when a model misbehaves.
  • Adopt staging / intermediate / marts on day one. Staging is 1:1 with sources, does not join, and is the only place source() appears.
  • dbt Core is the engine; dbt Cloud is scheduling and hosting around it. Everything technical in this guide is Core.
  • Install dbt_utils and commit package-lock.yml.

Next: models and ref() — how a SELECT becomes a table, and how one function call builds the entire dependency graph.

Written by Sepuri Sai Krishna

© 2026CC BY 4.0Source