# External Data and Environment Variables

Course: [Astro](https://www.devtutorials.ai/learn/astro) · Section: Content
Source: https://www.devtutorials.ai/learn/astro/content/external-data

> Fetching data in the component script with top-level await, handling fetch errors, headless CMS, custom content loaders, databases overview, .env files, import.meta.env, PUBLIC_ variables, built-in variables and type-safe astro:env with envField.

## External Data

- So far, all of Asha's data has lived in her project: Markdown files and JSON files.
- Real sites often get data from somewhere else: an API, a CMS, or a database.
- **API** = a web address that returns data (usually JSON) instead of a page.
- In Astro, the component script can `fetch()` that data while the site is built.
- In this lesson, you will load data from an API and keep secret keys safe.

**Simple lesson:** the component script can fetch data at build time, and the page ships the result as HTML.

### A Real-World Example

- Asha keeps her reading list in an online app. It has an API.
- She wants her site's "Now reading" box to show the list.
- She does not want readers' browsers to call the API on every visit.
- So her page fetches the list once, at build time. Readers get plain HTML.

Press **▶ Run**. The **Terminal** shows the API call made during the build.

<ul>
    {books.map((book) => (
      <li>
        <strong>{book.title}</strong> by {book.author}
        {book.status === "reading" && <em> (reading now)</em>}
      </li>
    ))}
</ul>
`,
  }}
/>

What happened:

- `await fetch(...)` called the API while the page was built.
- `response.json()` turned the reply into a JavaScript list.
- The **HTML sent to browser** has the finished list. It has no `fetch` and no script.

In this lab, `https://api.devjournal.dev` is a pretend API that runs inside the preview. It has `/posts`, `/posts/:id`, `/books` and `/weather`. On your computer, `fetch()` calls the real internet.

### What You Will Learn

In this lesson, you will learn:

1. How to fetch data in the component script with `await`.
2. How to handle a failed fetch.
3. What a headless CMS is.
4. How to load API data into a collection with a custom loader.
5. Where databases and backend services fit in.
6. How `.env` files and `import.meta.env` work.
7. Why `PUBLIC_` variables are different.
8. How `astro:env` checks your variables for you.

## Fetching Data

### fetch() in the Component Script

- `fetch()` is the standard way to call a web address from JavaScript.
- In the component script, it runs on your computer during the build, not in the reader's browser.
- So it can call APIs that need a secret key. The key never reaches the browser.

### Top-Level await

- **await** = "wait for this result before going on".
- In the component script, you can use `await` directly, at the top level. No `async` function is needed.
- Astro waits for the data before it builds the HTML.

### Fetching from a Remote API

- A typical pattern:
  1. `const res = await fetch(url);`
  2. `const data = await res.json();`
  3. Use `data` in the template.
- In a static site, this happens once per build. To show new data, you build again.

<figure className="ast-fig ngc-fig">
<svg viewBox="0 0 640 210" role="img" aria-labelledby="fig-fetch" style={{ width: "100%", maxWidth: 640, height: "auto" }}>
  <title id="fig-fetch">Animation: during the build, the page asks the API for data once. The finished HTML then goes to many readers, who never call the API.</title>
  <defs><marker id="ft-arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0 L10 5 L0 10 z" style={{ fill: "var(--ngc-slate)" }} /></marker></defs>
  <text x="170" y="24" textAnchor="middle" style={{ fill: "var(--ngc-slate)", fontFamily: "var(--sans)", fontSize: 12, fontWeight: 700 }}>Build time (once)</text>
  <text x="530" y="24" textAnchor="middle" style={{ fill: "var(--ngc-slate)", fontFamily: "var(--sans)", fontSize: 12, fontWeight: 700 }}>Every visit</text>
  <line x1="400" y1="14" x2="400" y2="200" style={{ stroke: "var(--line)", strokeDasharray: "4 5" }} />
  <rect x="20" y="70" width="110" height="56" rx="10" style={{ fill: "var(--ngc-blue-soft)", stroke: "var(--ngc-blue)" }} />
  <text x="75" y="102" textAnchor="middle" style={{ fill: "var(--ngc-blue)", fontFamily: "var(--sans)", fontSize: 13, fontWeight: 700 }}>API</text>
  <rect x="210" y="70" width="150" height="56" rx="10" style={{ fill: "var(--ngc-orange)" }} />
  <text x="285" y="96" textAnchor="middle" style={{ fill: "#fff", fontFamily: "var(--sans)", fontSize: 13, fontWeight: 700 }}>index.astro</text>
  <text x="285" y="114" textAnchor="middle" style={{ fill: "#fff", fontFamily: "var(--mono)", fontSize: 11 }}>await fetch()</text>
  <path d="M210 88 H134" style={{ stroke: "var(--ngc-slate)", strokeWidth: 2, fill: "none" }} markerEnd="url(#ft-arrow)" />
  <path d="M130 108 H206" style={{ stroke: "var(--ngc-slate)", strokeWidth: 2, fill: "none" }} markerEnd="url(#ft-arrow)" />
  <circle r="5" cy="88" style={{ fill: "var(--ngc-orange)" }}><animate attributeName="cx" values="210;134;134" keyTimes="0;0.4;1" dur="3s" repeatCount="indefinite" /></circle>
  <circle r="5" cy="108" style={{ fill: "var(--ngc-blue)" }}><animate attributeName="cx" values="130;130;206" keyTimes="0;0.5;0.9" dur="3s" repeatCount="indefinite" /></circle>
  <rect x="440" y="80" width="80" height="36" rx="6" style={{ fill: "var(--ngc-green-soft)", stroke: "var(--ngc-green)" }} />
  <text x="480" y="103" textAnchor="middle" style={{ fill: "var(--ngc-green)", fontFamily: "var(--mono)", fontSize: 12, fontWeight: 700 }}>HTML</text>
  <path d="M360 98 H436" style={{ stroke: "var(--ngc-slate)", strokeWidth: 2, fill: "none" }} markerEnd="url(#ft-arrow)" />
  <g style={{ fontFamily: "var(--sans)", fontSize: 12, fill: "var(--ink2)" }}>
    <text x="580" y="60">👩 Reader</text><text x="580" y="104">👨 Reader</text><text x="580" y="148">🧑 Reader</text>
  </g>
  <path d="M520 92 L572 58" style={{ stroke: "var(--ngc-green)", strokeWidth: 1.5, fill: "none" }} />
  <path d="M520 98 L572 100" style={{ stroke: "var(--ngc-green)", strokeWidth: 1.5, fill: "none" }} />
  <path d="M520 104 L572 144" style={{ stroke: "var(--ngc-green)", strokeWidth: 1.5, fill: "none" }} />
  <text x="320" y="190" textAnchor="middle" style={{ fill: "var(--ngc-slate)", fontFamily: "var(--sans)", fontSize: 12 }}>One API call per build. Zero API calls per reader.</text>
</svg>
<figcaption>The data is fetched once. Readers only download HTML.</figcaption>
</figure>

### Handling Fetch Errors

- An API can be down, slow, or return "404 Not Found".
- `fetch()` does **not** throw for a 404 or 500 status. Check `res.ok` yourself.
- `res.ok` is `true` for status 200 to 299.
- Wrap the call in `try`/`catch` for network errors, and show a friendly fallback.
- Decide on purpose: should a broken API **stop** the build, or should the page show a fallback?

Press **▶ Run**. The post with ID 99 does not exist. The page shows a fallback instead of breaking.

<p>{found ? found.title : "This post is not available."}</p>

<h2>Post 99</h2>
<p>{missing ? missing.title : "This post is not available."}</p>
`,
  }}
/>

**Simple lesson:** check `res.ok`, catch errors, and choose a fallback on purpose.

## Headless CMS and Custom Loaders

### Headless CMS Overview

- **CMS** (content management system) = an app where people write and edit content in a web form.
- **Headless CMS** = a CMS with no website of its own. It only offers the content through an API.
- Astro builds the website. The CMS holds the content. Examples: Contentful, Sanity, Storyblok, Strapi, WordPress (as an API).
- Many CMS tools have an official Astro guide or loader.

### The Content Loader API

- A collection's `loader` can be your own object, not just `glob()` or `file()`.
- **Loader object** = `{ name, load }`. `load()` gets a `store` and adds entries with `store.set({ id, data })`.
- Your entries then get the same schema check, `getCollection()` and types as file entries.

### Loading Remote Data into a Collection

Press **▶ Run**. The `posts` collection is loaded from the API by a custom loader, then checked by a schema.

<ol>
    {posts.map((post) => <li>{post.data.title} · ♥ {post.data.likes}</li>)}
</ol>
`,
  }}
/>

- On your computer, Astro also caches loader results between builds. A real loader can save time by skipping unchanged items.
- This is an advanced topic. For most sites, `glob()`, `file()` or a ready-made CMS loader is enough.

**Simple lesson:** a custom loader puts API data into a collection, so it gets the same checks and tools as files.

## Databases and Backend Services

### Querying a Database in Server Code

- A **database** stores data in tables you can search and update, such as users or orders.
- In Astro, you query a database from server code: the component script, endpoints or actions.
- For a static site, a query runs once, at build time. Pages that need fresh data on every visit are built on request. You will learn that in **On-Demand and Mixed Rendering**.

### Using node:sqlite or Drizzle ORM

- `node:sqlite` = a small database built into Node.js. Good for learning and small sites.
- **ORM** (object-relational mapper) = a library that lets you query a database with JavaScript instead of SQL text.
- **Drizzle ORM** = a popular, typed ORM. It works with SQLite, PostgreSQL and MySQL.

### Why Astro DB Is No Longer Available

- Older guides mention **Astro DB**, an Astro database add-on.
- It was removed in Astro 7. Use Drizzle, `node:sqlite` or a database service instead.

### Backend Services Overview

- **Backend service** = a hosted product that gives you a database, logins and file storage through an API.
- Examples: **Supabase** (PostgreSQL), **Firebase** (Google), **Turso** (SQLite in the cloud).
- You call them from server code, with a secret key kept in an environment variable.

**Simple lesson:** query databases from server code. Astro DB is gone, so use Drizzle, SQLite or a backend service.

## Environment Variables

### .env Files

- **Environment variable** = a named setting that lives outside your code, such as an API address or a secret key.
- Put them in a file called `.env` at the top of the project: `API_KEY=abc123`.
- Add `.env` to `.gitignore`, so secrets never go into Git. A new Astro project already does this.

### import.meta.env

- Read a variable in server code with `import.meta.env.API_KEY`.
- In the component script, every variable from `.env` is available.

### Public Variables (PUBLIC_ Prefix)

- Browser scripts can **only** read variables whose names start with `PUBLIC_`.
- Astro writes their values into the browser JavaScript at build time.
- Every other variable becomes `undefined` in the browser. This keeps secrets safe.

### Built-In Variables

Astro always sets these:

- `import.meta.env.MODE` = `"development"` or `"production"`.
- `import.meta.env.DEV` and `import.meta.env.PROD` = `true` or `false`.
- `import.meta.env.SITE` = the `site` from your config.
- `import.meta.env.BASE_URL` = the `base` from your config, or `/`.

Press **▶ Run**. The component script can read both variables. The browser script only gets the `PUBLIC_` one.

<p>The build can use the API key: {hasKey ? "yes" : "no"}</p>
<p id="browser">…</p>

<script>
    const out = document.getElementById("browser")!;
    out.textContent =
      "In the browser: name = " + import.meta.env.PUBLIC_SITE_NAME +
      ", key = " + import.meta.env.API_KEY;
</script>
`,
  }}
/>

- In the browser, `API_KEY` is `undefined`. Open **HTML sent to browser**: the secret is not in it.

### [TS] Typing Environment Variables in env.d.ts

- Describe your variables in `src/env.d.ts`, so the editor knows their names:

```ts
interface ImportMetaEnv {
  readonly PUBLIC_SITE_NAME: string;
  readonly API_KEY: string;
}

interface ImportMeta {
  readonly env: ImportMetaEnv;
}
```

- A better way, with checks too, is `astro:env` below.

Never write a secret into a `PUBLIC_` variable, a `<script>` or a template. Anything that reaches the browser can be read by anyone.

**Simple lesson:** secrets in `.env` without `PUBLIC_`. Only `PUBLIC_` variables reach the browser.

## Type-Safe Environment Variables

### astro:env and the env.schema Option

- `import.meta.env` does not check anything. A missing key is just `undefined`.
- `astro:env` lets you **describe** every variable in the config. Astro checks them before it builds.
- Describe them in `env.schema`, using `envField` from `astro/config`.

### envField

- `envField.string({...})`, `envField.number({...})`, `envField.boolean({...})` and `envField.enum({...})` describe one variable.
- Options: `context`, `access`, `optional` and `default`.

### Client and Server Variables

- `context: "client"` = may be used in browser code. Must be `access: "public"`.
- `context: "server"` = only in server code.

### Public and Secret Variables

- `access: "public"` = the value may be written into the built files.
- `access: "secret"` = the value is never written into the built files.

### Using getSecret()

- Import server variables from `astro:env/server`, and client variables from `astro:env/client`.
- `getSecret("NAME")` reads a secret by name, even one not in the schema.

Press **▶ Run**. Then delete the `API_URL` line from `.env` and run again. The build stops with a clear message.

<p>Posts per page: {POSTS_PER_PAGE} (a number, from the default)</p>
<p>The token has {tokenLength} characters. It is never shown.</p>
`,
  }}
/>

Use `astro:env` in real projects. A missing or wrong variable stops the build on your computer, instead of breaking the live site.

**Simple lesson:** describe variables with `envField`. Astro checks them and keeps secrets out of the browser.

## TL;DR

1. **fetch() in the script** = runs at build time. Use top-level `await`.
2. **Errors** = check `res.ok`, use `try`/`catch`, show a fallback on purpose.
3. **Headless CMS** = content through an API. Astro builds the site.
4. **Custom loader** = `{ name, load({ store }) }` with `store.set({ id, data })`.
5. **Databases** = from server code. Drizzle, `node:sqlite` or a backend service. Astro DB was removed.
6. **.env** = variables outside the code. Keep it out of Git.
7. **import.meta.env.NAME** = read a variable. Only `PUBLIC_` names reach the browser.
8. **Built-ins** = `MODE`, `DEV`, `PROD`, `SITE`, `BASE_URL`.
9. **astro:env** = `env.schema` with `envField`. Import from `astro:env/server` or `astro:env/client`.

## Common mistakes

<div className="tscroll">

| Mistake | Wrong idea | Correct idea | What goes wrong |
|---|---|---|---|
| Not checking `res.ok` | `fetch()` throws on a 404 | Check `res.ok` | The page shows an error object as data |
| Expecting fresh data on a static page | Each visit fetches again | Static pages fetch once per build | Readers see old data until the next build |
| A secret in a `PUBLIC_` variable | The prefix is just a name | `PUBLIC_` values go into browser files | Anyone can read the secret |
| Reading `API_KEY` in a `<script>` | All variables reach the browser | Only `PUBLIC_` ones do | The value is `undefined` |
| Committing `.env` to Git | Git is private | Keep `.env` in `.gitignore` | Secrets leak with the code |
| Following an Astro DB guide | Astro DB still exists | Use Drizzle, SQLite or a service | Install and build errors |

</div>

## Check Yourself

## Exercises

## What to Remember

- Fetch data in the component script with `await`. Static pages fetch once per build.
- Check `res.ok`, catch errors, and show a fallback.
- Custom loaders put API data into collections.
- Keep secrets in `.env`. Only `PUBLIC_` variables reach the browser.
- `astro:env` checks every variable before the build.

**Checkpoint:** Asha's site is now a complete static blog. It has:

- Markdown posts in a checked collection,
- a page per post, pagination and tag pages,
- optimised images and a custom font,
- data from an API.

Next: **How Astro Renders Pages**, where you learn exactly when and where your pages are built.
