← Back to skills

Domain skill

fred

Markdown synced from browser-harness domain skills.

Agent prompt

Use this skill

Copy this prompt into your coding agent to make it enable browser-harness domain skills and read this exact domain folder before automating.

Set up https://github.com/browser-use/browser-harness for me if it is not already installed. If setup is needed, read `install.md` first to install and connect it to my real browser. Then read `SKILL.md` for normal usage and always read `helpers.py` because that is where the browser-harness functions are.

Enable domain skills if they are not already enabled by setting `BH_DOMAIN_SKILLS=1` for browser-harness. Use the `fred` domain skill from `agent-workspace/domain-skills/fred/`. Read every markdown file for this domain before inventing an approach:
- agent-workspace/domain-skills/fred/scraping.md

Use those domain-skill notes to complete my task for `fred` in my real browser. When you open a setup, verification, or task tab, activate it so I can see the active browser tab.

Skill contents

What the agent will read

Federal Reserve Economic Data

scraping.md

Source
  • https://fred.stlouisfed.org / https://api.stlouisfed.org — the canonical source for US macroeconomic time series (800,000+ series). The REST API at api.stlouisfed.org requires a free registered key. The web endpoints...
  • Decision tree: pick one approach.
  • The web CSV/JSON/TXT URLs all timeout — do NOT attempt them:
  • Go to https://fred.stlouisfed.org/docs/api/apikey.html
Show full markdown

https://fred.stlouisfed.org / https://api.stlouisfed.org — the canonical source for US macroeconomic time series (800,000+ series). The REST API at api.stlouisfed.org requires a free registered key. The web endpoints at fred.stlouisfed.org (CSV, JSON, HTML) are all blocked to headless HTTP — they consistently timeout with no response. For zero-key access use the BLS API (unemployment, CPI, payrolls) or World Bank API (GDP, growth rates, annual data).

Do this first

Decision tree: pick one approach.

code
Need GDP, CPI, UNRATE, payrolls only? → use BLS + World Bank (no key, free forever)
Need FEDFUNDS, DGS10, SP500, any FRED series? → get a free FRED API key (5 min)
Need browser-visible chart data? → use CDP to intercept network requests

The web CSV/JSON/TXT URLs all timeout — do NOT attempt them:

python
# ALL OF THESE TIMEOUT — confirmed dead from headless HTTP:
# https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDP      ← timeout
# https://fred.stlouisfed.org/graph/fredgraph.json?id=GDP     ← timeout
# https://fred.stlouisfed.org/data/GDP.txt                    ← timeout
# https://fred.stlouisfed.org/series/GDP                      ← timeout

Getting a free FRED API key

  1. Go to https://fred.stlouisfed.org/docs/api/api_key.html
  2. Click "Request or view your API Keys"
  3. Sign in / register (free St. Louis Fed account)
  4. Key appears immediately — it's a 32-character lowercase alphanumeric string

The key is free, instant, and unlimited for reasonable use (120 req/min cap).


Option A: FRED REST API (requires free key, 800K+ series)

The only way to get FRED data programmatically. Set FRED_KEY in your .env file.

python
import json, os
FRED_KEY = os.environ["FRED_KEY"]   # 32-char lowercase alphanumeric
BASE = "https://api.stlouisfed.org/fred"

Series metadata

python
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"

meta = json.loads(http_get(f"{BASE}/series?series_id=GDP&api_key={FRED_KEY}&file_type=json"))
s = meta['seriess'][0]
print(s['title'])               # "Gross Domestic Product"
print(s['observation_start'])   # "1947-01-01"
print(s['observation_end'])     # "2025-10-01"
print(s['frequency'])           # "Quarterly"
print(s['frequency_short'])     # "Q"
print(s['units'])               # "Billions of Dollars"
print(s['units_short'])         # "Bil. of $"
print(s['seasonal_adjustment']) # "Seasonally Adjusted Annual Rate"
print(s['popularity'])          # 81  (0-100)
print(s['last_updated'])        # "2025-12-19 08:00:06-06"

Observations (the actual data)

python
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"

# Latest 10 values, most recent first
obs = json.loads(http_get(
    f"{BASE}/series/observations"
    f"?series_id=GDP"
    f"&api_key={FRED_KEY}"
    f"&file_type=json"
    f"&limit=10"
    f"&sort_order=desc"      # "desc" = newest first, "asc" = oldest first (default)
))
print(obs['count'])              # 314  (total observations)
print(obs['observation_start'])  # "1947-01-01"  (what's in the full series)

for o in obs['observations']:
    date  = o['date']    # "2025-10-01"
    value = o['value']   # "29726.4"  — always a STRING, may be "." for missing
    if value != '.':
        print(f"{date}: ${float(value):,.1f}B")
# 2025-10-01: $29,726.4B
# 2025-07-01: $29,339.1B
# 2025-04-01: $29,119.3B

Date-range filtering

python
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"

obs = json.loads(http_get(
    f"{BASE}/series/observations"
    f"?series_id=UNRATE"
    f"&api_key={FRED_KEY}"
    f"&file_type=json"
    f"&observation_start=2020-01-01"
    f"&observation_end=2024-12-31"
    f"&sort_order=desc"
))
for o in obs['observations'][:5]:
    print(f"{o['date']}: {o['value']}%")
# 2024-12-01: 4.1%
# 2024-11-01: 4.2%
# 2024-10-01: 4.1%

Key series IDs

FRED IDDescriptionFrequencyUnit
GDPGross Domestic ProductQuarterlyBillions of $, SAAR
GDPC1Real GDP (chained 2017 $)QuarterlyBillions of chained 2017 $
UNRATEUnemployment RateMonthlyPercent, SA
CPIAUCSLCPI: All Urban Consumers, SAMonthlyIndex 1982-84=100
CPIAUCNSCPI: All Urban Consumers, not SAMonthlyIndex 1982-84=100
FEDFUNDSFederal Funds Effective RateMonthlyPercent
DFFFederal Funds Rate (daily)DailyPercent
DGS1010-Year Treasury Constant MaturityDailyPercent
DGS22-Year TreasuryDailyPercent
SP500S&P 500DailyIndex
NASDAQCOMNASDAQ CompositeDailyIndex
PAYEMSTotal Nonfarm PayrollsMonthlyThousands of persons, SA
PCEPIPCE Price IndexMonthlyIndex 2017=100, SA
PCEPILFECore PCE Price IndexMonthlyIndex 2017=100, SA
DCOILBRENTEUBrent Crude OilDaily$ per Barrel
DEXUSEUUSD/EUR Exchange RateDailyUSD per EUR
M2SLM2 Money StockMonthlyBillions of $, SA
MORTGAGE30US30-Year Fixed Mortgage RateWeeklyPercent

Series search

python
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"

results = json.loads(http_get(
    f"{BASE}/series/search"
    f"?search_text=unemployment+rate"
    f"&api_key={FRED_KEY}"
    f"&file_type=json"
    f"&limit=5"
    f"&order_by=popularity"    # "popularity" | "search_rank" | "series_id" | "title" | "units" | "frequency" | "seasonal_adjustment" | "realtime_start" | "realtime_end" | "last_updated" | "observation_start" | "observation_end"
    f"&sort_order=desc"        # most popular first
))
for s in results['seriess']:
    print(f"{s['id']}: {s['title']} ({s['frequency_short']}, {s['units_short']})")
# UNRATE: Unemployment Rate (M, %)
# UNEMPLOY: Unemployment Level (M, Thous. of Persons)

Multiple series — parallel fetch

python
import json, os
from concurrent.futures import ThreadPoolExecutor
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"

def fetch_latest(series_id):
    obs = json.loads(http_get(
        f"{BASE}/series/observations?series_id={series_id}"
        f"&api_key={FRED_KEY}&file_type=json&limit=1&sort_order=desc"
    ))
    o = obs['observations'][0]
    return series_id, o['date'], o['value']

series_ids = ["GDP", "UNRATE", "CPIAUCSL", "FEDFUNDS", "DGS10", "SP500"]
with ThreadPoolExecutor(max_workers=6) as ex:
    results = list(ex.map(fetch_latest, series_ids))

for sid, date, val in results:
    print(f"{sid:15} {date}: {val}")
# GDP             2025-10-01: 29726.4
# UNRATE          2026-03-01: 4.3
# CPIAUCSL        2026-02-01: 321.457
# FEDFUNDS        2026-03-01: 4.33
# DGS10           2026-04-17: 4.34
# SP500           2026-04-17: 5282.70
# Confirmed: 6 parallel requests complete in ~0.4s

Parse observations into a list of (date, float) tuples

python
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"

obs = json.loads(http_get(
    f"{BASE}/series/observations?series_id=DGS10&api_key={FRED_KEY}&file_type=json"
    f"&observation_start=2024-01-01&sort_order=asc"
))

data = [
    (o['date'], float(o['value']))
    for o in obs['observations']
    if o['value'] != '.'   # '.' = missing value, skip it
]
print(f"{len(data)} observations")
print(f"First: {data[0]}")   # ('2024-01-02', 3.91)
print(f"Last:  {data[-1]}")  # ('2026-04-17', 4.34)

Handle errors

python
import urllib.error, json

try:
    r = http_get(f"https://api.stlouisfed.org/fred/series?series_id=BADID&api_key={FRED_KEY}&file_type=json")
    print(json.loads(r))
except urllib.error.HTTPError as e:
    err = json.loads(e.read().decode())
    # err['error_code']     → 400
    # err['error_message']  → "Bad Request.  The series does not exist."
    print(f"FRED error {err['error_code']}: {err['error_message']}")

Option B: BLS API (no key required, confirmed live)

Bureau of Labor Statistics. Covers unemployment, CPI, payrolls — the most-queried FRED series. Without a key: 10 requests/day limit. Free key registration at https://www.bls.gov/developers/ gives 500 req/day and 10 years of data per call (vs 3 years without key).

python
import json
# Single series GET — no auth needed
r = http_get("https://api.bls.gov/publicAPI/v2/timeseries/data/LNS14000000?startyear=2024&endyear=2024")
data = json.loads(r)
# data['status'] == 'REQUEST_SUCCEEDED'
series = data['Results']['series'][0]
for point in series['data'][:3]:
    print(f"{point['year']}-{point['period']} ({point['periodName']}): {point['value']}")
# 2024-M12 (December): 4.1
# 2024-M11 (November): 4.2
# 2024-M10 (October): 4.1

Multi-series POST (single call, multiple series)

python
import json, urllib.request

payload = json.dumps({
    "seriesid": ["LNS14000000", "CUSR0000SA0", "CES0000000001"],
    "startyear": "2023",
    "endyear": "2024"
    # "registrationkey": "YOUR_BLS_KEY"  # optional: lifts to 500/day, 10yr range
}).encode()

req = urllib.request.Request(
    "https://api.bls.gov/publicAPI/v2/timeseries/data/",
    data=payload,
    headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=20) as resp:
    data = json.loads(resp.read().decode())

for s in data['Results']['series']:
    pts = s['data']
    print(f"{s['seriesID']}: {len(pts)} points, latest={pts[0]['value']}")
# LNS14000000: 24 points, latest=4.1   (unemployment %)
# CUSR0000SA0: 24 points, latest=317.604  (CPI index)
# CES0000000001: 24 points, latest=158316  (nonfarm payrolls, thousands)

Key BLS series (FRED equivalents)

BLS Series IDFRED EquivalentDescription
LNS14000000UNRATEUnemployment rate, SA (%)
CUSR0000SA0CPIAUCSLCPI-U All Urban, SA
CUUR0000SA0CPIAUCNSCPI-U All Urban, not SA
CUSR0000SA0L1ECPILFESLCPI less food and energy, SA
CES0000000001PAYEMSTotal nonfarm payrolls (thousands)
LNS11000000CLF16OVCivilian labor force (thousands)
LNS12000000CE16OVCivilian employment (thousands)

BLS rate limits

Without keyWith free key
Requests/day10 (confirmed: call 11 returns REQUEST_NOT_PROCESSED)500
Series per request2550
Years per request310
Daily or seasonal adjustmentNoYes

Option C: World Bank API (no key, unlimited, annual data)

Free, no registration, no rate limit observed (10 rapid calls completed in 2.0s). Annual data only — no monthly or quarterly frequency.

python
import json

# Single country, single indicator
r = http_get("https://api.worldbank.org/v2/country/US/indicator/NY.GDP.MKTP.CD?format=json&per_page=5&mrv=5")
data = json.loads(r)
page_info = data[0]   # {'page': 1, 'pages': 1, 'per_page': 5, 'total': 5, 'lastupdated': '2026-04-08'}
items     = data[1]   # list of observations

for item in items:
    if item['value']:
        print(f"{item['date']}: ${item['value']/1e12:.2f}T")
# 2024: $28.75T
# 2023: $27.29T
# 2022: $25.60T

Date range filter and multi-country

python
import json

# Historical range: date=YYYY:YYYY
r = http_get("https://api.worldbank.org/v2/country/US/indicator/FP.CPI.TOTL.ZG?format=json&date=2015:2024&per_page=15")
data = json.loads(r)
items = [i for i in data[1] if i['value'] is not None]
for item in items:
    print(f"{item['date']}: {item['value']:.2f}%")
# 2024: 2.95%
# 2023: 4.12%
# 2022: 8.00%
# ...

# Multi-country: semicolon-separated ISO codes
r = http_get("https://api.worldbank.org/v2/country/US;CN;DE;JP;GB/indicator/NY.GDP.MKTP.CD?format=json&date=2023&per_page=10")
data = json.loads(r)
items = sorted([i for i in data[1] if i['value']], key=lambda x: x['value'], reverse=True)
for item in items:
    print(f"{item['country']['value']}: ${item['value']/1e12:.2f}T")
# United States: $27.29T
# China: $18.27T
# Germany: $4.56T

Key World Bank indicators (FRED equivalents)

WB Indicator CodeFRED EquivalentDescription
NY.GDP.MKTP.CDGDPGDP, current USD
NY.GDP.MKTP.KD.ZGA191RL1Q225SBEAGDP growth rate (%)
NY.GDP.PCAP.CDA939RX0Q048SBEAGDP per capita (USD)
FP.CPI.TOTL.ZGFPCPITOTLZGUSACPI inflation, annual %
FP.CPI.TOTLCPIAUCSL (annual)CPI level, 2010=100
SL.UEM.TOTL.ZSUNRATE (annual)Unemployment rate, ILO model
CM.MKT.LCAP.GD.ZSStock market cap / GDP ratio

Option D: Alpha Vantage (free registered key, select indicators)

Some economic indicators work with the demo key (no registration); most require a free registered key (25 requests/day, instant signup at https://www.alphavantage.co/support/#api-key).

python
import json
AV_KEY = "demo"  # or your registered key

# Unemployment rate (works with demo key — confirmed)
r = http_get(f"https://www.alphavantage.co/query?function=UNEMPLOYMENT&apikey={AV_KEY}")
data = json.loads(r)
# data['name']  = 'Unemployment Rate'
# data['interval'] = 'monthly'
# data['unit']     = 'percent'
# data['data']     → list of {date, value}, newest first

print(data['data'][0])   # {'date': '2026-03-01', 'value': '4.3'}
print(f"Total: {len(data['data'])} months since {data['data'][-1]['date']}")
# Total: 939 months since 1948-01-01

Which indicators work with demo vs registered key

Functiondemo keyRegistered key
UNEMPLOYMENTYESYES
INFLATIONYES (annual)YES
RETAIL_SALESYESYES
DURABLESYESYES
NONFARM_PAYROLLYESYES
REAL_GDP_PER_CAPITAYESYES
REAL_GDPNO (rate-limited)YES
CPINO (rate-limited)YES
FEDERAL_FUNDS_RATENO (rate-limited)YES
TREASURY_YIELDNO (rate-limited)YES
CONSUMER_SENTIMENTNO (rate-limited)YES
python
import json
AV_KEY = "YOUR_FREE_KEY"  # from alphavantage.co/support/#api-key

# Federal Funds Rate — monthly (requires registered key)
r = http_get(f"https://www.alphavantage.co/query?function=FEDERAL_FUNDS_RATE&interval=monthly&apikey={AV_KEY}")
data = json.loads(r)
for item in data['data'][:3]:
    print(f"{item['date']}: {item['value']}%")
# 2026-03-01: 4.33%
# 2026-02-01: 4.33%
# 2026-01-01: 4.33%

# 10-Year Treasury Yield
r = http_get(f"https://www.alphavantage.co/query?function=TREASURY_YIELD&maturity=10year&interval=monthly&apikey={AV_KEY}")
data = json.loads(r)
print(data['data'][0])   # {'date': '2026-04-17', 'value': '4.34'}

Option E: Browser + CDP (for interactive FRED charts)

When you need data from fred.stlouisfed.org that has no API equivalent (custom chart combos, release dates visible on page) — or when you have no API key — use the browser.

python
# Navigate to a series page
goto_url("https://fred.stlouisfed.org/series/GDP")
wait_for_load()

# Option 1: Intercept the fredgraph XHR that the chart fires
# The page's chart JS calls fredgraph.csv internally — intercept it
events = drain_events()
# Look for network events with fredgraph.csv in URL

# Option 2: Extract the latest value from the page text
latest_val = js("""
    // The last observation appears in the meta section
    const el = document.querySelector('.series-meta-observation-end');
    el ? el.textContent.trim() : null
""")

# Option 3: Read the data table if present
table_data = js("""
    const rows = Array.from(document.querySelectorAll('table.series-observations tr'));
    rows.map(r => {
        const cells = r.querySelectorAll('td');
        return cells.length >= 2 ? [cells[0].textContent.trim(), cells[1].textContent.trim()] : null;
    }).filter(Boolean);
""")

Rate limits

APILimitNotes
FRED REST API120 req/minWith registered key (free)
FRED REST APIblockedWithout key — HTTP 400
BLS (no key)10 req/dayConfirmed: call 11 → REQUEST_NOT_PROCESSED
BLS (with key)500 req/day, 50 series/reqFree registration at bls.gov/developers
World BankNo limit observed10 rapid calls: 2.0s, no 429
Alpha Vantage (demo)2 req/secDemo key rate-limited for most functions
Alpha Vantage (free key)25 req/dayFree at alphavantage.co/support/#api-key

Gotchas

  • fred.stlouisfed.org web endpoints ALL timeout — The CSV download (fredgraph.csv), JSON graph (fredgraph.json), text format (/data/*.txt), and HTML series pages all hang indefinitely from headless HTTP. This is not a UA or header issue — the server simply does not respond to non-browser connections. Confirmed with multiple UA strings, TCP connect succeeds but no HTTP response is sent.

  • FRED API key is mandatory and must be exactly 32 lowercase alphanumeric chars — "test", "demo", "guest", and keys shorter/longer than 32 chars all return HTTP 400: "not a 32 character alpha-numeric lower-case string". An unregistered 32-char key returns: "not registered".

  • Observation values are always strings, not numbers — The value field in FRED observations is always a JSON string: "4.1", not 4.1. Also "." (dot) means missing/not-yet-released. Always check if o['value'] != '.' before float(o['value']).

  • BLS 10 req/day without key burns fast — The limit is per-IP per-day. 10 calls is exhausted in one moderate script run. Either register a free BLS key immediately or use World Bank for the same data annually.

  • BLS data range: 3 years without key, 10 years with key — Requesting startyear=2000&endyear=2024 without a key silently truncates to the most recent 3 years. With a key it returns up to 10 years and includes a message field if the range was truncated: ['Year range has been reduced to the system-allowed limit of 10 years.'].

  • World Bank is annual only — No monthly or quarterly data. For monthly UNRATE or CPI, use BLS. For quarterly GDP, use FRED API or Alpha Vantage REAL_GDP.

  • World Bank response is a 2-element arraydata[0] is pagination metadata, data[1] is the observations list. Missing years have value: null (not "."). Filter with if item['value'] is not None.

  • Alpha Vantage demo key: 2 req/sec, covers only 6 economic functions — The other 6 economic functions (REAL_GDP, CPI, TREASURY_YIELD, etc.) return {"Information": "The demo API key is for demo purposes only..."}. No error code — just check for the Information key in the response.

  • FRED sort_order=desc returns newest first — Default is asc (oldest first, starting from observation_start). For "get the latest value" use limit=1&sort_order=desc.

  • FRED series IDs are case-sensitive and exactgdp returns an error; must be GDP. Check fred.stlouisfed.org/series/{ID} to verify a series exists before scripting.

  • Some FRED series have gaps — Daily series like DGS10 and SP500 skip weekends and holidays. Those dates simply don't appear in the observations array (not represented as "."). Weekly and monthly series use the first day of the period as the date (e.g., 2024-01-01 = January 2024).

  • FRED realtime_start/realtime_end in observations — Every observation has these fields reflecting vintage data. For current data, ignore them. They matter only for "real-time" research (what was the published value on a specific past date).