Domain skill
fred
Markdown synced from browser-harness domain skills.
- Host
- fred
- Files
- 1
Agent prompt
Use this skill
Copy this prompt into your coding agent to make it enable browser-harness domain skills and read this exact domain folder before automating.
Set up https://github.com/browser-use/browser-harness for me if it is not already installed. If setup is needed, read `install.md` first to install and connect it to my real browser. Then read `SKILL.md` for normal usage and always read `helpers.py` because that is where the browser-harness functions are. Enable domain skills if they are not already enabled by setting `BH_DOMAIN_SKILLS=1` for browser-harness. Use the `fred` domain skill from `agent-workspace/domain-skills/fred/`. Read every markdown file for this domain before inventing an approach: - agent-workspace/domain-skills/fred/scraping.md Use those domain-skill notes to complete my task for `fred` in my real browser. When you open a setup, verification, or task tab, activate it so I can see the active browser tab.
Skill contents
What the agent will read
Federal Reserve Economic Data
scraping.md
- https://fred.stlouisfed.org / https://api.stlouisfed.org — the canonical source for US macroeconomic time series (800,000+ series). The REST API at api.stlouisfed.org requires a free registered key. The web endpoints...
- Decision tree: pick one approach.
- The web CSV/JSON/TXT URLs all timeout — do NOT attempt them:
- Go to https://fred.stlouisfed.org/docs/api/apikey.html
Show full markdown
https://fred.stlouisfed.org / https://api.stlouisfed.org — the canonical source for US macroeconomic time series (800,000+ series). The REST API at api.stlouisfed.org requires a free registered key. The web endpoints at fred.stlouisfed.org (CSV, JSON, HTML) are all blocked to headless HTTP — they consistently timeout with no response. For zero-key access use the BLS API (unemployment, CPI, payrolls) or World Bank API (GDP, growth rates, annual data).
Do this first
Decision tree: pick one approach.
Need GDP, CPI, UNRATE, payrolls only? → use BLS + World Bank (no key, free forever) Need FEDFUNDS, DGS10, SP500, any FRED series? → get a free FRED API key (5 min) Need browser-visible chart data? → use CDP to intercept network requests
The web CSV/JSON/TXT URLs all timeout — do NOT attempt them:
# ALL OF THESE TIMEOUT — confirmed dead from headless HTTP:
# https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDP ← timeout
# https://fred.stlouisfed.org/graph/fredgraph.json?id=GDP ← timeout
# https://fred.stlouisfed.org/data/GDP.txt ← timeout
# https://fred.stlouisfed.org/series/GDP ← timeout
Getting a free FRED API key
- Go to
https://fred.stlouisfed.org/docs/api/api_key.html - Click "Request or view your API Keys"
- Sign in / register (free St. Louis Fed account)
- Key appears immediately — it's a 32-character lowercase alphanumeric string
The key is free, instant, and unlimited for reasonable use (120 req/min cap).
Option A: FRED REST API (requires free key, 800K+ series)
The only way to get FRED data programmatically. Set FRED_KEY in your .env file.
import json, os
FRED_KEY = os.environ["FRED_KEY"] # 32-char lowercase alphanumeric
BASE = "https://api.stlouisfed.org/fred"
Series metadata
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"
meta = json.loads(http_get(f"{BASE}/series?series_id=GDP&api_key={FRED_KEY}&file_type=json"))
s = meta['seriess'][0]
print(s['title']) # "Gross Domestic Product"
print(s['observation_start']) # "1947-01-01"
print(s['observation_end']) # "2025-10-01"
print(s['frequency']) # "Quarterly"
print(s['frequency_short']) # "Q"
print(s['units']) # "Billions of Dollars"
print(s['units_short']) # "Bil. of $"
print(s['seasonal_adjustment']) # "Seasonally Adjusted Annual Rate"
print(s['popularity']) # 81 (0-100)
print(s['last_updated']) # "2025-12-19 08:00:06-06"
Observations (the actual data)
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"
# Latest 10 values, most recent first
obs = json.loads(http_get(
f"{BASE}/series/observations"
f"?series_id=GDP"
f"&api_key={FRED_KEY}"
f"&file_type=json"
f"&limit=10"
f"&sort_order=desc" # "desc" = newest first, "asc" = oldest first (default)
))
print(obs['count']) # 314 (total observations)
print(obs['observation_start']) # "1947-01-01" (what's in the full series)
for o in obs['observations']:
date = o['date'] # "2025-10-01"
value = o['value'] # "29726.4" — always a STRING, may be "." for missing
if value != '.':
print(f"{date}: ${float(value):,.1f}B")
# 2025-10-01: $29,726.4B
# 2025-07-01: $29,339.1B
# 2025-04-01: $29,119.3B
Date-range filtering
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"
obs = json.loads(http_get(
f"{BASE}/series/observations"
f"?series_id=UNRATE"
f"&api_key={FRED_KEY}"
f"&file_type=json"
f"&observation_start=2020-01-01"
f"&observation_end=2024-12-31"
f"&sort_order=desc"
))
for o in obs['observations'][:5]:
print(f"{o['date']}: {o['value']}%")
# 2024-12-01: 4.1%
# 2024-11-01: 4.2%
# 2024-10-01: 4.1%
Key series IDs
| FRED ID | Description | Frequency | Unit |
|---|---|---|---|
GDP | Gross Domestic Product | Quarterly | Billions of $, SAAR |
GDPC1 | Real GDP (chained 2017 $) | Quarterly | Billions of chained 2017 $ |
UNRATE | Unemployment Rate | Monthly | Percent, SA |
CPIAUCSL | CPI: All Urban Consumers, SA | Monthly | Index 1982-84=100 |
CPIAUCNS | CPI: All Urban Consumers, not SA | Monthly | Index 1982-84=100 |
FEDFUNDS | Federal Funds Effective Rate | Monthly | Percent |
DFF | Federal Funds Rate (daily) | Daily | Percent |
DGS10 | 10-Year Treasury Constant Maturity | Daily | Percent |
DGS2 | 2-Year Treasury | Daily | Percent |
SP500 | S&P 500 | Daily | Index |
NASDAQCOM | NASDAQ Composite | Daily | Index |
PAYEMS | Total Nonfarm Payrolls | Monthly | Thousands of persons, SA |
PCEPI | PCE Price Index | Monthly | Index 2017=100, SA |
PCEPILFE | Core PCE Price Index | Monthly | Index 2017=100, SA |
DCOILBRENTEU | Brent Crude Oil | Daily | $ per Barrel |
DEXUSEU | USD/EUR Exchange Rate | Daily | USD per EUR |
M2SL | M2 Money Stock | Monthly | Billions of $, SA |
MORTGAGE30US | 30-Year Fixed Mortgage Rate | Weekly | Percent |
Series search
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"
results = json.loads(http_get(
f"{BASE}/series/search"
f"?search_text=unemployment+rate"
f"&api_key={FRED_KEY}"
f"&file_type=json"
f"&limit=5"
f"&order_by=popularity" # "popularity" | "search_rank" | "series_id" | "title" | "units" | "frequency" | "seasonal_adjustment" | "realtime_start" | "realtime_end" | "last_updated" | "observation_start" | "observation_end"
f"&sort_order=desc" # most popular first
))
for s in results['seriess']:
print(f"{s['id']}: {s['title']} ({s['frequency_short']}, {s['units_short']})")
# UNRATE: Unemployment Rate (M, %)
# UNEMPLOY: Unemployment Level (M, Thous. of Persons)
Multiple series — parallel fetch
import json, os
from concurrent.futures import ThreadPoolExecutor
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"
def fetch_latest(series_id):
obs = json.loads(http_get(
f"{BASE}/series/observations?series_id={series_id}"
f"&api_key={FRED_KEY}&file_type=json&limit=1&sort_order=desc"
))
o = obs['observations'][0]
return series_id, o['date'], o['value']
series_ids = ["GDP", "UNRATE", "CPIAUCSL", "FEDFUNDS", "DGS10", "SP500"]
with ThreadPoolExecutor(max_workers=6) as ex:
results = list(ex.map(fetch_latest, series_ids))
for sid, date, val in results:
print(f"{sid:15} {date}: {val}")
# GDP 2025-10-01: 29726.4
# UNRATE 2026-03-01: 4.3
# CPIAUCSL 2026-02-01: 321.457
# FEDFUNDS 2026-03-01: 4.33
# DGS10 2026-04-17: 4.34
# SP500 2026-04-17: 5282.70
# Confirmed: 6 parallel requests complete in ~0.4s
Parse observations into a list of (date, float) tuples
import json, os
FRED_KEY = os.environ["FRED_KEY"]
BASE = "https://api.stlouisfed.org/fred"
obs = json.loads(http_get(
f"{BASE}/series/observations?series_id=DGS10&api_key={FRED_KEY}&file_type=json"
f"&observation_start=2024-01-01&sort_order=asc"
))
data = [
(o['date'], float(o['value']))
for o in obs['observations']
if o['value'] != '.' # '.' = missing value, skip it
]
print(f"{len(data)} observations")
print(f"First: {data[0]}") # ('2024-01-02', 3.91)
print(f"Last: {data[-1]}") # ('2026-04-17', 4.34)
Handle errors
import urllib.error, json
try:
r = http_get(f"https://api.stlouisfed.org/fred/series?series_id=BADID&api_key={FRED_KEY}&file_type=json")
print(json.loads(r))
except urllib.error.HTTPError as e:
err = json.loads(e.read().decode())
# err['error_code'] → 400
# err['error_message'] → "Bad Request. The series does not exist."
print(f"FRED error {err['error_code']}: {err['error_message']}")
Option B: BLS API (no key required, confirmed live)
Bureau of Labor Statistics. Covers unemployment, CPI, payrolls — the most-queried FRED series. Without a key: 10 requests/day limit. Free key registration at https://www.bls.gov/developers/ gives 500 req/day and 10 years of data per call (vs 3 years without key).
import json
# Single series GET — no auth needed
r = http_get("https://api.bls.gov/publicAPI/v2/timeseries/data/LNS14000000?startyear=2024&endyear=2024")
data = json.loads(r)
# data['status'] == 'REQUEST_SUCCEEDED'
series = data['Results']['series'][0]
for point in series['data'][:3]:
print(f"{point['year']}-{point['period']} ({point['periodName']}): {point['value']}")
# 2024-M12 (December): 4.1
# 2024-M11 (November): 4.2
# 2024-M10 (October): 4.1
Multi-series POST (single call, multiple series)
import json, urllib.request
payload = json.dumps({
"seriesid": ["LNS14000000", "CUSR0000SA0", "CES0000000001"],
"startyear": "2023",
"endyear": "2024"
# "registrationkey": "YOUR_BLS_KEY" # optional: lifts to 500/day, 10yr range
}).encode()
req = urllib.request.Request(
"https://api.bls.gov/publicAPI/v2/timeseries/data/",
data=payload,
headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req, timeout=20) as resp:
data = json.loads(resp.read().decode())
for s in data['Results']['series']:
pts = s['data']
print(f"{s['seriesID']}: {len(pts)} points, latest={pts[0]['value']}")
# LNS14000000: 24 points, latest=4.1 (unemployment %)
# CUSR0000SA0: 24 points, latest=317.604 (CPI index)
# CES0000000001: 24 points, latest=158316 (nonfarm payrolls, thousands)
Key BLS series (FRED equivalents)
| BLS Series ID | FRED Equivalent | Description |
|---|---|---|
LNS14000000 | UNRATE | Unemployment rate, SA (%) |
CUSR0000SA0 | CPIAUCSL | CPI-U All Urban, SA |
CUUR0000SA0 | CPIAUCNS | CPI-U All Urban, not SA |
CUSR0000SA0L1E | CPILFESL | CPI less food and energy, SA |
CES0000000001 | PAYEMS | Total nonfarm payrolls (thousands) |
LNS11000000 | CLF16OV | Civilian labor force (thousands) |
LNS12000000 | CE16OV | Civilian employment (thousands) |
BLS rate limits
| Without key | With free key | |
|---|---|---|
| Requests/day | 10 (confirmed: call 11 returns REQUEST_NOT_PROCESSED) | 500 |
| Series per request | 25 | 50 |
| Years per request | 3 | 10 |
| Daily or seasonal adjustment | No | Yes |
Option C: World Bank API (no key, unlimited, annual data)
Free, no registration, no rate limit observed (10 rapid calls completed in 2.0s). Annual data only — no monthly or quarterly frequency.
import json
# Single country, single indicator
r = http_get("https://api.worldbank.org/v2/country/US/indicator/NY.GDP.MKTP.CD?format=json&per_page=5&mrv=5")
data = json.loads(r)
page_info = data[0] # {'page': 1, 'pages': 1, 'per_page': 5, 'total': 5, 'lastupdated': '2026-04-08'}
items = data[1] # list of observations
for item in items:
if item['value']:
print(f"{item['date']}: ${item['value']/1e12:.2f}T")
# 2024: $28.75T
# 2023: $27.29T
# 2022: $25.60T
Date range filter and multi-country
import json
# Historical range: date=YYYY:YYYY
r = http_get("https://api.worldbank.org/v2/country/US/indicator/FP.CPI.TOTL.ZG?format=json&date=2015:2024&per_page=15")
data = json.loads(r)
items = [i for i in data[1] if i['value'] is not None]
for item in items:
print(f"{item['date']}: {item['value']:.2f}%")
# 2024: 2.95%
# 2023: 4.12%
# 2022: 8.00%
# ...
# Multi-country: semicolon-separated ISO codes
r = http_get("https://api.worldbank.org/v2/country/US;CN;DE;JP;GB/indicator/NY.GDP.MKTP.CD?format=json&date=2023&per_page=10")
data = json.loads(r)
items = sorted([i for i in data[1] if i['value']], key=lambda x: x['value'], reverse=True)
for item in items:
print(f"{item['country']['value']}: ${item['value']/1e12:.2f}T")
# United States: $27.29T
# China: $18.27T
# Germany: $4.56T
Key World Bank indicators (FRED equivalents)
| WB Indicator Code | FRED Equivalent | Description |
|---|---|---|
NY.GDP.MKTP.CD | GDP | GDP, current USD |
NY.GDP.MKTP.KD.ZG | A191RL1Q225SBEA | GDP growth rate (%) |
NY.GDP.PCAP.CD | A939RX0Q048SBEA | GDP per capita (USD) |
FP.CPI.TOTL.ZG | FPCPITOTLZGUSA | CPI inflation, annual % |
FP.CPI.TOTL | CPIAUCSL (annual) | CPI level, 2010=100 |
SL.UEM.TOTL.ZS | UNRATE (annual) | Unemployment rate, ILO model |
CM.MKT.LCAP.GD.ZS | — | Stock market cap / GDP ratio |
Option D: Alpha Vantage (free registered key, select indicators)
Some economic indicators work with the demo key (no registration); most require a free registered key (25 requests/day, instant signup at https://www.alphavantage.co/support/#api-key).
import json
AV_KEY = "demo" # or your registered key
# Unemployment rate (works with demo key — confirmed)
r = http_get(f"https://www.alphavantage.co/query?function=UNEMPLOYMENT&apikey={AV_KEY}")
data = json.loads(r)
# data['name'] = 'Unemployment Rate'
# data['interval'] = 'monthly'
# data['unit'] = 'percent'
# data['data'] → list of {date, value}, newest first
print(data['data'][0]) # {'date': '2026-03-01', 'value': '4.3'}
print(f"Total: {len(data['data'])} months since {data['data'][-1]['date']}")
# Total: 939 months since 1948-01-01
Which indicators work with demo vs registered key
| Function | demo key | Registered key |
|---|---|---|
UNEMPLOYMENT | YES | YES |
INFLATION | YES (annual) | YES |
RETAIL_SALES | YES | YES |
DURABLES | YES | YES |
NONFARM_PAYROLL | YES | YES |
REAL_GDP_PER_CAPITA | YES | YES |
REAL_GDP | NO (rate-limited) | YES |
CPI | NO (rate-limited) | YES |
FEDERAL_FUNDS_RATE | NO (rate-limited) | YES |
TREASURY_YIELD | NO (rate-limited) | YES |
CONSUMER_SENTIMENT | NO (rate-limited) | YES |
import json
AV_KEY = "YOUR_FREE_KEY" # from alphavantage.co/support/#api-key
# Federal Funds Rate — monthly (requires registered key)
r = http_get(f"https://www.alphavantage.co/query?function=FEDERAL_FUNDS_RATE&interval=monthly&apikey={AV_KEY}")
data = json.loads(r)
for item in data['data'][:3]:
print(f"{item['date']}: {item['value']}%")
# 2026-03-01: 4.33%
# 2026-02-01: 4.33%
# 2026-01-01: 4.33%
# 10-Year Treasury Yield
r = http_get(f"https://www.alphavantage.co/query?function=TREASURY_YIELD&maturity=10year&interval=monthly&apikey={AV_KEY}")
data = json.loads(r)
print(data['data'][0]) # {'date': '2026-04-17', 'value': '4.34'}
Option E: Browser + CDP (for interactive FRED charts)
When you need data from fred.stlouisfed.org that has no API equivalent (custom chart combos, release dates visible on page) — or when you have no API key — use the browser.
# Navigate to a series page
goto_url("https://fred.stlouisfed.org/series/GDP")
wait_for_load()
# Option 1: Intercept the fredgraph XHR that the chart fires
# The page's chart JS calls fredgraph.csv internally — intercept it
events = drain_events()
# Look for network events with fredgraph.csv in URL
# Option 2: Extract the latest value from the page text
latest_val = js("""
// The last observation appears in the meta section
const el = document.querySelector('.series-meta-observation-end');
el ? el.textContent.trim() : null
""")
# Option 3: Read the data table if present
table_data = js("""
const rows = Array.from(document.querySelectorAll('table.series-observations tr'));
rows.map(r => {
const cells = r.querySelectorAll('td');
return cells.length >= 2 ? [cells[0].textContent.trim(), cells[1].textContent.trim()] : null;
}).filter(Boolean);
""")
Rate limits
| API | Limit | Notes |
|---|---|---|
| FRED REST API | 120 req/min | With registered key (free) |
| FRED REST API | blocked | Without key — HTTP 400 |
| BLS (no key) | 10 req/day | Confirmed: call 11 → REQUEST_NOT_PROCESSED |
| BLS (with key) | 500 req/day, 50 series/req | Free registration at bls.gov/developers |
| World Bank | No limit observed | 10 rapid calls: 2.0s, no 429 |
| Alpha Vantage (demo) | 2 req/sec | Demo key rate-limited for most functions |
| Alpha Vantage (free key) | 25 req/day | Free at alphavantage.co/support/#api-key |
Gotchas
-
fred.stlouisfed.org web endpoints ALL timeout — The CSV download (
fredgraph.csv), JSON graph (fredgraph.json), text format (/data/*.txt), and HTML series pages all hang indefinitely from headless HTTP. This is not a UA or header issue — the server simply does not respond to non-browser connections. Confirmed with multiple UA strings, TCP connect succeeds but no HTTP response is sent. -
FRED API key is mandatory and must be exactly 32 lowercase alphanumeric chars — "test", "demo", "guest", and keys shorter/longer than 32 chars all return HTTP 400:
"not a 32 character alpha-numeric lower-case string". An unregistered 32-char key returns:"not registered". -
Observation values are always strings, not numbers — The
valuefield in FRED observations is always a JSON string:"4.1", not4.1. Also"."(dot) means missing/not-yet-released. Always checkif o['value'] != '.'beforefloat(o['value']). -
BLS 10 req/day without key burns fast — The limit is per-IP per-day. 10 calls is exhausted in one moderate script run. Either register a free BLS key immediately or use World Bank for the same data annually.
-
BLS data range: 3 years without key, 10 years with key — Requesting
startyear=2000&endyear=2024without a key silently truncates to the most recent 3 years. With a key it returns up to 10 years and includes amessagefield if the range was truncated:['Year range has been reduced to the system-allowed limit of 10 years.']. -
World Bank is annual only — No monthly or quarterly data. For monthly UNRATE or CPI, use BLS. For quarterly GDP, use FRED API or Alpha Vantage
REAL_GDP. -
World Bank response is a 2-element array —
data[0]is pagination metadata,data[1]is the observations list. Missing years havevalue: null(not"."). Filter withif item['value'] is not None. -
Alpha Vantage demo key: 2 req/sec, covers only 6 economic functions — The other 6 economic functions (
REAL_GDP,CPI,TREASURY_YIELD, etc.) return{"Information": "The demo API key is for demo purposes only..."}. No error code — just check for theInformationkey in the response. -
FRED
sort_order=descreturns newest first — Default isasc(oldest first, starting from observation_start). For "get the latest value" uselimit=1&sort_order=desc. -
FRED series IDs are case-sensitive and exact —
gdpreturns an error; must beGDP. Checkfred.stlouisfed.org/series/{ID}to verify a series exists before scripting. -
Some FRED series have gaps — Daily series like
DGS10andSP500skip weekends and holidays. Those dates simply don't appear in the observations array (not represented as"."). Weekly and monthly series use the first day of the period as the date (e.g.,2024-01-01= January 2024). -
FRED
realtime_start/realtime_endin observations — Every observation has these fields reflecting vintage data. For current data, ignore them. They matter only for "real-time" research (what was the published value on a specific past date).