Reddit Scraper: Export Posts and Comments in 2026

A Reddit scraper is useful when you need public subreddit posts, Reddit comments, search results, and thread URLs in a spreadsheet instead of scattered across browser tabs. The catch in 2026 is that old advice about PRAW, Pushshift, and free API access is often stale, so teams end up with empty Python results, rate limits, or data they cannot reuse safely.

What still works: scraping Reddit through a real browser for visible public pages, carefully using Reddit's public page data for small research pulls, or using Reddit's Data API under the current terms for approved app use. This guide compares Clura, Python, browser scraping, Reddit API workflows, comment-thread exports, and Pushshift alternatives so you can pick the right level of risk and setup.

What Changed in Reddit Scraping and What's Still Exportable?

Public Reddit posts, comments, subreddit pages, and search result pages are still visible in the browser, but Reddit's current Data API and Developer Terms restrict commercial use, excessive access, model training, and resale without approval. Treat Reddit scraping as a public-page export workflow, not an unrestricted data source.

The old Reddit scraping playbook changed after 2023. PRAW still exists, but commercial and higher-volume uses now need to respect Reddit's current Data API and Developer Terms. Pushshift, the archive many researchers used for historical Reddit data, is no longer the reliable default. And new Reddit pages are JavaScript-heavy enough that simple Python requests often return empty shells instead of useful post rows.

The practical takeaway: public Reddit pages are still accessible to users in a browser, but automated access should be narrow, respectful, and tied to a legitimate research or internal workflow. Reddit's Data API Terms say commercial use, research beyond rate limits, and unapproved reuse may require a separate agreement. That is why this guide focuses on exporting visible public rows for review, not building a Reddit data resale product.

Can You Use Reddit's JSON Endpoint for Scraping?

Some public Reddit URLs still return structured page data when .json is appended, but this is not a stable commercial API contract. Use it only for low-volume research, identify your client, respect rate limits, and switch to Reddit's approved Data API path for apps, products, or commercial workflows.

Many public Reddit pages expose structured page data when you append `.json` to the URL. This is useful for research and debugging because you can inspect post titles, comment counts, authors, timestamps, and thread structure without parsing a React-rendered page. It is not the same thing as having permission to build a commercial Reddit data product.

Some examples of what this looks like in practice:

• `https://www.reddit.com/r/programming.json` — top 25 posts with full post metadata: title, score, author, timestamp, comment count, URL, flair, subreddit
• `https://www.reddit.com/r/programming/comments/abc123.json` — full comment thread with nested replies, author karma, timestamps, and vote counts for every comment
• `https://www.reddit.com/r/programming/search.json?q=python+scraping&sort=new` — search results as structured JSON, sortable by new/top/relevance
• `https://www.reddit.com/user/username.json` — user post and comment history (up to last 1,000 items), public profiles only
• Add `?limit=100` to any listing endpoint to get up to 100 items per request instead of the default 25

For small pulls, the safest pattern is slow and identified access: set a descriptive `User-Agent`, wait between requests, and back off immediately on 429 responses. Do not rotate identities, hide your purpose, or keep hammering endpoints after Reddit rate-limits you.

The important caveat: public page JSON can change without notice. For one-time research, QA, or internal analysis, it can be enough. For an app, customer-facing feature, commercial reporting product, or anything near model training, use Reddit's approved API path and current terms instead of treating page JSON as a durable contract.

How to Scrape Reddit With Python Without Paying for the API

The simplest Python Reddit scraper uses public page JSON or old.reddit.com HTML with a descriptive User-Agent and conservative request timing. This can work for small internal pulls, but apps, commercial workflows, and higher-volume research should use Reddit's approved Data API route.

Python requests can work on Reddit for narrow public-page pulls because the common failure mode is usually rate limiting or empty JavaScript-rendered HTML, not an immediate CAPTCHA wall. That makes Reddit easier to inspect than some social platforms, but it also makes overcollection tempting. Keep the workflow small, transparent, and source-backed.

Two Python paths in 2026: the `.json` endpoint (no auth, structured output, faster parsing) or `old.reddit.com` HTML scraping (more stable DOM, simpler HTML than new Reddit). New Reddit's pages are React-rendered — the HTML shell is nearly empty until JavaScript runs, which means `requests.get('https://www.reddit.com/r/...')` returns essentially nothing useful. Use `old.reddit.com` for HTML scraping, or use the `.json` endpoint and skip HTML entirely.

A minimal working approach using the JSON endpoint:

• Set `User-Agent: 'python:myapp:v1.0 (by /u/yourusername)'` — Reddit rejects generic requests and UA-less clients faster than identified ones
• Add delays between requests and back off on 429 responses instead of retrying aggressively
• Use the `after` cursor parameter from each response to paginate: each response includes an `after` value pointing to the next page of results
• Handle 429 responses by backing off for 60 seconds and retrying — Reddit's rate limit windows reset quickly

For any ongoing monitoring pipeline, think carefully about what you actually need before collecting. A targeted daily export of one subreddit search is very different from crawling thousands of threads. See the dynamic websites scraping guide for how to handle more complex rendering scenarios if you run into new.reddit.com pages that require JavaScript.

Scraping Reddit Comment Threads and Nested Replies

Reddit comment threads are nested: top-level comments can contain child replies, which can contain more replies. For spreadsheet analysis, flatten visible public comments into rows with author, text, timestamp, score, depth, and source URL so the conversation can be reviewed without losing context.

Comment threads are the most valuable data Reddit has and the most annoying to scrape. The nesting structure means you can't just pull a flat list — a comment at depth 5 is a reply to a reply to a reply to a reply to a root comment. The `.json` endpoint returns this as a nested object tree, which you have to traverse recursively.

Structured thread responses usually separate the post data from the comments. Comments are not flat records; they include reply trees, collapsed sections, timestamps, authors, scores, and nesting depth. A parser needs to walk that tree and preserve parent-child context if the analysis depends on who replied to whom.

For most qualitative research, the visible high-engagement comments are enough to identify objections, product language, competitor comparisons, and recurring pain points. If you need complete thread coverage for a public report or academic study, use an approved workflow and document the collection method.

For thread research where you need to read and analyze comments as part of a qualitative workflow — not just export a dataset — Clura handles this faster than writing a recursive parser. Open the thread on Reddit, let it load (Reddit auto-expands the top threads in the browser), click Clura, and export all visible comment text as a flat CSV. You get author, text, score, and timestamp without traversing any JSON trees. See the full social media scraper guide for the complete workflow across all platforms.

Pushshift Is Dead — What Replaced It?

Pushshift went offline in June 2023 after Reddit revoked its API access. The only partial replacement for historical Reddit data is Arctic Shift, a community-maintained archive that preserved data through early 2023. For data after June 2023, there is no historical archive — you need to have been scraping it live at the time, or accept that historical depth is unavailable.

Pushshift was the backbone of most serious Reddit research setups — it indexed Reddit's full post and comment history going back to 2005, making it possible to search, filter, and pull historical data in ways the official API never allowed. When Reddit revoked Pushshift's API access in June 2023, that entire historical layer disappeared.

Arctic Shift (https://arctic-shift.photon-reddit.com) preserved a significant chunk of data through early 2023 and runs as a free community service. It covers most major subreddits with full post and comment text, searchable by keyword, subreddit, date range, and author. It doesn't have Reddit's full data — some subreddits weren't indexed before the shutdown, and recent data (post-June 2023) doesn't exist there. But for historical trend analysis, brand research going back several years, or academic use, it's the best available option.

For anything after June 2023, there is no historical archive. The only way to have historical data from that period is if you were running a live scraper at the time and storing the results yourself. This is the argument for setting up a lightweight scheduled Reddit scraper now — even if you don't need the data today, having a rolling 90-day archive of a subreddit costs almost nothing to maintain and is impossible to reconstruct retroactively.

• Arctic Shift — community-maintained Pushshift archive through early 2023; free, searchable by keyword/subreddit/date
• Reddit's own search — available at reddit.com/search, limited to ~last 1,000 posts in any query, no historical depth
• Google site:reddit.com [keyword] — surfaces older indexed posts; not comprehensive but catches high-upvote historical threads
• Internet Archive Wayback Machine — captures individual subreddit snapshots; useful for occasional historical checkpoints, not bulk data

Is Scraping Reddit Legal?

Scraping publicly visible Reddit pages for internal research is different from republishing, reselling, training models on, or productizing Reddit data. Reddit's current terms restrict unapproved automated and commercial use, so keep exports narrow, public, source-backed, and reviewed by counsel for sensitive or commercial workflows.

In the US, public-web scraping law and platform terms are separate questions. Legal precedent has generally treated public pages differently from access-controlled systems, but Reddit's current terms still restrict automated access, commercial use, resale, and model-training uses without approval. That means a small internal export for research is a very different risk profile from a commercial product built on Reddit data.

Responsible Reddit scraping means exporting only public pages you can view in the browser, keeping source URLs with every row, avoiding private subreddits and personal data enrichment, and not republishing user comments out of context. If the data is going into a public product, customer deliverable, model-training workflow, or resale dataset, use Reddit's approved API and legal review.

For broader legal context, read the web scraping legality guide. For Reddit specifically, start with Reddit's current Data API Terms and Developer Terms because they define what Reddit permits contractually, even when a page is publicly visible.

Frequently Asked Questions

What is a Reddit scraper?

A Reddit scraper is a tool that extracts public post data, comment threads, subreddit search results, timestamps, scores, authors, and source URLs from Reddit without manual copy-paste. The main 2026 approaches are browser-based tools like Clura for visible pages, Python scripts for small internal pulls, and Reddit's Data API route for approved app or commercial use.

Does PRAW still work after the 2023 API changes?

Yes. PRAW still works as a Python wrapper for Reddit's API, but you need a registered Reddit app, OAuth credentials, and compliance with Reddit's current API and Developer Terms. For commercial, higher-volume, or productized access, do not assume old free API guidance still applies; verify your use case with Reddit's current terms.

What is the Reddit JSON endpoint and how do I use it?

The Reddit JSON endpoint works by appending .json to any public Reddit URL. reddit.com/r/python.json returns the top 25 posts as structured JSON. reddit.com/r/python/comments/abc123.json returns a full comment thread. Add ?limit=100 for up to 100 results per page and use the `after` value in each response to paginate. This endpoint doesn't require OAuth or an API key — rate limiting is enforced at ~1 request per 2 seconds with a proper User-Agent header.

Is Pushshift still available?

No. Pushshift went offline in June 2023 after Reddit revoked its API access. The community-maintained Arctic Shift project preserved data through early 2023 and is still accessible as a free archive. For data after June 2023, there is no historical archive — you need to have been scraping it live during that period, or accept that the historical depth doesn't exist.

How do I scrape Reddit comments from a thread?

The most direct method: call reddit.com/r/[sub]/comments/[post-id].json to get the thread as nested JSON. The response includes post data and a comment tree — top-level comments contain arrays of child replies. Reddit returns up to 500 top-level comments per request; deeper subtrees are returned as `more` objects that require additional calls to /api/morechildren. For on-demand qualitative research, Clura extracts visible comments from a thread to CSV without traversing the JSON structure.

What data can I scrape from a subreddit?

Public subreddits expose post titles, scores, author usernames, timestamps, comment counts, post URLs, flairs, and subreddit names. Public threads expose comment text, authors, timestamps, scores, and nesting depth. Private subreddits, direct messages, mod queues, and account-only areas should not be scraped or bypassed.

Is scraping Reddit legal?

Scraping public Reddit pages for narrow internal research is lower risk than collecting private data, bypassing access controls, republishing comments, reselling datasets, or training models. Reddit's terms restrict unapproved automated and commercial use, so sensitive workflows should use Reddit's approved API path and legal review.

Why does my Python Reddit scraper return empty results on the main reddit.com?

New reddit.com is a React application — the HTML shell delivered to Python's requests library is nearly empty, with post content injected by JavaScript after page load. Python's requests library doesn't run JavaScript. The fix is either using old.reddit.com (plain HTML, no JS required) or the .json endpoint (bypass HTML entirely and get structured JSON directly). Playwright works on new.reddit.com but is significantly slower for something that old.reddit.com handles without a headless browser.

Conclusion

The Reddit scraping landscape in 2026 is workable but narrower than it was before the API changes. Public posts, comments, and search result pages can still be exported for research, but commercial or productized use needs much more care. For one-time subreddit research and comment analysis, Clura keeps the workflow browser-based and reviewable.

The biggest mistake I see in new Reddit scraping setups is using requests on new.reddit.com and assuming empty HTML means a permanent block. Switch to a visible browser workflow, old.reddit.com, or a structured page/API approach depending on your use case. Keep source URLs attached, avoid private data, and choose Reddit's approved API path when the workflow becomes commercial or ongoing.

Explore related guides:

• Social Media Scraper Guide — All eight platforms compared — TikTok, Reddit, Facebook, X, Instagram, YouTube, Pinterest, Telegram — what each public workflow can expose.
• Twitter / X Scraper — X also killed its free API tier — same story, different technical approach to scraping public profiles and threads.
• Avoid Getting Blocked — Rate limiting vs TLS fingerprinting vs behavioral detection — why Reddit's defense is different from TikTok or Google.
• Scraping Dynamic Websites — Why new.reddit.com returns empty HTML and how to handle JavaScript-rendered pages with and without a headless browser.
• Web Scraping for Lead Generation — Combining Reddit research with outreach — from finding the right subreddits to building a contact pipeline.
• Scraper API Comparison — Managed scraping APIs compared — when a cloud service makes more sense than running your own Reddit scraper.
• Telegram Scraper — Telegram kept its API free while Reddit locked it down — how the MTProto approach compares, and where Telegram still bans accounts.