robots.txt parser

HTTP /api/v1/http/robots
Fetch and parse robots.txt — User-agent groups, Disallow/Allow, Sitemap, Crawl-delay.
Try: https://host.com https://checkhost.com https://hostinfo.com https://crypt.tools
Related: HTTP headers Security headers grader Redirect chain tracer SSL certificate inspector robots.txt parser sitemap.xml inspector
https://gameoff.github.com/robots.txt 200 17251 bytes 0 User-agent groups
Raw robots.txt
<!DOCTYPE HTML><html lang="en"><head><meta charset="UTF-8"/><META name="GENERATOR" content="IBM HomePage Builder 2001 V5.0.0 for Windows"><meta name="msvalidate.01" content="3BB4D18369B9C21326AF7A99FCCC5A09" /><meta property="fb:app_id" content="537395183072744" /><title>Game Off 2026 - itch.io</title><meta name="csrf_token" value="WyJ5c1dWIiwxNzg5NTIwNzM2LCI5ZEdORXBNWDVPRGVhb0siXQ==.J55HBFImv+ANq6uF0xCrUZXLH7w=" /><meta content="Game Off 2026" property="og:title"/><meta content="itch.io" property="og:site_name"/><meta content="4503599627724030" property="twitter:account_id"/><link href="/static/manifest.json" rel="manifest"/><script type="application/ld+json">{"itemListElement":[{"@type":"ListItem","position":1,"item":{"@id":"https:\/\/itch.io\/jams","name":"Jams"}}],"@context":"http:\/\/schema.org","@type":"BreadcrumbList"}</script><meta content="summary_large_image" name="twitter:card"/><meta content="https://img.itch.zone/aW1nLzI4MDk2NjcyLnBuZw==/508x254%23mb/zh56YX.png" property="twitter:image"/><meta content="@itchio" name="twitter:site"/><meta content="Game Off 2026" name="twitter:title"/><meta content="A game jam from 2026-11-01 to 2027-01-01 hosted by GitHub. Game Off is an annual game jam challenging individuals and teams to build a game during the month of November. Use whatever programming languages, gam..." name="twitter:description"/><meta content="A game jam from 2026-11-01 to 2027-01-01 hosted by GitHub. Game Off is an annual game jam challenging individuals and teams to build a game during the month of November. Use whatever programming languages, gam..." property="og:description"/><meta content="A game jam from 2026-11-01 to 2027-01-01 hosted by GitHub. Game Off is an annual game jam challenging individuals and teams to build a game during the month of November. Use whatever programming languages, gam..." name="description"/><meta content="width=device-width, initial-scale=1" name="viewport"/><meta content="#FA5C5C" name="theme-color"/><meta content="https://img.itch.zone/aW1nLzI4MDk2NjcyLnBuZw==/original/9sb4ib.png" property="og:image"/><link href="https://static.itch.io/jam.css?1789519975" rel="stylesheet"/><script type="text/javascript">window.itchio_translations_url = 'https://static.itch.io/translations';</script><script src="https://static.itch.io/lib.min.js?1789519975" type="text/javascript"></script><script src="https://static.itch.io/bundle.min.js?1789519975" type="text/javascript"></script><script data-src="https://static.itch.io/react.min.js?1789519975" id="lib_react_src"></script><script src="https://static.itch.io/jam.min.js?1789519975" type="text/javascript"></script><script type="text/javascript">I.current_user = null;</script><script type="text/javascript">if (!window.location.hostname.match(/localhost/)) {      window.dataLayer = window.dataLayer || [];
      function gtag(){dataLayer.push(arguments);}
      gtag('js', new Date());
      gtag('config', "G-36R7NPBMLS", {});
      (function(d, t, s, m) {
        s = d.createElement(t);
        s.src = "https:\/\/www.googletagmanager.com\/gtag\/js?id=G-36R7NPBMLS";
        s.async = 1;
        m = d.getElementsByTagName(t)[0];
        m.parentNode.insertBefore(s, m);
      })(document, "script");
      }</script><style type="text/css" id="jam_theme">:root{--itchio_ui_bg: #2d2936;}body{background-color: #190343;}.jam_page_wrap{color: #ffffff;background-color: #190343;font-family: Lato,Lato,LatoExtended,sans-serif;font-size: 100%;--itchio_border_radius: initial;--itchio_link_color: #FF32C5;--itchio_border_color: #381966;--itchio_button_color: #FF32C5;--itchio_button_fg_color: #ffffff;--itchio_button_shadow_color: #eb1eb1;--itchio_button_border_color: #eb1eb1;--itchio_button_hover_color: #f82bbe;--itchio_button_active_color: #ff38cb;}.jam_body{border-radius: var(--itchio_border_radius,0);}.view_jam_page h1,.view_jam_page h2,.view_jam_page h3,.view_jam_page h4,.view_jam_page h5,.view_jam_page h6{}.view_jam_page .jam_body,.view_jam_page .grid_outer{background-color: rgba(33, 2, 79, 1);}.view_jam_page .jam_body{box-shadow: 0 0 0 2px transparent;}.view_jam_page .grid_outer .itch_footer{border-top: 1px solid #422370 !important;background-color: #381966;}.view_jam_page .game_grid_widget .game_cell .game_text,.view_jam_page .grid_outer .game_count,.view_jam_page .grid_outer .footer_submissions{color: #ad8edb !important;}.view_jam_page .game_grid_widget .game_cell .game_platform,.view_jam_page .game_grid_widget .game_cell .game_genre,.view_jam_page .game_grid_widget .game_cell .game_author a,.view_jam_page .grid_outer .powerd_by{color: #8a6bb8 !important;}.view_jam_page .header_tabs .community_activity{background-color: #381966;color: #ffffff;}.view_jam_page .jam_filter_picker .filter_item{color: #8a6bb8 !important;}.view_jam_page .jam_filter_picker .filter_item:hover,.view_jam_page .jam_filter_picker .filter_item.active{color: #ad8edb !important;border-color: #FF32C5 !important;}.view_jam_page .jam_filter_picker .divider{color: #ad8edb;} </style></head><body class="locale_en jam_layout_widget layout_widget no_theme_toggle responsive" data-page_name="view_jam" data-host="itch.io"><div id="jam_layout_header_9345562" class="small_logo jam_layout_header_widget header_widget base_widget gray"><a class="skip_to_main" href="#maincontent">Skip to main content</a><div class="primary_header"><h1 title="itch.io - indie game hosting marketplace" class="title"><a class="header_logo" href="/" title="itch.io - indie game hosting marketplace"><span class="visually_hidden">itch.io</span><img alt="itch.io logo &amp; title" width="775" height="199" class="full_logo" src="https://static.itch.io/images/logo-black-new.svg"/><img alt="itch.io logo" width="262" height="235" class="minimal_logo" src="https://static.itch.io/images/itchio-textless-black.svg"/></a></h1><div class="header_buttons"><a data-label="browse" href="/games" class="header_button browse_btn">Browse Games</a><a data-label="jams" href="/jams" class="header_button jams_btn">Game Jams</a><a data-label="developers" href="/developers" class="header_button developers_btn"><span aria-hidden="true" class="icon icon-upload"></span>Upload Game</a><a data-label="devlogs" href="/devlogs" class="header_button devlogs_btn">Developer Logs</a><a data-label="community.home" href="/community" class="header_button community.home_btn">Community</a></div><form class="game_search" action="/search"><input class="search_input" placeholder="Search for games, jams, tags or creators" type="text" required="required" name="q"/><button class="submit_btn" aria-label="Search"><svg viewBox="0 0 24 24" version="1.1" stroke-linecap="round" stroke-linejoin="round" fill="none" stroke-width="3" aria-hidden width="18" height="18" class="svgicon icon_search" role="img" stroke="currentColor"><circle cx="11" cy="11" r="8"></circle><line x1="21" y1="21" x2="16.65" y2="16.65"></line></svg></button></form><div id="user_panel_9188974" class="user_panel_widget base_widget"><a data-label="log_in" href="/login" class="panel_button" data-register_action="header">Log in</a><a data-label="register" href="/register" class="panel_button register_button" data-register_action="header">Register</a></div></div><div class="header_dropdown" data-target="browse"><a href="/games/store">Indie game store</a><a href="/games/free">Free games</a><a href="/games/fun">Fun games</a><a href="/games/tag-horror">Horror games</a><div class="divider"></div><a href="/tools">Game development</a><a href="/game-assets">Assets</a><a href="/comics">Comics</a><div class="divider"></div><a href="/sales">Sales</a><a href="/bundles">Bundles</a><div class="divider"></div><a href="/jobs">Jobs</a><div class="divider"></div><a href="/tags">Tags</a><a href="/game-development/engines">Game Engines</a></div></div><div class="jam_page_wrap" id="wrapper"><div id="view_jam_4709979" class="view_jam_page view_jam_base_page page_widget base_widget view_jam_default before_start has_banner"><div class="jam_banner_outer"><img src="https://img.itch.zone/aW1nLzI4MDk2NzAzLnBuZw==/original/OrpHqX.png" class="jam_banner "/></div><div class="jam_body"><div id="jam_header_7494069" class="jam_header_widget base_widget"><div class="header_inner"><div class="header_columns"><div class="title_colum"><h1 class="jam_title_header"><a href="/jam/game-off-2026">Game Off 2026</a></h1><div class="jam_host_header">Hosted by <a href="https://github.itch.io">GitHub</a> · <a href="https://bsky.app/hashtag/GitHubGameOff">#GitHubGameOff</a></div></div><div class="stats_container"><div class="stat_box"><div class="stat_value">194</div><div class="stat_label">Joined</div></div></div></div><div class="header_tabs"><a data-label="tab_view_jam" href="/jam/game-off-2026" class="header_tab active">Overview</a><div class="tab_group"><a href="/jam/game-off-2026/community" class="header_tab">Community</a><span class="community_activity">1</span></div></div></div></div><div class="submitter_outer"><div id="jam_submitter_4656555" class="jam_submitter_widget base_widget before_start"><div class="date_data">Submissions open from <span class="date_format" data-format="MMMM Do YYYY [at] h:mm A">2026-11-01 21:37:27</span> to <span class="date_format" data-format="MMMM Do YYYY [at] h:mm A">2026-12-01 21:37:00</span></div><div class="countdown"><span class="countdown_label">Starts in</span><div class="times"><div data-name="years" class="time_block"><div class="block_value"></div><div class="block_label">years</div></div><div data-name="months" class="time_block"><div class="block_value"></div><div class="block_label">months</div></div><div data-name="days" class="time_block"><div class="block_value"></div><div class="block_label">days</div></div><div data-name="hours" class="time_block"><div class="block_value"></div><div class="block_label">hours</div></div><div data-name="minutes" class="time_block"><div class="block_value"></div><div class="block_label">minutes</div></div><div data-name="seconds" class="time_block"><div class="block_value"></div><div class="block_label">seconds</div></div></div></div><div class="state_action"><button class="button fat submitter_button join_jam_btn" data-method="post" data-href="/jam/417834/join">Join jam</button></div></div></div><div class="jam_content user_formatted"><p>Game Off is an annual game jam challenging individuals and teams to build a game during the month of November. Use whatever programming languages, game engines, or libraries you like.&nbsp;</p><p>The theme will be announced on November 1st, 2026.</p>
<figure><br>
<img height="80" src="https://img.itch.zone/aW1nLzEzNjAyOTg3LnBuZw==/original/NHgMM%2F.png" loading="lazy"></figure>
<ol><li>Sign up. <a href="https://github.com/join" referrerpolicy="origin" rel="nofollow noopener">Create a free GitHub account</a> if you don’t have one.
</li><li>Join the jam. Hop onto the <a href="https://itch.io/jam/game-off-2026">itch.io Game Off page</a>. If you don’t already have an itch.io account, you can <a href="https://itch.io/login?intent=jam_submit&return_to=https%3A%2F%2Fitch.io%2Fjam%2Fgame-off-2026">sign in with your GitHub account</a>.
</li><li>Create a public repository. Store your source code on GitHub&mdash;open-source magic at its finest. Push your game before December 1 at 13:37 PST.
</li><li>Submit your game on itch.io. Once submitted, you’ll be able to play other entries and cast your votes!
</li></ol>
<p><img height="80" src="https://img.itch.zone/aW1nLzEzNjAzMDU5LnBuZw==/original/SyIy67.png" loading="lazy">
</p>
<p>After the submission period ends, participants will vote on each other’s games. Entries will be evaluated in the following categories:
</p>
<ul><li>Overall
</li><li>Gameplay
</li><li>Graphics
</li><li>Audio
</li><li>Innovation
</li><li>Theme Interpretation
</li></ul>
<p>Voting will end on January 1, 2027, at 13:37 PST.&nbsp;</p>
<p><img height="80" src="https://img.itch.zone/aW1nLzEzNjAyOTY3LnBuZw==/original/3qNvfT.png" loading="lazy">
</p>
<h2>New to game development?</h2>
<p>Game jams are a great excuse to build your first game, play with a new game engine, or learn a new language. There are plenty of free, open source and “source available” engines to explore. Here are a few suggestions for some popular languages:
</p>
<ul><li>JavaScript: you might be interested in <a href="http://phaser.io/" referrerpolicy="origin" rel="nofollow noopener">Phaser</a> or <a href="https://sprig.hackclub.com/" referrerpolicy="origin" rel="nofollow noopener">Sprig</a>.
</li><li>Python: check out <a href="https://www.pygame.org/" referrerpolicy="origin" rel="nofollow noopener">Pygame</a> or <a href="https://godotengine.org/" referrerpolicy="origin" rel="nofollow noopener">Godot</a> (Godot uses GDScript, which is similar to Python).
</li><li>C++ or C#: look at <a href="https://godotengine.org/" referrerpolicy="origin" rel="nofollow noopener">Godot</a>, <a href="https://unity.com/" referrerpolicy="origin" rel="nofollow noopener">Unity</a>, and <a href="https://www.unrealengine.com/en-US/" referrerpolicy="origin" rel="nofollow noopener">Unreal Engine</a>.
</li><li>Rust: you might like <a href="https://bevyengine.org/" referrerpolicy="origin" rel="nofollow noopener">Bevy</a>.
</li><li>Java: take a look at <a href="http://www.libgdx.com/" referrerpolicy="origin" rel="nofollow noopener">libGDX</a>.
</li><li>Flutter: take a peek at <a href="https://flame-engine.org/" referrerpolicy="origin" rel="nofollow noopener">Flame</a>.
</li><li>Go: you might want to look at <a href="https://ebitengine.org/" referrerpolicy="origin" rel="nofollow noopener">Ebitengine</a>.
</li><li>Lua: check out <a href="https://love2d.org/" referrerpolicy="origin" rel="nofollow noopener">LÖVE</a> or <a href="https://www.defold.com/" referrerpolicy="origin" rel="nofollow noopener">Defold</a> or <a href="https://ramilego4game.itch.io/liko12">LIKO-12</a> (especially if you like retro games)!
</li></ul>
<p>The <a href="https://itch.io/jam/game-off-2026/community">Game Off 2026 Community</a> is a great place to ask questions or look for teammates. There’s also a fantastic and friendly community-run <a href="https://discord.gg/4hAr7sa" referrerpolicy="origin" rel="nofollow noopener">Discord server</a>
</p>
<h2>New to Git or GitHub?</h2>
<p>Game Off is the perfect opportunity to check it out (version control pun intended)!
</p>
<ul><li><a href="https://git-scm.com/doc" referrerpolicy="origin" rel="nofollow noopener">Git Documentation</a>: learn everything about version control and how to get started.
</li><li><a href="https://help.github.com/" referrerpolicy="origin" rel="nofollow noopener">GitHub Help</a>: explore tutorials and FAQs about GitHub.
</li><li>Any questions about GitHub? Visit the <a href="https://github.com/orgs/community/discussions/" referrerpolicy="origin" rel="nofollow noopener">GitHub Community Forum</a>.
</li></ul>
<p>Good luck and have fun! Can’t wait to play, err, test all your games! &lt;3
</p>
<p><br>
</p>
<p><br></p></div></div><script id="add_game_to_jam_lightbox_tpl" type="text/template"><div class="add_game_to_jam_lightbox_widget lightbox_widget base_widget  force_login has_fields lightbox" role="dialog" id="add_game_to_jam_lightbox_7070353" tabIndex="-1"><button class="close_button"><svg viewBox="0 0 24 24" version="1.1" stroke-linecap="round" stroke-linejoin="round" fill="none" stroke-width="2" aria-hidden width="18" height="18" class="svgicon icon_cross2" role="img" stroke="currentColor"><line x1="18" y1="6" x2="6" y2="18"></line><line x1="6" y1="6" x2="18" y2="18"></line></svg></button><h2>Log in to continue</h2><p>This jam is using <a target="_blank" href="http://itch.io">itch.io</a> to host entries. You&#039;ll need to register or log in on
      itch.io in order to <span class="on_submit">submit your game</span><span class="on_join">join this jam</span>.</p><p class="on_submit">After signing in you can create a new game on itch.io then
      return here and click the <em>submit game</em> button again to enter your game.</p><p><a class="button" href="/login?return_to=https%3A%2F%2Fitch.io%2Fjam%2Fgame-off-2026&amp;intent=jam_submit" data-register_action="jam_submit">Log in</a><span class="or_split"> or </span><a class="button" href="/register?return_to=https%3A%2F%2Fitch.io%2Fjam%2Fgame-off-2026&amp;intent=jam_submit" data-register_action="jam_submit">Register</a></p></div></script></div></div><script id="loading_lightbox_tpl" type="text/template"><div class="lightbox loading_lightbox" aria-live="polite"><div class="loader_outer"><div class="loader_label">Loading</div><div class="loader_bar"><div class="loader_bar_slider"></div></div></div></div></script><script type="text/javascript">new I.ViewJam('#view_jam_4709979', {"start_date":"2026-11-01 21:37:27","end_date":"2026-12-01 21:37:00","id":417834,"voting_end_date":"2027-01-01 21:37:00"});init_JamLayoutHeader('#jam_layout_header_9345562', {"autocomplete_props":{"i18n":{"search_placeholder":"Search for games, jams, tags or creators","search":"Search"},"ca_source":25,"ca_types":{"jam":4,"game":1,"bundle":6,"featured_tag":7,"browse_facet":8},"autocomplete_url":"\/autocomplete","search_url":"\/search"}});
I.setup_page();</script></body></html>
Common questions

Is robots.txt parser free?
Yes — every tool is free on the web with a 200/hour rate limit per IP. The matching API endpoint /api/v1/http/robots is free up to 100 requests/hour, no key required.
Where does the data come from?
Real-time probes against authoritative sources (DNS root, RIRs, registries, the target server itself), plus partner data feeds from hostinfo.com (GeoIP/ASN) and hostcheck.com (reputation).
How fresh are the results?
Live by default. Cached for 5 minutes to make repeat queries instant; pass ?nocache=1 for a forced refresh.
Can I run this from the command line?
Yes — every tool ships with a copy-as-curl. There's also an official CLI: host.tools http robots YOUR_INPUT.
Can I monitor results over time?
Pro tier lets you schedule any tool to run every 1/5/15/60 min and alert on diff. See monitors.
host.tools Pro
Run robots.txt parser on a schedule. Get pinged when it changes.

Pro gets you bulk lookups, monitors, webhook alerts, history, exports and 10,000 API calls/day. $19/mo.
See pricing Tour monitors
✓Schedule any tool — every 1, 5, 15, 60 min
✓Diff against last run, alert on change
✓Webhook + email + Slack + PagerDuty + OpsGenie
✓Bulk CSV upload, 1,000 inputs per job
✓Export results as CSV / NDJSON / Excel
✓90-day history, comparison view