Fun With HTTP Headers
Introduction
Like any good web developer, I have a tendency to poke around at people’s web sites to see if I can figure out how they’re implemented. After poking at enough sites, I started noticing that people were putting some weird and interesting stuff in their HTTP headers. So, a couple of weeks ago, I decided to actually go out and see what I could find by scrounging around in HTTP headers in the wild. A header safari, if you will. These are the results of my hunt.
Headers?
HTTP is the protocol used to transmit data on what we know as “the web”. At the beginning of every server response on the web, there’s a bit of text like:
HTTP/1.1 200 OK Content-Type: text/html Connection: close
The top line specifies the protocol version of HTTP and a response code (200 in this case) used to indicate the outcome of a request. Following that are a bunch of lines that should consist of a field name (like “Connection”), followed by a colon, and then followed by a value (like “close” or “keep-alive”). These lines of text are the HTTP response headers. Immediately after the headers is a blank line, followed by the content of the message, such as the text of a web page or the data of an image file.
Technical Mumbo Jumbo
Want to examine the headers of a site for yourself? Try curl:
curl -i http://www.nextthing.org/
In the output of the above the first few lines are the headers, then there are a couple of line breaks, and then the body. If you just want to see the headers, and not the body, use the -I option instead of -i. Be forewarned, however, that some servers return different headers in this case, as curl will be requesting the data using a HEAD request rather than a GET request.
What I did to gather all of these headers was very similar. First, I downloaded an RDF dump of the Open Directory Project’s directory, and pulled out every URL from that file. Then, I stuck all of the domain names of these URL’s in a big database. A simple multithreaded Python script was used to download all of the index pages of these URL’s using PycURL and stick the headers and page contents in a database. When that was done, I had a database with 2,686,155 page responses and 23,699,737 response headers. The actual downloading of all of this took about a week.
This is, of course, not anywhere near a comprehensive survey of the web. Netcraft received responses from 70,392,567 sites in its August 2005 web survey, so I hit around 3.8% of them. Not bad, but I’m sure there’s a lot of interesting stuff I’m missing.
Obligatory Mention of Long Tail
First of all, yes, HTTP headers form something like a long tail:
In particular, hapax legomena (one-offs) make up over half of the headers found. I expected this. Unfortunately for me, however, a lot of the really interesting stuff is over on that long flat section of the long tail. Which means I spent a lot of time poring over one-offs looking for interesting stuff. Weee.
It’s a good thing I’m easily amused.
Off with Her Headers
I found 891 instances of:
X-Pad-For-Netscrape-Bug: 0123456789
Which brought back memories of the days when Netscape was reviled by developers the world ’round, and had not yet achieved its ultimate (albeit posthumous) glory with Firefox. It’s nice to know comments by frustrated engineers have such a long half-life on the Internet. There are a few variants on this header:
X-Pad: avoid browser bug XX-Pad: Padding aheader: WOULDN'T YOU LIKE TO KNOW! X-BrowserAlignment: problem
Similarly, people are still blocking Microsoft’s Dumb Tags:
X-MS-Smart-Tags: We have nothing to do with them. X-Meta-MSSmartTagsPreventParsing: TRUE
Speaking of Microsoft, apparently the IIS team felt the need to advertise the domain of the site the user was accessing in every page request:
Server: Microsoft-IIS/5.0 jvc.org: jvc.org
How completely and utterly unnecessary…
They’re not the only ones, though. WebObjects powered sites spit out:
HTTP/1.1 200 Apple
Go team!
This cute header is courtesy of Caudium, a webserver written partially in Pike:
X-Got-Fish: Yes
The webmaster of www.kfki.hu should be commended for being on the bleeding edge, both using Caudium and including lots of Dublin Core metadata in the headers. Although, 32 headers seems a bit much, which is why I’m not going to show them all:
DC.Subject: physics DC.Type: organizational homepage SCHEMA.DCTERMS: http://purl.org/dc/terms/ X-Got-Fish: Yes
Contrary to popular belief, there are people out there using Smalltalk on the web. Two of them. One Smalltalk software company running a web server written in Smalltalk, and another:
Server: Swazoo 0.9 (Columbus) X-WIKI-ENGINE: SmallWiki 1.0 CACHE-CONTROL: no-cache X-WIKI-COPYRIGHT: Software Composition Group, University of Berne, 2003
running a Smalltalk user’s group web site with a wiki written on Smalltalk on a web server written in, you guessed it: Smalltalk. Cool.
And, of course, it wouldn’t be the Internet without an appearance by a BOFH:
X-BOFH: http://www.xxxxx.de/bofh/xxxxxx.html
The actual URL it points to has been obscured to protect the guilty, and a local mirror provided in its stead.
Missed Cneonctions
This header:
Cneonction: close
and its variant:
nnCoection: close
were two of the headers which first spurred my interest in HTTP headers.
imdb.com, amazon.com, gamespy.com, and google.com have all at various times used these or similar misspellings of connection, and I’m not by any means the first to have noticed. My first thought was that this was just a typo. After more consideration, however, I now believe this is something done by a hackish hardware load balancer trying to “remove” the connection close header when proxying for an internal server. That way, the connection can be held open and images can be transmitted through the same TCP connection, while the backend web server doesn’t need to be modified at all. It just closes the connection and moves on to the next request. Ex-coworker and Mudd alumus jra has a similar analysis.
Another data point which would back this up is the Oracle9iAS Web Cache rewriting:
Connection: close
as
yyyyyyyyyy: close Connection: Keep-Alive
Headers with “X-Cnection: close” appear to be the result of a similar trick.
One ISP/web host is kind enough to include their web address and phone number in every request to any of their hosted servers:
Phone: (888) 817-8323 Web: www.wgn.net
This is just super-awesome. I once spent a good hour trying to find a technical contact for a certain monstrous job site to tell them their servers had been compromised and were displaying the following message to visitors:
You are being sniffed by Carnivore.
Your nation is secure.…………..OCR IS WATCHING YOU…………..
Content-type: text/html
The message, funnily enough, was being relayed by modifying the HTTP headers.
C is for Cookie
Cookies 2 were defined in RFC 2965, way back in October of 2000. As far as I know, Opera is the only browser in widespread use to support them. It’s sad, really, as the original cookie spec that Netscape came up with is kind of lame. Specifically, Netscape’s spec defines the expiration as a date, which is vulnerable to clock skew on the user’s system making the cookie expire early. The Cookies-2 spec, on the other hand, uses a max-age attribute, specifying the lifetime of the cookie in seconds:
Set-Cookie2: Meyer_Sound_777=68.126.233.177.1122451925660461; path=/; max-age=1209600; domain=.meyersound.com; version=1
There are also Comment and CommentURL fields which explain what the cookie is for, but I have yet to find a header which uses them. *sigh* On the other hand, I did find 518 Set-Cookie2 headers, which, while miniscule compared to the 764,976 SetCookie headers I received, is more than I expected. It looks like software written by Sun is responsible for most of these.
A bunch of servers spit out:
shmget() failed: No space left on device
Doh! Time to cycle some log files.
Pingback discovery headers like this show up a lot (2370 times):
X-Pingback: http://www.nextthing.org/wordpress/xmlrpc.php
I don’t even know what to say to this, found at ebrain.ecnext.com:
HTTP/1.1 200 OK Date: Sun, 24 Jul 2005 18:39:54 GMT Server: Apache Transfer-Encoding: chunked Content-Type: text/html; charset=ISO-8859-1
At least they’re not alone, as www.charlottesweb.hungerford.org will keep them company:
Turn off Pictures Popup Toolbar in IE 6.0:
And www.station.lu:
XHTML: <!DOCTYPE html PUBLIC"-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
The list goes on…
The Coral Content Distribution Network has been getting some buzz lately, so I was interested to see some
X-Coral-Control: redirect-home
headers show up. This header is used to tell Coral that if Coral can’t handle the load of requests for cached copies of your page, it should redirect these requests back to your site.
Why anyone would think to themselves, “Gee, if a massively scalable caching service running on hundreds of geographically distributed computers can’t handle the load of people wanting to look at my site, I’ll just have them bounce people back at me”, I don’t know. Masochism perhaps?
Speaking of P2P technologies, I was interested to run across a KaZaA server:
HTTP/1.0 404 Not Found X-Kazaa-Username: anonymous_user X-Kazaa-Network: KaZaA X-Kazaa-IP: xx.xx.xx.xx:1348 X-Kazaa-SupernodeIP: xx.xx.xx.x:3699
It looked like it was running on someone’s DVR. Anyone have any pointers as to what software does that?
Along the same lines, haha:
X-Kaza-Username: hrosen X-Kaza-Network: RIAA X-Kaza-IP: 146.82.174.12:80 X-Kaza-SupernodeIP: 68.163.90.12:80 X-Disclaimer: All Your Base Are Belong To Us X-Pizza-Phone: 961.1.351904
They’re not even the only ones using “X-Disclaimer”, a bunch of other sites do too:
X-Disclaimer: The local sysadmins have *nothing* to do with the content of this server.
It looks like Tux Games is trying to extend the venerable RFC 1097 to the web:
X-Subliminal: You want to buy as many games as you can afford
Personally, I would’ve gone for: “X-Superliminal: Hey you, buy some games!”.
I’m sure these kind folks would be first adopters:
X-Cotton: The Fabric of Our Lives
This person wants to make their opinion known, so here it is:
Veto: Usage of server response for statistics and advertising is disagreed!
To which I say: Take off every ‘zig’!! You know what you doing.
Robot Rock
I’d never really paid much attention to the Robots header:
ROBOTS: index,follow,cache
as it’s mostly used to disable indexing of a page and is intended to be used in a meta tag in the HTML itself, not in the HTTP headers.
However, it seems Google has added a new NOARCHIVE attribute, so let’s see who’s using it in their headers rather than in the meta tags like Google specifies.
It looks like the Singapore-based “Ministry of Pets” doesn’t want to be cached, as does the Civil Engineering department at São Paulo Polytechnic University, the realtime-3d software company MultiGen Paradigm, Swiss handicraft company Schweizer Heimatwerk, a Swiss kitesailing site, the Ragin’ Cajun Cafe in Hermosa Beach, CA, the London-based BouncingFish web consultancy, and the French financial paper La Tribune. That’s it.
BouncingFish even goes so far as to use an additional GOOGLEBOT header:
GOOGLEBOT: NOCACHE
How many of these sites are not being cached by Google? Zero. Which just goes to show that one shouldn’t just expect mix-and-matching of specs to work.
Along the same vein, I don’t think the first two headers below will work as expected:
X-Meta-ROBOTS: ALL X-Meta-Revisit-After: 1 days Robots: INDEX, FOLLOW
Except, possibly, in spiders using Perl’s HTML::HeadParser module. And, of course, we’ve already seen that the third header probably won’t work, either.
While I’m on the subject of Google… all Blogspot sites spit out:
test: %{HOSTNAME}e
So Blogger folks, whatcha doin’?
It’s Funny, Laugh
The fine folks at www.basement.com.au want to make it clear that:
Mickey-Mouse: Does_Not_Live_Here
Some people have a lot of fun with headers, as seen here:
Limerick: There was a young fellow named Fisher Limerick: Who was fishing for fish in a fissure, Limerick: When a cod, with a grin, Limerick: Pulled the fisherman in Limerick: Now they're fishing the fissure for Fisher.
This is the only ascii art I found:
<!-- Content-type: text/html ************************************************************************* * Welcome to schMOOze University * * * * ==> To connect to an existing player type: CONNECT NAME PASSWORD * * ==> To connect as a guest type: CONNECT GUEST * * * ************************************************************************* * all text is copyrighted by the various authors * * TIME FLIES LIKE AN ARROW FRUIT FLIES LIKE A BANANA * * *** * * * * * * * * * * * * * * * * * * * (__) * * * * (OO) * * * * ____________ / * * * * /| / * * * * / | |------ | | * * * * * | |^^ | | * * * * ^ ^ ^ ^ * * ************ ************ Nobody is connected. --> <HTML> <HEAD> <TITLE>Welcome to schMOOze!</TITLE> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"></HEAD> <meta http-equiv="refresh" content="0;URL=http://schmooze.hunter.cuny.edu/"> </BODY></HTML>
and it had me puzzled, until I realized it’s a telnet server, and the above is a really clever hack to redirect browsers towards HTML-land.
This made me laugh:
X-Powered-By: Intravenous Caffeine Drips X-kluged-by: Nick, Mic, Ash, Andy X-munged-by: The powers that be X-Sanity-Provided-By: Ashleigh
Apparently the site has an alter-ego, as well.
www.wrestlingdb.com has some interesting headers. A few requests gets:
X-Stone-Cold-Steve-Austin: And that's the bottom line, cause Stone Cold says so. X-Mick-Foley: Have a nice day! X-Ric-Flair: To be the man, WHOOO!, you've got to beat the man. X-Rock: If you smell what The Rock is cooking. X-Booker-T: Can you dig it, SUCKAAAA? X-Kurt-Angle: It's true, it's DAMN true. X-Hurricane: Stand Back! There's a Hurricane Coming Through! X-Kane: FREAKS RULE!
which is about as entertaining as watching a real wrestling match.
Totally Ellet
Just so everyone knows, Frostburg students are so totally leet, they don’t even need to spell it correctly:
Owned And Operted By FSU Computer Club: 31137
Speaking of which, apparently some guy named morris would like his visitors to know that he 0wnzor$ them:
X-You-Are-Owned-By: morris
Not sure where that box you rooted and are browsing the web from is located? Never fear, mobileparty.net will tell you:
X-Detected-Country: US
And, for those who were wondering, the Texarkana Police are the world’s finest, at least in the HTTP headers department:
TEXPOLICE: LAW_ENFORCEMENTS_FINEST
These nederlanders are representin’ for the westside:
X-Side: : WESTSIDE-FOR-LIFE
Western Europe, that is. Jaaa.
Speaking of furriners, anyone care to translate:
X-Sarrazin-Says: Ciccio, lascia perdere, e' un blowfish a 448 bit.
X-beliebig: Dieser Header dient der allgemeinen Verwirrung =:) X-Gleitschirmfliegen: macht Spaaaasss!
Going back to my discussion on standards, localizing headers that are used to actually do stuff is a bad idea:
Ultima Modificação: Thu, 28 Jul 2005 15:12:07 GMT
ObRef
The Democrats called, they want you to know they found their sense of humor:
X-Dubya: You teach a child to read and he or her will be able to pass a literacy test.
Make sure to hit it a few times for optimum goodness:
X-Dubya: We're in for a long struggle, and I think Texans understand that. And so do Americans. X-Dubya: Africa is a nation that suffers from incredible disease. X-Dubya: We're making the right decisions to bring the solution to an end. X-Dubya: Families is where our nation finds hope, where wings take dream.
In the politics vein:
X-Powered-By: MonkeyMag 0.02.01, (c) Niel Bornstein and Kendall Clark X-Shout-Out: No Power Without Accountability X-Mos-Defology: Speech is my hammer//Bang the world into shape//Now let it fall X-American-Leftist-Salute: Doing Woody's Work! X-Billy-Braggage: Sun, Sea, Socialism!
Yes! Someone just made my day. I love Al Bundy quotes:
X-Bundy: Here we have 3 of the seven dwarfs, puffy, crabby and horny. X-Bundy: You know I never danced unless it was gonna get some sex for me. X-Bundy: I blame it on TV myself. X-Bundy: To know me is to love me.
I was disappointed in the lack of mention of mules, donkeys, or garden gnomes, but at least llamas, mice, and loons are well represented:
X-Llamas-Version: 2.0
From: www.teevee.org
X-Favourite-Animal: Mouse
From: www.kingssing.de
X-Loons-Version: 1.5.1
From: www.eod.com
Speaking of strange characters, apparently the Wicked Witch of the West and Spongebob Squarepants cohabitate at www.harbor-club.com:
X-Wicked-Witch: West X-Spongebob: Squarepants!
Who knew?!
As if we needed further proof that the soft underbelly of the Internet is full of cults, slowly corrupting the moral fabric of society, I present:
X-SAVIOUR: BOB_DOBBS
From the looks of things, Living Slack Master Bob Dobbs is giving Jesus a run for his money among Oregonian carpenters and their web designers. They join such luminaries as R. Crumb, Devo, and Bruce Cambell.
And if you thought that was an obscure meme, try this on for size:
X-Lerfjhax: Extra yummy
When I first saw an X-Han header, I thought for sure the contents would be “Shot first!”, but instead I found something more obscure:
X-Han: 'I look forward to a tournament of truly epic proportions.'
While we’re on pop culture allusions:
X-Powered-By: Twine X-Towelie: You wanna get high?
And it would be difficult to be more obscure than this:
X-Sven: look out for the fruits of life
Finally, old school Mac-diehards should appreciate:
X-Blam: Frog blast the vent core!
Connection: close
Back when I was interviewing for an internship at Tellme Networks, they had a comment buried on their homepage that said:
<!-- (c) Copyright 2000 Tellme Networks. --> <!-- If you're looking at our HTML source, you're exactly the person who should send us your resume. We recently redesigned our site; Tell us all about how you would make it better and better yet, if you have an illustrious career of web-hacking, drop us an email at jobs@tellme.com. -->
I thought this was just way awesome. However, if I was disappointed when they removed that comment, I’m even more disappointed to report that I have yet to find a single HTTP header offering me a cool job. What’s wrong with you people?! I’m supposed to be able to find anything on the Internet!
I was, at least, thanked for my efforts, and I found the answer to life, the universe, and everything!
X-Thank-You: for bothering to look at my HTTP headers X-Answer: 42
You’re welcome! And thank you all, for making the Internet so interesting!
August 8th, 2005 at 2:34 PM
Man that is entertaining! I’m still reading, but thought that I should quickly translate this:
…before I forget it.
“X-beliebig” is a wordplay that can’t be translated without destroying the joke: “beliebig” means “arbitrary”, but is often prefixed with “x-” in informal German to illustrate the arbitrariness. so “x-beliebig” means something to the effect of “very arbitrary”.
August 8th, 2005 at 4:27 PM
Cool, thanks for the translation!
August 8th, 2005 at 4:55 PM
X-Sarrazin-Says: Ciccio, lascia perdere, e’ un blowfish a 448 bit.
Translated literally, “ciccio” means fat – but in colloquial Italian, it means more something like “hey, you”. So the message says “Hey you, just leave it, it’s a 448-bit blowfish”. I think it’s mean to discourage you from trying to crack something. 🙂
August 8th, 2005 at 8:03 PM
Haha, this was awesome, very cool.
August 9th, 2005 at 1:56 AM
Nice! Any chance of a follow up with entertaining and idiotic meta tags?
August 9th, 2005 at 5:46 AM
you made my day by proving to me that at least one person saw my x-sven header 🙂 i enjoyed the rest of your post as well!
August 9th, 2005 at 1:27 PM
My personal favorite is the X-Bender header on Slashdot. Arguably more fun than Slashdot itself these days.
August 10th, 2005 at 12:12 AM
Sweet post, Andrew! Very entertaining and impressive. Must’ve taken a long time to write.
August 10th, 2005 at 3:06 AM
[…] http://www.nextthing.org/archives/2005/08/07/fun-with-http-headers […]
August 10th, 2005 at 11:02 AM
Re: Smalltalk on the Web – if you’re at all interested in that, check out Seaside, if you haven’t already.
August 10th, 2005 at 12:00 PM
Andrew,
I noticed this post in our referrer logs for HouseDems.com.
Good find on our George W. Bush quote randomizer! It was inspired by Slashdot. We did it with a PHP auto_prepend.
-Bryan
August 11th, 2005 at 2:30 AM
Http Header
Fun With HTTP HeadersIntroductionLike any good web developer, I have a tendency to poke around at people’s web sites to see if I can figure out how they’re implemented. After poking at enough sites, I started noticing that people were putting some …
August 13th, 2005 at 2:58 PM
[…] Andrew Wooster’s nextthing.org » Fun With HTTP Headers […]
August 13th, 2005 at 7:06 PM
[…] Thanks to GeekGrrl for this gem. I downloaded Python and PyCurl just to try it out. I’ll let ya know how it goes. Andrew Wooster’s nextthing.org » Fun With HTTP Headers Similarly, people are still blocking Microsoft’s Dumb Tags: X-MS-Smart-Tags: We have nothing to do with them. X-Meta-MSSmartTagsPreventParsing: TRUE […]
August 14th, 2005 at 9:52 AM
[…] Inspiriert hat mich dazu übrigens vielleicht der Artikel Fun With HTTP Headers, den ich letzte Tage gelesen habe. Fast schon krank, wofür sich manche Leute beim Besuch von Webseiten interessieren… […]
August 14th, 2005 at 7:11 PM
[…] Fun With HTTP Headers Tagged as: cool random web […]
August 15th, 2005 at 8:51 AM
[…] Andrew Wooster’s nextthing.org » Fun With HTTP Headers […]
August 16th, 2005 at 11:32 AM
The easy way to get the headers, and only the headers, is
curl --headorlynx -head. You always have lynx.August 16th, 2005 at 9:49 PM
This is terrific stuff. I’ve heard of and seen many of these, but you’ve put together a really entertaining list here. Hell, most end users don’t look at the source code to Web pages, let alone know what happens at the protocol level. Nice work!
August 17th, 2005 at 1:29 AM
[…] read more | digg story […]
August 17th, 2005 at 5:46 AM
Slashdot hands out Futurama quotes with every page (but not the images) in the form of X-Fry or X-Bender headers, for instance:
X-Fry: I learned how to handle delicate social situations from a little show called "Three's Company."August 17th, 2005 at 7:49 AM
Check out http://livehttpheaders.mozdev.org/ for viewing headers in firefox while browsing. It’s a lot easier than curl.
August 17th, 2005 at 12:39 PM
Okay Andrew, good read, but what does this one mean?
X-The-REAL-Answer: 47
Btw, I use the Firefox web developer toolbar extension for viewing headers.
August 18th, 2005 at 12:09 AM
[…] L’altro giorno stavo girando per la rete e studiando un po’ di teoria sugli HTTP Headers quando mi sono imbattuto nel Ueberman’s sleep. L’idea e’ quella di dormire nello stesso modo di Leonardo DaVinci. Per me e’ stata un’assoluta novita’: ma a quanto pare alcuni miei amici ne erano a conoscenza… Il grande genio infatti riposava solamente grazie a delle pennichelle ridotte di una ventina di minuti ca, intervallate da periodi di veglia di 4h. Un rapido calcolo vi fara’ capire che dormiva ogni giorno 2h. […]
August 18th, 2005 at 4:24 AM
omg p30pl3 with too much time on their hands.
August 18th, 2005 at 11:21 AM
[…] It’s amazing some of the things you can stumble upon online. For instance, I stumbled upon this post this morning, which takes us on a tour of crazy HTTP Headers across the ‘net. Very interesting indeed. I’d also never have thought about using the DMOZ database as a starting point. […]
August 21st, 2005 at 8:46 PM
[…] HTTP Header Fun – Ha! # […]
August 22nd, 2005 at 12:17 AM
Okay, now that’s just too cool. Inspirational, too (:
Well done, Andrew, and thanks.
August 23rd, 2005 at 10:29 AM
[…] Surfen bedeutet f?r die meisten, einen Link anzuklicken oder die Adresse einzugeben und die Ergebnisse im Browser anzuschauen. Doch dabei entgeht dem Nutzer vieles. Im konkreten Falle n?mlich die HTTP-Header. Andrew Wooster hat die Header von 2,7 Millionen Seiten ausgewertet und steiss dabei auf sehr interessante Ergebnisse. Es fanden sich Telefonnummern (Phone: (888) 817-8323), subliminale Botschaften (X-Subliminal: You want to buy as many games as you can afford) und anderen Nettigkeiten. Die Seite ist unbedingt sehenswert! […]
August 23rd, 2005 at 2:53 PM
Read the book “The Hitch Hikers Guide to the Galaxy” (or see movie),
Short description:
The ultimate answer.
BTW: I modified brockal.funpic.de a little bit after I read this article 🙂
Alexander
August 24th, 2005 at 8:00 AM
[…] Ya, some sysadmins have a lot of fun. Andrew Wooster has blogged on his analysis of strange http headers, including webservers that send out ascii art! That’s pretty intersting. And of course, you already know the Strange NIST Time Server and Star Wars. […]
August 28th, 2005 at 2:48 PM
[…] Pour en savoir plus sur les découvertes de Andrew Wooster, rendez-vous sur son site. […]
September 6th, 2005 at 4:50 AM
[…] Andrew digs headers […]
September 6th, 2005 at 8:17 AM
[…] It’s funny what people put in their HTTP headers these days! (via) Related Stories (fun, http, headers) […]
September 6th, 2005 at 6:08 PM
[…] Ya, some sysadmins have a lot of fun. Andrew Wooster has blogged on his analysis of strange http headers, including webservers that send out ascii art! That’s pretty intersting. And of course, you already know the Strange NIST Time Server and Star Wars. […]
September 25th, 2005 at 8:34 AM
[…] Andrew Wooster: “After poking at enough sites, I started noticing that people were putting some weird and interesting stuff in their HTTP headers. So, a couple of weeks ago, I decided to actually go out and see what I could find by scrounging around in HT (tags: interesting funny geekery) […]
October 19th, 2005 at 6:50 PM
Damn. The dang darn Dems have done gone ASP!
Date: Thu, 20 Oct 2005 01:47:16 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 1.1.4322
December 2nd, 2005 at 2:33 AM
[…] You wouldn’t believe the tremedous fun it is to check out other websites HTTP header information. After finding Andrew Wooster’s original article about idiocy and fun in HTTP headers, I went out to check some sites out for myself. And you should too! So you should start with Andrew’s Blog too. […]
December 29th, 2005 at 5:13 PM
[…] Here. // Used for showing and hiding user information in the comment form function ShowUtils() { document.getElementById(“authorinfo”).style.display = “”; document.getElementById(“showinfo”).style.display = “none”; document.getElementById(“hideinfo”).style.display = “”; } function HideUtils() { document.getElementById(“authorinfo”).style.display = “none”; document.getElementById(“showinfo”).style.display = “”; document.getElementById(“hideinfo”).style.display = “none”; } […]
January 10th, 2006 at 10:35 AM
[…] I then examined the HTTP headers, and compared them with a working file. (It’s off-topic for this post, but I stumbled across this while Googling for solutions. If you’re a geek and want to see some fun headers in the wild, visit Fun With HTTP Headers.) […]
January 10th, 2006 at 9:09 PM
i had the crazyes one on my old site 😛
April 24th, 2006 at 2:32 PM
On the mispeled connection header, if you notice the positions of the characters replaced, you will find that the packet tcp & IP checksum won’t be modified with the rearrangement, which allows for higher performance when attempting to remove the header. And yes, I do know what company makes the hardware that does this.
May 15th, 2006 at 12:58 PM
Ok i have a simple question ( Aleast i hope its simple)
Wht is a Robot Header in a nut shell?
May 29th, 2006 at 6:21 AM
[…] Ever tried apt-get moo within your debian? If you think that the result is funny you should consult Andrew’s blog entry “Fun with HTTP Headers” or the Unix Error Messages Page. Have fun. […]
June 30th, 2006 at 4:07 PM
Never thought about headers until I viewed your site. Great stuff!
August 31st, 2006 at 6:15 AM
[…] here is an in-depth analysis of how headers work, and other design and technical nuances of headers.read more | digg story […]
February 21st, 2007 at 4:23 PM
[…] nextthing.org » Fun With HTTP Headers (tags: analysis statistics internet http) […]
March 12th, 2007 at 11:58 PM
[…] Last October I got bored and set my spider loose on the robots.txt files of the world. Having had a good deal of positive feedback on my HTTP Headers survey, I had decided to poke around in robots.txt files and see what sorts of interesting things I could find. […]
April 27th, 2007 at 7:10 PM
If it’d been 42 that’d be obvious. Something like
X-Thursdays: Never could get the hang of Thursdays
might be a little more obscure 😉
And yes, Han shot first and that’s the way it was meant to be.
May 22nd, 2007 at 9:45 AM
[…] read more | digg story […]
August 2nd, 2007 at 10:28 AM
That was a very interesting, instructive yet entertaining read—thank you!
August 4th, 2007 at 4:07 AM
concerning the cool job-offerings in http-headers; check out the wordpress.com-site, they have a header that reads:
“X-hacker If you’re reading this, you should visit automattic.com/jobs and apply to join the fun, mention this header.”
good luck with your applications guys! 😉
August 5th, 2007 at 12:34 PM
Awesome post! I was all prepared to post the automattic job offer header, but futtta beat me to it. Damn.
September 2nd, 2007 at 1:03 PM
Interesting read, searching for something stumbled here,
glad i did 😀
thanx.
M.
September 3rd, 2007 at 12:58 PM
just got inspired and added some 😀
Rahul.
September 5th, 2007 at 4:46 AM
Thanks a TON!
September 23rd, 2007 at 10:31 AM
An incredible article. I like it very much. It took me an hour to read! May be I’ll translate it too…
September 23rd, 2007 at 6:30 PM
[…] nextthing.org » Fun With HTTP Headers X-Favourite-Animal: Mouse (tags: analysis http headers response web survey internet www research) […]
September 24th, 2007 at 12:01 PM
[…] Interesting web surveys: robots.txt and http headers (via Simon Willison). […]
October 26th, 2007 at 9:03 AM
I’d like to see these headers (Maybe next time I run my own server):
X-Men: Wolverine
X-Factor: Was a better book
X-Force: Allred’s run rocked
Yes, I am a geek.
February 29th, 2008 at 10:13 AM
I noticed a funny typo in the headers sent by http://www.ibm.com
Note the ‘Kp-eeAlive’ instead of ‘Keep-Alive’.
Does anyone know if it’s really a typo or if it was intended ?
In the later case, what for ? to trick proxy ?
March 12th, 2008 at 7:55 AM
[…] Fun met HTTP-header […]
May 5th, 2008 at 9:12 AM
‘Kp-eeAlive’ makes a lot of sense for MS IE6 when using SSL as it does not support it properly. HTTP POST requests used for logins and uploads usually fail so one may want to disable keep alives.
August 28th, 2008 at 11:11 AM
[…] at ways to add fancy headers for this server as well … 🙂 You can read more about it on this Fun With HTTP Headers article! Share and Enjoy:I’m not writing this blog to get rich – but because I like it. So if you […]
August 30th, 2008 at 1:39 PM
[…] amazing some of the things you can stumble upon online. For instance, I stumbled upon this post this morning, which takes us on a tour of crazy HTTP Headers across the ‘net. Very […]
November 5th, 2008 at 5:09 AM
[…] manchen Sites scheinen da auch echt merkwürdige Sachen drinzustehen. Unter dem Titel Fun With HTTP Headers hat jemand mal getestet, was für Sachen in diesen Headern manchmal drinstehen. Gefunden hat er […]
November 9th, 2008 at 11:14 AM
[…] Fun With HTTP Headers — о сколько есть открытий чудных, для тех кто знает curl […]
February 11th, 2009 at 8:35 AM
the “Cneonction: close” thing is a quirk of Netscaler loadbalancers. It’s done to nullify any “Connection: close” headers the webserver spits out, as the Netscaler wants to manage it better. It’s scrambled instead of removed so that it doesn’t have to regenerate packets (length is the same) and it’s scrambled semi-randomly so that people don’t just assume it’s a misspelling and add compatibility for it.
February 11th, 2009 at 9:35 AM
Andrew, thank you for taking the time to do this — especially for trawling through to find the interesting bits and pieces!!
Perhaps you could throw the dump of the Headers up somewhere and a pseudo-digg-clone with +1 could be setup to collectively identify other interesting headers?
Anyways, just a thought… Keep up the interesting finds!
—
tav
February 11th, 2009 at 11:30 AM
Awesome, thanks for the great read!
February 11th, 2009 at 2:56 PM
FYI – CoralCDN rejects a lot of its traffic due to bandwidth limits on the experimental network test bed it runs on (PlanetLab). The header allows sites to use CoralCDN without worrying that it might silently reject its traffic.
February 11th, 2009 at 3:00 PM
“X-I-Am-Le-Tired: Fine, take a nap.”
Glad to see you’re taking part too. Checked you out using firebug (FF addon), the webdeveloper toolbar also allows headers to be checked. FWIW.
February 11th, 2009 at 7:14 PM
Andrew, This is one of the most entertaining and informative posts I have read in a long term. Thanks for investing the time to give us this good stuff 😀
February 11th, 2009 at 10:41 PM
[…] of Religion Map Fun With HTTP Headers Why We’re Probably in For a Long Recession How to write a Linux virus in 5 easy steps How I Built a […]
February 12th, 2009 at 1:41 AM
curl -s -I slashdot.org | grep -e “Bender\|Fry\|Leela” | sed ‘s/X-//’
February 12th, 2009 at 2:09 AM
Stupid curly quotes. 🙂
February 12th, 2009 at 2:34 AM
shorter, and working (modulo wordpress screwing with the quotes….)
curl -sI slashdot.org|egrep ‘Bender|Fry|Leela’|cut -b3-
February 12th, 2009 at 2:35 AM
and yep, it wrecked the quotes. stupid wordpress. if i want curly quotes, i’ll damn well type them–it’s easy on my mac.
February 12th, 2009 at 2:40 AM
Hah, yeah, I have nothing but bad things to say about WordPress and its munging of my text. *sigh*
February 28th, 2009 at 1:59 PM
Эх…, а я тут за свои годы, как-то привык ко всему этому, даже внимания на это не обращаю 🙂 Вы тоже привыкнете со временем 🙂
March 5th, 2009 at 9:44 AM
Ah, this was well worth a read.
Gave me a few good laughs at times, and reminded me of the time I was reading through the Amazon.com source code to check on something I thought might have been a bug, to realise that the very bottom line was ““.
Seriously, go check it.
March 5th, 2009 at 9:49 AM
Apologies, I forgot that would HTML comment itself out.
[!– MEOW –]
March 24th, 2009 at 8:27 AM
[…] You can find some information about uncommon HTTP headers here. […]
June 17th, 2009 at 4:08 PM
[…] out this is probably intentional – this post and this message claim that the header is changed by some load-balancing service, overriding the […]
June 18th, 2009 at 12:00 PM
[…] reply that includes a non-descriptive “Server: Server” header and a misspelled “Cneonction: close” header coming from Amazon’s load […]
September 25th, 2009 at 5:45 PM
[…] 这是一个普通的HTTP返回消息，它包含一个非描述性“Server: Server”HTTP头和一个来自Amazon负载均衡器的“Cneonction: close”头，显然拼写有误。 […]
October 12th, 2009 at 10:44 AM
Very interesting read. Thanks! You are right on with the misspelled connection header. LB hardware does it, connection and content-length headers are commonly modified.
November 23rd, 2009 at 12:55 AM
Interesting…
Thanks 🙂
Had lot of fun with headers 😉
December 1st, 2010 at 3:07 PM
Look at what Zappos.com has:
X-Recruiting: If you’re reading this, maybe you should be working at Zappos instead. Check out jobs.zappos.com
LOL…
January 21st, 2011 at 8:55 PM
I know this is old, but I think reddit.com deserves a mention on your list 🙂
Server: ‘; DROP TABLE servertypes; —
January 21st, 2011 at 10:01 PM
Wow, you can literally find the answer to life, the universe and everything in headers!
January 21st, 2011 at 10:19 PM
Thanks for a great article, super nerdy fun, cheers
January 22nd, 2011 at 1:06 AM
[…] In particular, hapax legomena (one-offs) make up over half of the headers found. I expected this. Unfortunately for me, however, a lot of the really interesting stuff is over on that long flat section of the long tail. Which means I spent a lot of time poring over one-offs looking for interesting stuff. Weee. via nextthing.org […]
January 22nd, 2011 at 2:03 AM
re: header job offerings
I’m very sure Google has them. I came across it when working with one of their APIs, considered it for a while, and decided I wasn’t looking for a job at Google. I think it was Analytics, in case you want to work at Google.
January 22nd, 2011 at 3:00 AM
The Norwegian news site vg.no has a link to a YouTube video in its headers.
January 22nd, 2011 at 5:32 AM
Great article. Sounds like you had a lot of fun.
Aristus is partially right about the load balancers and Connection:close, but it’s not limited to netscalers – it’s a feature of any decent LB. The original Connection: close was sent from the web server to the LB. You, the client, have different keepalive settings, because it’s smart to optomize your inner and outer networks differently.
TCP packets all have a checksum. Rather than spending the CPU time of the LB re-calculating the checksums because it wants to delete Connection: close, it’s far easier to just replace it with something jumbled. The end result it that it can process far more packets per second.
January 22nd, 2011 at 5:44 AM
Wow, some real blasts from the past here! I remember when working for various design/development companies headers were one of the most popular ways to prove you were the original author as very few people would think to remove them from the code (opposed to any html/body credits).
January 23rd, 2011 at 1:53 PM
Your colleagues are correct: “Cneonction: close” headers are a load balancer trick to get clients to keep connections open in spite of what the servers behind the LB say.
http://support.citrix.com/article/CTX108955
Why not just remove the header? Mangling it is simpler, faster, and aids in troubleshooting.