Remote JSON - JSONP

原文:Remote JSON - JSONP

 

Thebrowser security model dictates that XMLHttpRequest, frames, etc. musthave the same domain in order to communicate. That's not a terribleidea, for security reasons, but it sure does make distributed (serviceoriented, mash-up, whatever it's called this week) web development suck.

There are traditionally three solutions to solving this problem.

Local proxy:

Needs infrastructure (can't run a serverless client) and you getdouble-taxed on bandwidth and latency (remote - proxy - client).

Flash:

Remote host needs to deploy a crossdomain.xml file, Flash isrelatively proprietary and opaque to use, requires learning a one-offmoving target programming langage.

Script tag:

Difficult to know when the content is available, no standard methodology, can be considered a "security risk".

---

I'm proposing a new technology agnostic standard methodology for the script tag method for cross-domain data fetching: JSON with Padding, or simply JSONP.

The way JSONP works is simple, but requires a little bit of server-side cooperation. Basically, the idea is that you let the client decideon a small chunk of arbitrary text to prepend to the JSON document, andyou wrap it in parentheses to create a valid JavaScript document (andpossibly a valid function call).

The client decides on the arbitrary prepended text by using a queryargument named jsonp with the text to prepend. Simple! With an emptyjsonp argument, the result document is simply JSON wrapped inparentheses.

Let's take the del.icio.us JSON API as an example. This API has a "script tag" variant that looks like this:

http://del.icio.us/feeds/json/bob/mochikit+interpreter:

if(typeof(Delicious) == 'undefined') Delicious = {};
Delicious.posts = [{
    "u": "http://mochikit.com/examples/interpreter/index.html",
    "d": "Interpreter - JavaScript Interactive Interpreter",
    "t": [
        "mochikit","webdev","tool","tools",
        "javascript","interactive","interpreter","repl"
    ]
}]

In terms of JSONP, a document semantically identical to this would be available at the following URL:

http://del.icio.us/feeds/json/bob/mochikit+interpreter?jsonp=if(typeof(Delicious)%3D%3D%27undefined%27)Delicious%3D%7B%7D%3BDelicious.posts%3D

That's not very interesting on its own, but let's say I wanted to benotified when the document is available. I could come up with a littlesystem for tracking them:

var delicious_callbacks = {};
function getDelicious(callback, url) {
    var uid = (new Date()).getTime();
    delicious_callbacks[uid] = function () {
        delete delicious_callbacks[uid];
        callback();
    };
    url += "?jsonp=" + encodeURIComponent("delicious_callbacks[" + uid + "]");
    // add the script tag to the document, cross fingers
};

getDelicious(doSomething, "http://del.icio.us/feeds/json/bob/mochikit+interpreter");

The fetched URL from this hypothetical experiment would look something like this:

http://del.icio.us/feeds/json/bob/mochikit+interpreter?jsonp=delicious_callbacks%5B12345%5D

delicious_callbacks[12345]([{
    "u": "http://mochikit.com/examples/interpreter/index.html",
    "d": "Interpreter - JavaScript Interactive Interpreter",
    "t": [
        "mochikit","webdev","tool","tools",
        "javascript","interactive","interpreter","repl"
    ]
}])

See,because we're wrapping with parentheses, a JSONP request can translateinto a function call or a plain old JSON literal. All the server needsto do differently is prepend a little bit of text to the beginning andwrap the JSON in parentheses!

Now, of course, you'd have libraries like MochiKit, Dojo, etc. abstracting JSONP so that you don't have to write the ugly DOM script tag insertion yourself, etc.

Of course, this just solves the standardization problem. Your pageis still toast if the remote host decides to inject malicious codeinstead of JSON data. However, if implemented, it'd save a lot ofdevelopers some time and allow for generic abstractions, tutorials, anddocumentation to be built.

15 Comments »

1. I implementedsomething similar in Tagneto (the SvrScript.js library). The spec Iused for the implementation allows for an success and failure callbackas well as allowing “multipart” SCRIPT SRC requests when the request isbigger than what will fit in one GET request. Seetagneto.org/how/reference/js/DynamicScriptRequest.html

   Comment by James — 2005-12-12 @ 9:46 pm

2. It looks like Yahoo! agrees with your (sage, IMO) suggestion: http://ws1.inf.scd.yahoo.com/common/json.html#callbackparam

   Mike

   Comment by Mike Shaver — 2005-12-18 @ 9:56 am

3. Hi Bob,

   Great idea! A few days after your post Yahoo made public somethingquite similar to (but maybe less generic than) what you’re proposingabove as part of their Web Services.

   http://www.theurer.cc/blog/2005/12/15/web-services-json-dump-your-proxy/

   = tmk =

   Comment by tmk — 2005-12-18 @ 8:34 pm

4. Thecallback implementation that Yahoo! deployed is *almost* JSONP, but notquite. The difference is that callback filters out anything past thefirst identifier given in callback, so you don’t have the opportunityto throw in an “del.icio.us style” assignment. Originally I had thoughtto do exactly what Yahoo! did, but I didn’t see a reason not to justlet arbitrary text get prepended because it allows for compatibillitywith legacy API, and simplicity in some situations.

   Comment by bob — 2005-12-26 @ 6:28 pm

5. You describe ActionScript (the development language used for the Flash VM as:

   >a one-off moving target programming langage

   Just an fyi, but ActionScript is based on ECMA script. The currentversion is not 100% compatible (although very close). The next versionaims to be 100% compatible with the next version of ECMA script.

   Just an fyi…

   mike chambers

   [e-mail]

   Comment by mike chambers — 2006-01-12 @ 11:09 am

6. I like the idea very much, just added support for it to an RDF store (http://www.appmosphere.com/en-arc_rdf_store).

   cheers,
   bnj

   Comment by Benjamin Nowack — 2006-02-20 @ 9:25 am

7. This is a much needed feature.

   I was looking at ways to integrate my del.icio.us (which has someJSON support) categories in my blogger blog, and I had the exact sameidea.

   Now being a beginner in JS, I have what might be a stupid question :isn’t it possible to build a bridge server that supports JSONP andforward the JSON calls?

   regards,
   nicolas rolland
   http://technofinance.blogspot.com

   Comment by nicolas rolland — 2006-05-26 @ 8:33 am

8. Yes, of course it is possible to build a proxy.

   Comment by bob — 2006-05-26 @ 11:52 am

9. Iam not completely clear about this. How does it stay compatible withexisting APIs? You seem to use it with del.icio.us without any updates.

   And the point of the parentheses is so that the same API can becalled via regular XMLHttpRequest, and the data saved via assignment,or from an offsite page via a script tag, and the data saved via thecallback? I guess the parentheses is the thing I really don’t get.

   Would it not be more safe to have the server call a pre-definedfunction name only? I am thinking about a situation where an attackerembeds some malicious call-back code in a GET url and includes it in animage tag or something like that. Maybe this is no different thanincluding any old offsite .js file.

   Anyway…

   Comment by Evan — 2006-06-07 @ 10:55 am

10. It’s a new API, that del.icio.us does not support. It was a hypothetical example.

    The point of the parentheses is that it’s not valid JavaScriptwithout them, and you need valid JavaScript for a script tag.XMLHttpRequest is irrelevant, because you would use regular JSON forthat.

    I think you’re confused about the security implications. It’sdefinitely not an attack vector for the server because all it does isconcatenate strings. If there’s already malicious code on the client,this is totally irrelevant.

    The only attack vector is stealing cookies from the JSONP-hostingserver, which is why yahoo’s version doesn’t allow anything but asingle identifier to be used as the callback.

    Comment by bob — 2006-06-07 @ 1:47 pm

11. Completelyunmoderated javascript code injection (providing code containing anycharacters) is not a good idea, but what level of restraint would be?There is call some level of callback layout restrictions, and I wouldlike to standardize a good practice, which is also ideally trivial toimplement for JSONP providers.

    Forbidding parentheses goes a long way, but it still allows you tosteal document.cookie by storing it in a variable. Forbidding = tooslightly harms the most trivial JSONP consumption model (callback“my_variable=”). Yahoo seems to have set the bar for themselves onlimiting to US-ASCII alphanumerics, underscore, period and squarebrackets. There is some merit to that; for one thing it skips scenarioswith unbalanced quotation marks in the callback name. The outcome is atleast useful enough.

    I would find it useful if JSONP appeared as an informal standard(IETF RFC, informational, for instance), stating good practices,conformance criteria, and so on, simply to have a document to referimplementors and service providers to, without forcing a high level ofunderstanding of the domain on them. I think I more or less volunteerto writing it, but I would like you to be on board, so to speak, asinventor, originator or similar, even if you do not pull the actualweight.

    It would mostly be something to cover up the hole until Crockford’sJSONRequest is somewhere near as deployed as XMLHttpRequest is today,but that will take some time, and JSONP works now. It could just use alittle help along the way, here and there.

    Comment by Johan Sundström — 2007-02-12 @ 5:54 am

12. Whilethe call back param option is really nice, we need to be really carefulwhile operating of sensitive JSON data for mash-ups. This is exactlywhat caused the gmail contact list XSRF hole. Isn’t it? http://jeremiahgrossman.blogspot.com/2007/01/gmail-xsrf-json-call-back-hackery.html

    Comment by Kishore Senji — 2007-06-04 @ 12:42 pm

13. No, it’s completely different than the XSRF hole in gmail.

    Even if it was, the whole point of JSONP is cross-site requests anyway… so it’s not really forgery nor a hole.

    Comment by bob — 2007-06-04 @ 1:43 pm

14. I have written json file which i want to pass to a javascript function on client side.
    Basically i am trying for callback of the function in my url at client side.
    It is not working can u please help me with the settings for callback.

    Comment by derek — 2008-10-15 @ 5:30 am

15. alert(document.cookie)

    Comment by Sundar — 2009-02-02 @ 8:09 pm