Abandon hope of removing sync XHR from the web platform?

[foolip]

https://xhr.spec.whatwg.org/#sync-warning

Synchronous XMLHttpRequest outside of workers is in the process of being removed from the web platform as it has detrimental effects to the end user's experience. (This is a long process that takes many years.) Developers must not pass false for the async argument when the JavaScript global environment is a document environment. User agents are strongly encouraged to warn about such usage in developer tools and may experiment with throwing an InvalidAccessError exception when it occurs.

There are some Blink use counters for sync XHR:
https://www.chromestatus.com/metrics/feature/timeline/popularity/465 (measure only)
https://www.chromestatus.com/metrics/feature/timeline/popularity/472 (deprecated)
https://www.chromestatus.com/metrics/feature/timeline/popularity/581 (deprecated)
https://www.chromestatus.com/metrics/feature/timeline/popularity/677 (async for comparison)

This does not look at all promising. There's a slight downwards trend, but starting from >2% it's hard to see it dropping to the point where removal is possible.

[foolip]

Some discussion (by me) at https://code.google.com/p/chromium/issues/detail?id=392311#c17

[caitp]

Software developers generally don't like warnings in their terminals (or dev consoles). If they know why they shouldn't use it, they're less likely to ignore it. If warnings show up, their users are slightly more Likely to report issues with their product.

There are good reasons to remove this use pattern from the platform, so folks should not give up on the idea.

[foolip]

At this level of usage, it doesn't seem likely that console warning will be able to drive down usage enough so that sync XHR can be removed. If console warnings is the only tool in our tool box, I think giving up hope is appropriate.

[hallvors]

Probably discussed to death, but if the browser can just keep its UI responsive does it really matter if a script author sometimes makes his/her page freeze by a sync call to an unreliable or slow server?

[glittle]

To me, this is very strange. When I click on a link to go to another page, the browser UI become unresponsive as it goes to get that page.

I have a few cases in my code that I need the browser to wait while it talks to the server using XHR. The user has just clicked a button, and is expecting a short wait.

If I can't do that call synchronously, I'll have to build an artificial approach to freeze the UI until the response is received!

I can live with a "warning" in the developer terminal, but to remove this feature from the spec is going to hurt a lot!

Timeouts can be used to ensure the UI isn't locked for too long!

[marcoscaceres]

@glittle, can you explain how you "need" to pause the browser? Maybe show us some code? It can't be that 98% of other developers don't have this issue, but you do. Can you explain what is so critical in the interaction with the server that the user is forced to not use the browser or the page?

[glittle]

In this case, the call for data is sent down a number of levels, and an answer is expected. If the server call is async, there won't be an answer returned. I can see that that sounds like a bad approach in an async world. I guess I'll have to bite the bullet and redesign each of the levels of code to use async calls, and keep the user entertained while they wait!

[marcoscaceres]

@glittle thanks for the additional details. Best of luck with the refactor! Your users, and local Browser Vendor, will thank you for it :)

In the slim chance you are using ES6, there are ways of keeping your code looking like it's sync, but actually running async (with generators and promises).

[annevk]

@hallvors yes, because the user experience of such an application sucks. Sites need to be motivated to use good practices so they don't become abandonware.

@foolip apart from the desire to remove, anyone using this also should really reconsider. Is the warning really that harmful?

[foolip]

@annevk this is the console warning in Blink with by far the highest usage, around 2%:
https://www.chromestatus.com/metrics/feature/timeline/popularity/581

There's trade-off here, between training web developers to ignore console warnings because they happen too often and for things that aren't actually going to break, and nudging people away from sync XHR even if it will never be removed. 1% would be better than 2%, after all.

What the spec says on the topic (quoted above) goes too far, I think. I don't think sync XHR is "in the process of being removed from the web platform" and no browser will be able to throw InvalidAccessError any time soon. If it were made a bit more tentative like "sync XHR on the main thread is bad for the user experience, browsers please show console warnings and developers please stop using it, if usage is eventually low enough maybe we'll remove it" that would be better.

[annevk]

Given the discussion in https://lists.w3.org/Archives/Public/public-webapps/2014JanMar/thread.html#msg232 I guess we can consider alternative wording. I think the wording was based on showModelDialog()...

[shakyjake]

I don't think we'll see a drop off until support for FormData in Web Workers improves (currently only Chrome and Opera support it). Until then there's no real alternative for developers.

[foolip]

That's a connection I was not aware of, which uses of sync XHR can be replaced by FormData in Web Workers?

[shakyjake]

Well I can only speak from personal experience - I'm currently using sync XHR with FormData for a drag & drop file upload system. Async causes issues with script timeouts on the server when uploading multiple large files so I need to keep it synchronous. Web Workers seemed like the ideal solution for this but the poor support for FormData means I can't use it yet.

[foolip]

I'm not sure I understand, does the server timeout not result in an error event in the async case like it should? Is this to work around an implementation bug, or is there something actually broken in the specs here?

[shakyjake]

No, it's just that using async in this situation causes multiple uploads to start at (roughly) the same time, slowing the time required for each file to finish. Error events all happen (and ar received) as expected.

The spec is fine:)

[foolip]

Oh, so I suppose it's a matter of bookkeeping, using sync XHR saves you the trouble of keeping track of pending uploads and starting them when one is finished?

[mreshadi]

If sync XHR is taken away, what alternative mechanism are you proposing to achieve same effect. We have a javascript-streaming optimization which breaks the js into smaller pieces and loads the missing ones on demand. It has been very effective in reducing the size of js and removing unused parts (http://www.instartlogic.com/products/performance/streaming/javascript-streaming). This optimizations needs to pause the js execution until network response is received and as of now, only sync XHR can achieve this.

Browser is constantly touted as a platform that can replace even OS. In a platform scenario, there will all kinds of people offering different tools / value.
Removal of sync XHR because it slows the page, assumes that the web community only consists of browser gurus and naive web developers who can shoot themselves in the foot!!

If the page is slow, google search ranking will anyways punish the page, why do you want to dumb down the browser for the rest of us?! How does this helps browser to become a true platform, and not a nanny framework for less capable programmers?!

[foolip]

@reshadi, this is a bug about accepting that we cannot remove sync XHR, and having the spec reflect that.

[mreshadi]

@foolip thanks for clarification. I think I miss read it. I'm glad this decision is made! :)

[JamesDunne]

There are several valid and useful patterns that require the use of synchronous XHR for which there is no alternative available. This proposal of deprecating synchronous XHR and the inclusion of a warning in developer console is an unnecessary scare tactic. Yes, I understand the warning is aimed at developers who don't know better with the presumably good intention to incite action aimed at improving the "general" user experience once a page is loaded, but to smack the hands of the capable and take away a powerful tool for which there is no alternative is not an acceptable solution.

[foolip]

@JamesDunne, this is a bug about accepting that we cannot remove sync XHR, and having the spec reflect that.

[TheJaredWilcurt]

Can we get some examples put in place for better practices designed to be as easy and simple to understand as possible, so those baby programmers this warning is meant for can actually update their code accordingly. Meanwhile put a harder to understand message for senior developers to understand that this is a false warning and XHR Sync, like Disco, will never die.

[hallvors]

Isn't this issue handled by promises and other work on making JS itself more "asyncish"? In other words, I would consider an implementation somewhat obnoxious if it warned me against doing something like

new Promise(function(resolve, reject){
    var foo = new XMLHttpRequest();
    foo.open('GET', 'bar.txt', false);
    foo.onload = function(){resolve(foo.responseText)}
    foo.onerror = function(){reject(foo}
    foo.send();
}).then( /* do the work that required waiting for the response */ );

FWIW, I think I'd vote for allowing synchronous APIs in the web platform and solving the problem with browsers being so single-threaded by focusing on that problem.

[hallvors]

(I admit that I have not tested that code to see if browsers implement sync-xhr-in-promise the way I'd expect them to do it ;). Perhaps we should add a test or five in due course?)

[JamesDunne]

@hallvors Promises are orthogonal to the issue here. The foo.send() line in your code above produces blocking behavior, i.e. JavaScript execution will pause at that line until the request is completed, making the rest of the Promise wrapper rather pointless in terms of asynchrony.

[hallvors]

Bugger. Well, thanks for clearing that up. Too bad.

[littlebee]

I have heard the arguments, some religious, and up until a few months ago, I would have agreed that there really isn't a need for async: false. But then in the process of removing the last entanglement requiring server rendering of the base DOM and layout by PHP for the app I work on, I happened upon a very real use case.

During my app's initialization it uses an XMLHttpRequest with async:false to pause all scripts from loading while it fetches application options and current user context information. This information was previously bootstrapped into the page by the php layout and much of the UI code uses information in the context data in static initializers or other ways that assume it is available at time of script load.

I've tried reworking the code to make the api call to get the context information async and then dynamically generate script tags to load the remaining scripts that depend on this data being set. The result was much slower page load times stemming mostly from the browser not caching the scripts loaded via dynamically generated script tags. :(

I sure hope that it's just an idle threat to remove async:false.

BTW, I also think it's just a little pretentious to assume that you understand my user's experience needs better than I do. Just a little.

[ricea]

@littlebee Would it be possible to format the options and context as Javascript, so that you can use a <script> tag to prevent other script from being executed before it has loaded?

[littlebee]

Not sure I'm following you, @ricea. I think maybe the approach you are suggesting is what I did to attempt to make it asynchronous. Keep in mind that there is no server rendered DOM, the initial DOM is delivered from a static index.html file.

The speedy thing about the synchronous solution is you have this in your static index.html:

    <script type="text/javascript"  crossorigin="anonymous" src="/js/dist/applicationCore.js.gz"></script>
    <script type="text/javascript"  crossorigin="anonymous" src="/v7/v7Core.min.js.gz"></script>
    <script type="text/javascript"  crossorigin="anonymous" src="bundle.js.gz"></script>

the browser immediately starts to simultaneously load all 3 js bundles and during eval of the first, applicationCore.js.gz, stops the eval of the remaining scripts by making a single synchronous ajax call to a super fast redis backed api that does the usual session cookie lookup of the user info and provides the same system config data that is bootstrapped into the older pages. There isn't a browser that this doesn't work well on and none have an issue caching all three bundles.

The only way I came up with for making the call asynchronously, without a 1-2 month refactoring effort was to instead do this:

    <script type="text/javascript"  crossorigin="anonymous" src="/js/lib/zukeeper-auth.js"
      data-dependent-scripts="/js/dist/applicationCore.js.gz,/v7/v7Core.min.js,bundle.js"
      data-dependent-scripts-debug="/js/dist/applicationCore.js.gz,/v7/v7Core.js,bundle.js"
    ></script>

First download the auth code that was removed from applicationCore, and then have it inject script tags in the head after it loads:

module.exports = class ZukeeperAuth 
  
  # see README.md in root for doc and options.  returns the context data
  # returned by the /applicationContext api if user has valid session
  @getContext: (options={}) ->
    console.log("zuKeeperAuth: getContext(#{JSON.stringify(arguments)[1...-1]})") if options.verbose
    options.dependentScripts ?= []
    options.dependentScriptsDebug ?= []
    options.verbose = false
    
    httpRequest = new XMLHttpRequest();

    if (!httpRequest) 
      throw new Error('zuKeeperAuth: Cannot create an XMLHTTP instance')

    httpRequest.onreadystatechange = () => @_onReadyStateChange(httpRequest, options);
    url = '/applicationContext/getApplicationContext'

    # this is for the static examples and tests. zombie.js can't handle non http(s)?
    if document.location.origin == "file://"
      url = 'https://events.zulily.com' + url
      
    console.log('calling api ', url) if options.verbose
    httpRequest.open('GET', url);
    httpRequest.send();    
  
  
  @_onReadyStateChange: (httpRequest, options={}) ->
    if httpRequest.readyState == XMLHttpRequest.DONE
      if httpRequest.status == 200
        @_onAuthSuccess(httpRequest, options)
      else 
        @_onAuthError(httpRequest, options)

    
  @_onAuthSuccess: (httpRequest, options={}) ->
    console.log("zuKeeperAuth: successfully authenticated:", httpRequest.responseText) if options.verbose
    context = JSON.parse(httpRequest.responseText)

    window.ZuKeeper ?= {}
    window.ZuKeeper.context = context

    headEl = document.getElementsByTagName("head")[0] || document.documentElement;
    dependentScripts = if context.isDebug then options.dependentScriptsDebug ? options.dependentScripts else options.dependentScripts
    for scriptUrl in dependentScripts
      scriptTagEl = document.createElement('script')
      scriptTagEl.setAttribute('src', scriptUrl)
      scriptTagEl.setAttribute('type', 'text/javascript' );
      # scriptTagEl.setAttribute('crossorigin', 'anonymous') 
      scriptTagEl.async = false

      if scriptUrl == dependentScripts[dependentScripts.length - 1] && options.onSuccess?
        scriptTagEl.onload = options.onSuccess
        
      headEl.appendChild(scriptTagEl)
    
      
  @_onAuthError: (httpRequest, options={}) ->
    console.error("zuKeeperAuth: There was a problem with the request. #{JSON.stringify(httpRequest)}");
    # none of the dependentScripts are loaded anyway so just call directly
    options.onError?(httpRequest, options)

    document.location = "/auth?redir=#{document.location.pathname}"            
  

[ricea]

What I mean is you can do:

<script src="/applicationContext/getApplicationContext"></script>
<script type="text/javascript"  crossorigin="anonymous" src="/js/dist/applicationCore.js.gz"></script>
<script type="text/javascript"  crossorigin="anonymous" src="/v7/v7Core.min.js.gz"></script>
<script type="text/javascript"  crossorigin="anonymous" src="bundle.js.gz"></script>

This requires that getApplicationContext return valid Javascript that sets context itself, but if you control the server this just means prefixing the existing JSON with something like window.context = . The other Javascript files are guaranteed to be executed after it is loaded.

[Z3TA]

I have a module loader (like in Node.JS) that sync loads modules in the browser. Any ideas how I can achieve that without sync xhr ?

function foo() { var bar = require("bar"); ... }

[domenic]

@Z3TA https://github.com/substack/wreq

[Z3TA]

Thanks @domenic. wreq however requires you to change all the script tag's type to "text/wreq". And it parses the script, which might freeze the page longer then a xhr would. It will also require a lot of maintenance to update the parser to handle new JS syntax. And the project itself is not maintained.

[domenic]

JS parsing will freeze the page for much less time than XHR + the browser's JS parsing will.

Regardless, I was pointing you at a proof of concept you could use to adapt your module loader to no longer freeze the page.

[Jemt]

We are working on a product that has JS code dating back 15 years. Not being able to dynamically load these snippets of code and inject it when needed, while holding back DOM processing, would pretty much cost us 300 hours of refactoring. While this is not impossible, I really don't see the value in removing support for sync execution. How is it different from <p>test</p><script src=".."></script><p>more content</p> behaviorally? I'm aware that the browser would be able to optimize by downloading the JS code asynchronously before executing it synchronously alongside the DOM, but the order of execution is the import part here which is what we use sync XMLHttpRequests for.
I'm fine with Developer Tools throwing one warning about the use of Synchronous XMLHttpRequests, but it should not be marked as Deprecated.
Seriously, we are developers, not kids in need of course correction. We can't just rewrite every poorly designed (but functional) snippet of JavaScript out there.

Also, at least give us a time frame - when will it be acceptable for browsers to remove the functionality?

[thw0rted]

I am a web developer, and a web user.

As a web developer -- particularly one who has worked in a big corporate environment, on legacy code, that requires a huge effort to migrate away from bad design decisions made a decade or more ago -- I can absolutely appreciate the value in making the web a "purely additive platform" (per @bedney). Any time we drop an existing feature, we break legacy code, and a lot of that legacy code lives in unseen systems that don't have a proper support tail. Any such decision should be made only after careful consideration.

But. I am also a web user, and as a user, I love to see developers forced to stop using shoddy design patterns that make my experience worse. As a user, I don't care if they're saddled with some architecture decisions somebody made 20 years ago, or a 3rd-party library that their business relationships force them to integrate. Burn it to the ground and do it right. If you honestly believe the browser APIs you have available to you don't allow you to design your application without blocking the UI thread for a network call, get on the WHATWG boards (hello!) and lobby for what you need.

I don't expect sync to go away any time in the next 5-10 years, but I agree with the principle of doing everything we can to discourage it -- yes, even for your use case, that you think is "legit". It may well have been, back when you started your project, but you could do it all asynchronously now. I'm not saying to rewrite everything from scratch today, but you will have to eventually, if not for this breaking change, then for the next one. The web is alive, and any product you make that doesn't have a maintenance plan, that isn't kept up to date with living standards, will become unusable -- not tomorrow, maybe not next year, but eventually it will.

So: I like #178, the idea of site-specific opt-out. I'd like to take it a step further, and encourage browser manufacturers to put sync support behind a config flag, exposed to (power) users. Of course it would default to permissive mode, but maybe in 5+ years I could realistically turn it off. That's how the web gets better -- we deprecate the bad features. Sync XHR was one of them.

[nhatchett]

Removing the ability to perform synchronous AJAX calls will certainly make my life far more difficult. As a business oriented web and desktop designer/developer I often face the challenge of modelling a highly synchronous process into a asynchronous program/web application - this is a problem that has been around for many years and will be around long after I have ceased programming. Examples of this problem are applications that require user interaction and do large database updates, file updates/reads, javascript web apps that maintain state with a server and passing data to other computers on a network, network messaging.

Unfortunately many business processes are synchronous (i.e. step B cannot be performed until step A is complete). Many synchronous processes can be modelled (i.e. programmed) using an event driven system. The major problem with such an approach (i.e. using asynchronous programming to represent a synchronous process) is how you deal with gap between submitting a request and receiving the reply (we could all start mining for bitcoins). In short the user has to wait for the reply regardless of whether the program is synchronous or asynchronous - the main advantage of the asynchronous approach is you can enable the user to abort the process before it is completed (totally unacceptable in many business applications).

The solution to this problem is to ensure that the back-end server, that is servicing the AJAX request, is man enough for the job and the application has been intelligently designed. Removing this facility to cure the problems caused by useless designers and developers is totally stupid (similar to the people causing this problem).

[timoreith]

I am looking for an alternative for sync XHR requests for our SCORM platform. The following code is an example as found in thousands of existing and in use SCORM e-learning modules.

This (simplified) code is called by the content, follows the SCORM API, and can not be changed(!):

Example 1:

var initResult = api.Initialize();
if (initResult) {
	// start SCORM communication
} else {
	// something went wrong 
}

Example 2:

var maxCount = api.getValue('interaction.count');
api.setValue('interaction.' + maxCount + '.foo', 'bar');

Example 3:

var isPassed = api.getValue('passing_score');
if (isPassed > 0.8) {
	// do something...
}

I think you got it ... ;-D

The (SCORM-)api methods "Initialize", "getValue" and "setValue" are implemented by our LMS application and use sync XHR to ensure data integrity.

As mentioned:

the code is part of third party content and can not be changed to use promises, workers or the like.

I would be pleased about suggestions, because currently I see no alternative for sync XHR in this case of SCORM API communication.

[foolip]

@timoreith if "can not be changed" is true then it's game over, no way around it. If usage across the web dropped by enough (unlikely IMHO) and sync XHR was removed from any browser this would be broken forever. (If not, then "can not be changed" was not true.)

[timoreith]

@foolip Thanks for your reply. It could be changed with a new SCORM standard and adapting any existing content and LMS's (like Moodle) to it. This is not likely.
Therefore, removing sync XHR would cause huge problems for a large part of the e-learning industry based on SCORM and its developers, who know why they use sync XHR. No user feels offended by a loading animation that takes a second.

This discussion feels more like demonizing an unloved technology (disliking supportive comments...?). There are indeed applications in which it is justified and its shutdown after years of availability (without an equal alternative) cannot be the solution.

If the hint in the dev console is not just an educational measure for careless developers, official dates about when sync XHR will really be removed would be helpful.

[foolip]

@timoreith, did you come here from the Chrome DevTools warning? I'm wondering if perhaps we should change that to make it very clear no change is planned as of now.

[timoreith]

I've been familiar with that message from Chrome for some time. The message in Edge was more confusing because it sounds like it already doesn't support sync XHR anymore (I can't reproduce it right now).

[thw0rted]

I guess this explains why I've never had a good experience with a SCORM platform, then. The API designers did a bad job, and they should feel bad.

Now, you say it "cannot be changed", but one of the beautiful, terrible things about Javascript is that if you dig deep enough, everything is global (eventually), and you can monkey-patch it. I'm not saying that's the right solution for your case, but I'd say there's an argument to be made. If nothing else, your front-end could talk asynchronously to a Web Worker that calls the (awful) synchronous API in the background. Anything is better than a wait-loop in the UI thread.

[ibash]

Another use case: synchronously reading a file object into memory. We can do this async, but this is a trade off we want to make (not done often, and since the file is local the delay should be limited, etc).

[ricea]

@anubi Use async / await makes this really easy and natural to do asynchronously with fetch(). It may also protect you from overflowing your stack.

But, as you say, I can think of no scenario in which recursive ajax wouldn't be a terrible idea.

[thw0rted]

None of this is hard with promises (as returned by fetch):

function factorial(n) {
  if (n <= 1) { return Promise.resolve(1); }
  return factorial(n-1).then(result => mult(n, result));
}

// Pretend this is a fetch to a REST service that multiplies numbers for some reason
function mult(x, y) { return new Promise(resolve => setTimeout(() => resolve(x * y), 1000)); }

[Nashorn]

Framework designers and architects of API's and DSL languages and tools rely on synchronous XHR requests. Do not remove this feature. Developers are whining like baby-crys because of a little bad experience they had, none.

Here's the truth from a 20+ year engineering veteran. Elite engineers and computer science professionals know how to use thread blocking features like this at the right time, in the right place, (hey isn't that pragmatic?) coupled with intelligent framework hooks and tools for post removal or disabling of these blocking calls.

We leverage xhr sync to load import statements just-in-time in the browser in under 50lines compared to using an over-engineered babel or post build tools. For production buillds, the build tool stitches any import calls so live run-time code never relies on XHR, it will be all baked by then. We're talking framework design here. Not your mom's cup cake shop that you work on in the weekends (to the kid fresh out of college with a BA in computer science) So its a great feature for real-time debugging and quick development (minus the builds on every change).

TRUTH OF THE DECADE:
You see, ECMAScript was infiltrated by lobbyist groups who don't know dynamically typed language concepts at all, probably never heard of the SELF language(Sun Microsystems) or used Smalltalk, just a bunch of young soy boys with wishes about how they want the lang to operate. The committee sold out to massive corporations of employees that instead of making javascript even more dynamic and real-time, is now pushing for strongly typed, strictness and anything the noob developers see as a problem, they lobby for removal. Developers are just that. Engineers build stacks. Developers are complaining about technology like sync xhr that they have no idea when and where to use. Developers only have a faint memory of some bad experience they had when xhr momentarily seized up their browser during an exit beacon api call. NOT our faults you're browsing some xxx website and/or torrent, ad-spamming trash sites.

Even Mozilla, Chrome.....sold out. The Rhino animal was the mascot given to JavaScript, it came from the Rhino book (https://www.amazon.com/JavaScript-Definitive-Guide-Activate-Guides/dp/0596805527). The creature itself is tough, resilient, silent and deadly, armored, is the unicorn, surviving ages, is rugged and built to last.

But at the rate ECMAscript is changing to a soy-boy language.....i think it's being hunted by poachers for it's ivory, the beauty of the language.

[ghost]

@Nashorn wow, well said.

I have used xhr sync extensively in non-production, framework, code, and it has been incredibly useful. I think usage numbers are often equated to usefulness; this is a mistake in that some features--such as xhr sync--are more likely to be used in "not your mom's cup cake shop" code that never goes to production, so of course usage numbers would be low.

If xhr sync were to be removed, an option would need to be retained in the browser to enable it. Maybe make harder to do, but do not remove it.

As an aside, I dislike the mob/totalitarian mentality that has taken over web technology, i.e., "if we (your average unimaginative and inexperienced web dev groupie) think x should not be done, we will not be content till x cannot be done"...as @Nashorn said, their concern is about the ivory.