Foreign Fetch for Service Workers

[Gozala]

I have being lately exploring service workers to see if they could address some of the limitations in beaker space beakerbrowser/specs#21 which lead me to realizing that service workers may potentially unlock some of the other (than Web Extension) avenues for integrating IPFS / Dat / SSB / ... into web ecosystem, which I though I'd share here. I'll use IPFS for describing this although it should be applicable to most others just as well.

• As far as I know apps using IPFS essentially embedding whole IPFS node implementation, which is both adds overhead and also seems counter to how browser would have exposed this if it shipped implementation of the protocol. I think it would make a lot more sense to have one node per browser instance that all apps / tabs talk to.
• In theory service workers are great fit for that. http://ipfs.io/ could register service worker which would act as local IPFS node (using whatever transports are available) and allow apps to read / write content to the network.
• In practice Service Workers only serving requests from the same origin which makes it fairly impractical as apps in practice are unable to talk to it. Worth noting that in the past handling of requests from other origins were part of the spec in the past which had being removed due to reasons described here Remove foreign fetch w3c/ServiceWorker#1188
• I'm thinking we should try to make case for reintroducing foreign fetch by communicating use cases that it would unlock + finding a way to propose solution that avoids double-keying issue, which in my understanding is to prevent tracking.

I believe that this has huge potential as it would:

1. Reduce overhead associated duplicated nodes.
2. Could remove dependency on ipfs.js library obsolete by exposing IPFS node interface through REST API from the service worker.
3. Could enable local apps to collaborate on IPFS data even offline.
4. Would enable more effective threat mitigation system for libdweb, specifically exposing additional capabilities on specific origin is far less risky than allowing it for all origins.
5. It may even render protocol handler API of libdweb somewhat obsolete. Sure ipfs://${hash}/ is looks nicer than https://ipfs.io/ipfs/${hash}/ but later would require no extension and other browsers could be pursued to adopt this far easier IMO.

It is also worth noting that "foreign fetch" limitation can be overcome to some degree today by using proxy iframe say <iframe src="https://ipfs.io/node" /> that could proxy all the requests received on window.onmessage to a service worker and proxy responses back to event.source. It would no be able to:

1. Enable embedding of images or other resources into other origins
2. And I suspect iframe may fail to load if browser is offline, but maybe browser cache or legacy app cache could be used to overcome it.

Dose this sound interesting ? Would you consider this valuable ? Would you be interested in exploring this ?

[Gozala]

I was also suggested to read through these when talking to people involved in standards bodies
https://www.mnot.net/blog/2015/08/18/distributed_http
https://webkit.org/blog/7675/intelligent-tracking-prevention/

[Gozala]

Another thing I'll be attempting to figure out is what the status of this spec https://w3c.github.io/webappsec-suborigins/ ? I think given the raise in popularity of IPFS there is a chance to advocated for getting it in browsers. What's position with-in Mozilla is for this work

[Gozala]

Alright I have spend quite some time discussing double-keying & foreign-fetch with people who work on this in Firefox and I think I have better understanding now. Bad news is it's unlikely we'll be able to succeed in making foreign-fetch comeback!

Here are some key takeaways:

• Browser are starting to fight trackers by putting more boundaries. Which is what double keying is about and it affects huge surface. What it means in practice is instead of just using origin browsers are starting to use origin+top.origin for all the things origin was used. e.g. iframe foo.io embeded in a.io document will have local storage keyed to a.io+foo.io while same iframe in b.io will have local storage keyed to b.io+foo.io and if foo.io is loaded in top level it will be just foo.io.
• What that means that even if foreign-fetch was enabled a.io requesting foo.io and b.io requesting same foo.io will result in two different service worker instances that have access to two different storages / caches.

I was trying to make an argument for allowing sharing same SW instead, but that seems to raise different concerns:

• Today SW is active as long as corresponding site is loaded in the tab. If you allow any site to activate a SW that would result in far more many SW instances being active and little to no way for a user to see / act on that.

  I think that is fare argument. Today user can close tab and be done, making SW lifetime independent of it's activator would have impact and complicate UX. If we were to pursue this we need a compelling story around that.

• There is some concerns how this could be abused by crypto-mining SW which could be installed once but than being reactivated by every site you visit using same patterns used by trackers today.

  I'm not entirely convinced as bad actors could just as easily register some SW instead.

• What if site is loading 100s or resources and each has corresponding SW do we spawn 100 threads that not something browsers would be able to handle. And chances are high that SW could get just as abused as notification permission requests are today.

  Reasonable concern, although probably could be addressed by limiting to n number of SW and forwarding rest to server. If demand is accounted for prioritization into which SW to keep active it's likely work ok in practice.

• Quoting

  Much of the rationale thus far has just been that all browsers have been struggling to implement and optimize the core of ServiceWorkers itself.

I also got some suggestions along the way

• Writable-files spec https://github.com/WICG/writable-files has being getting some traction. Maybe there could be something like share targeted at IPFS.

  Have not dived into the spec to tell if that's relevant or helpful for use cases here.

• Use of "iframe + SW" as I described earlier was also proposed. I was also pointed out that "embedding limitation" I was concerned with could be addressed in few different ways:

  • Embedding images or other self contained files isn't a huge deal you could obtain ArrayBuffer for it and then make blob url to embed.
  • For things like CSS / JS that imports other stuff, little more tricky but doable sever steps:
    1. App e.g. peerpad.net can register own service worker.
    2. App can embed hidden iframe for ipfs.io which registers IPFS node as service worker.
    3. App obtains MessagePort though iframe from ipfs.io service worker and transfers it to own service worker.
    4. Now two service workers have message channel for direct exchange which allows peerpad.net to serve any content from IPFS through it's SW by forwarding data from message channel.

[Gozala]

I think exploring iframe + sw suggestion is worth exploring, even if we choose to pursue making a case for foreign-fetch as it would be educational and would a useful polyfill even if foreign-fetch was to be added.

[Gozala]

Linking this issue here mozilla/standards-positions#31

I was also told that spec has a number of unsolved problems and nobody is driving it.

[Gozala]

* Maybe there could be something like share targeted at IPFS.

I think this was in reference to following spec:
https://wicg.github.io/web-share/

[lidel]

@Gozala thank you for looking into this and providing highly insightful explanation for removal of foreign-fetch. Quick feedback from IPFS front:

• We will explore iframe + sw in Origin isolation warnings, tests and enforcement ipfs/service-worker-gateway#30 (as a way of reducing overhead associated with duplicated nodes). Need to check if double keying impacts this use case.
• Suborigins would provide an elegant fix (Suborigins  ipfs/in-web-browsers#66). The HTTP Gateway provided by go-ipfs already implements the Suborigin header (based on a spec draft). Unfortunately no vendor supports it, and the only viable option today is use of subdomains.
• web-share and writable-files look interesting, will follow them

[daviddias]

I've learned today through Master of Web Puppets: Abusing Web Browsers for Persistent and Stealthy Computation that a Service Worker will shutdown (or freeze) automatically after ~1 minute passes since the user left the Website.

The main reason for this behaviour is due to security as it is described by 1 - Chrome and 2 -Firefox, why this is relevant for the Service Worker discussion is that it means that unless the user keeps the tab open where the Service Worker was installed (e.g. https://js.ipfs.io), the Service Worker will be automatically shutdown after one minute, meaning it can't be used as a Service Worker Gateway by other apps, even if foreign fetch was an implemented feature.

Using an iframe might be a work around for this showstopper as well, however, it does mean that the Website builders will have to have an iframe loaded in their Website html if they want their ipfs.io Gateway urls to it a Service Worker Gateway or if they want their DApp to use the Node inside the Service Worker.

[Gozala]

I've learned today through Master of Web Puppets: Abusing Web Browsers for Persistent and Stealthy Computation that a Service Worker will shutdown (or freeze) automatically after ~1 minute passes since the user left the Website.

The main reason for this behaviour is due to security as it is described by 1 - Chrome and 2 -Firefox, why this is relevant for the Service Worker discussion is that it means that unless the user keeps the tab open where the Service Worker was installed (e.g. https://js.ipfs.io), the Service Worker will be automatically shutdown after one minute, meaning it can't be used as a Service Worker Gateway by other apps, even if foreign fetch was an implemented feature.

It's not necessarily where service worker was install but rather any tab where service worker is activated meaning page with a matching origin that was loaded after SW is registered.

@daviddias that is true, however foreign-fetch was supposed to allow other origins to activate the service worker, implying that different origins could have activated it.

I also would like to point out that it's less about page / tab being open and more about SW serving requests. SW will be deactivated even when activating tab remains open unless SW is handling message or fetch event.

Using an iframe might be a work around for this showstopper as well, however, it does mean that the Website builders will have to have an iframe loaded in their Website html if they want their ipfs.io Gateway urls to it a Service Worker Gateway or if they want their DApp to use the Node inside the Service Worker.

In lunet I end up using with a similar setup, except via SharedWorker that is polyfilled on Safari with a ServiceWorker that iframe keeps alive.

[rektide]

What that means that even if foreign-fetch was enabled a.io requesting foo.io and b.io requesting same foo.io will result in two different service worker instances that have access to two different storages / caches.

I would still like for Foreign Fetch for SW to exist, even if they have different service workers as per http partitioning.

Being able to issue http requests to an origin & get a reply while offline is an extremely interesting capability for the web. Even if caches and data sharing are hobbled from the extensive de-web-ification of all these new tracking-protection layers being thrown against the web.

Not as useful as I want it to be, but I continue to believe, very strongly, that Foreign Fetch is the most elegant & yet powerful way to empower a fully featured, competent offline web experience. Without Foreign Fetch, we have to rely on odious backchannel means of passing data around, whereas with Foreign Fetch, http can remain the transport protocol of choice for connecting the web. Anyhow, thank you ever so much @Gozala for this wonderful issue & your fine work bringing in information on it!!! I am so impressed & so thankful & so surprised there's such a great source of information on this amazing topic.