To help fight the opioid crisis, a new tool from Maps and Search

To help fight the opioid crisis, a new tool from Maps and Search

https://meson.in/2Xi5s49

In 2017, the Department of Health and Human Services (HHS) declared the opioid crisis a public health emergency, with over 130 Americans dying every day from opioid-related drug overdoses.  Last month, we saw that search queries for “medication disposal near me” reached an all-time high on Google.

opioids_data

53 percent of prescription drug abuse starts with drugs obtained from family or friends, so we’re working alongside government agencies and nonprofit organizations to help people safely remove excess or unused opioids from their medicine cabinets. Last year, we partnered with the U.S. Drug Enforcement Administration (DEA) for National Prescription Take Back Day by developing a Google Maps API  locator tool to help people dispose of their prescription drugs at temporary locations twice a year. With the help of this tool, the DEA and its local partners collected a record 1.85 million pounds of unused prescription drugs in 2018.

Today, we’re making it easier for Americans to quickly find disposal locations on Google Maps and Search all year round. A search for queries like “drug drop off near me” or “medication disposal near me” will display permanent disposal locations at your local pharmacy, hospital or government building so you can quickly and safely discard your unneeded medication.

opioid_gif

This pilot has been made possible thanks to the hard work of many federal agencies, states and pharmacies. Companies like Walgreens and CVS Health, along with state governments in Alabama, Arizona, Colorado, Iowa, Massachusetts, Michigan and Pennsylvania have been instrumental in this project, contributing data with extensive lists of public and private disposal locations. The DEA is already working with us to provide additional location data to expand the pilot.

For this pilot, we also looked to public health authorities—like HHS—for ideas on how technology can help communities respond to the opioid crisis. In fact, combining disposal location data from different sources was inspired by a winning entry at the HHS’s Opioid Code-A-Thon held a year ago.

We’ll be working to expand coverage and add more locations in the coming months. To learn more about how your state or business can bring more disposal locations to Google Maps and Search, contact RXdisposal-data@google.com today.

Product.google

via The Official Google Blog https://meson.in/2HntRgD

February 22, 2019 at 02:14AM

Allegations Against the Maker of OxyContin Are Piling Up. Here’s What They Could Mean for the Billionaire Family Behind Purdue Pharma

Allegations Against the Maker of OxyContin Are Piling Up. Here’s What They Could Mean for the Billionaire Family Behind Purdue Pharma

https://meson.in/2GETwEG

Executives from Purdue Pharma, the manufacturer of the powerful opioid painkiller OxyContin, admitted in federal court in 2007 that Purdue’s marketing practices and interactions with doctors had understated the strength and addictive potential of the drug — an omission that many experts believe contributed to an opioid epidemic that claimed nearly 50,000 American lives in 2017 alone.

But on Thursday, the release of a previously sealed deposition from 2015 showed that Purdue executives knew of OxyContin’s strength long before that $600 million settlement. The deposition, which had been filed in court, revealed that Dr. Richard Sackler — part of the family that founded and controls Purdue, and who has served as Purdue’s president and co-chairman of the board — knew as early as 1997 that OxyContin was much stronger than morphine, but chose not to share that knowledge with doctors.

“We are well aware of the view held by many physicians that oxycodone [the active ingredient in OxyContin] is weaker than morphine. I do not plan to do anything about that,” Purdue’s head of sales and marketing, Michael Friedman, wrote in an email to Sackler, according to the deposition, which was obtained by ProPublica and co-published with STAT. “I agree with you,” Sackler wrote back. “Is there a general agreement, or are there some holdouts?”

The document’s publication comes just weeks after the release of an unredacted 277-page lawsuit filed against Purdue by Massachusetts Attorney General Maura Healey — itself just one of thousands of legal complaints brought against Purdue and other pharmaceutical companies by plaintiffs across the country, many of which have been rolled into one multi-district litigation in Ohio federal court. And as the evidence mounts, legal experts say Purdue could face serious consequences, from astronomical fines to injunctions that could threaten its ability to do business.

“One theme that clearly emerges from this deposition, brick by brick, is the foundation that is laid, that shows how even after this guilty plea there was a shocking lack of care for people that were at risk of abusing this drug and instead a singular focus on profit,” says Joseph Khan, a Philadelphia-based attorney who is currently bringing suits against corporations involved in the opioid epidemic.

As the New York Times reported, parts of Sackler’s deposition are in conflict with his previous testimony. For example, a 2006 Department of Justice report suggested he knew in 1999 that users in internet chatrooms were discussing abuse of the drug. In the deposition, however, Sackler said he first learned of its street value in a 2000 Maine newspaper article.

In a statement provided to TIME, Purdue said the “intentional leak of the deposition is a clear violation of the court’s order and, as such, is regrettable.” The statement adds that, “Dr. Sackler described Purdue’s efforts to adhere to all relevant laws and regulations and to appropriately reflect OxyContin’s risks of abuse and addiction as the science of opioid pain therapy evolved over time.”

Much of the material included in the deposition pertains to activity carried out before the company’s 2007 settlement, while Healey’s suit relates to post-2007 behavior. But Khan says the ramifications of the document are still relevant today, given the judgements Purdue could face from juries.

“There are straight contradictions between what’s in here and what the Department of Justice has put together. This is not something that will play well in front of a jury,” Khan says. “They don’t have as much leverage as they might want.”

The Massachusetts complaint also includes dramatic accusations about how much Purdue executives knew about their blockbuster drug, and when they knew it.

According to lawsuit, members of the Sackler family and other Purdue executives purposefully downplayed the addictive properties of OxyContin, and promoted sales tactics meant to encourage doctors to prescribe as much OxyContin, in the highest doses and longest durations, as possible — despite the potential risks for abuse, and despite the terms of Purdue’s prior settlement with the federal government. The suit also details Purdue’s plans to sell addiction treatments, helping them dominate “the pain and addiction spectrum.” Purdue’s board, controlled by the Sacklers, also voted to pay out $4 billion to the family between 2007 and 2018, the documents show.

In a statement provided to TIME, a Purdue representative said the attorney general’s office “seeks to publicly vilify Purdue, its executives, employees and directors by taking out of context snippets from tens of millions of documents and grossly distorting their meaning. The complaint is riddled with demonstrably inaccurate allegations,” they said, and “offers little evidence to support its sweeping legal claims.” Purdue fought to keep portions of the suit from being released publicly.

If successful, Massachusetts’ lawsuit could force Purdue to pay not only significant fines, but also require the company to cease certain behaviors and make efforts to remedy the damages it has allegedly caused, Khan says.

“If you think about what would restitution look like, these are staggering, almost incalculable costs,” Khan says. But the problem goes beyond money. “What would it mean to stop this epidemic they’re accused of putting into place?” he asks. “You’re not going to find anyone who knows anything about the opioid epidemic who will just say you can solve this problem overnight with a quick fix.”

Further complicating matters, Purdue’s future hinges on far more than a single lawsuit.

John Jacobi, a professor of health law and policy at Seton Hall Law School, called the Massachusetts complaint “extraordinary in the length and depth of the allegations against individual defendants,” but says it is “more or less consistent” with the roughly 1,200 complaints included in the Ohio MDL, as well as the hundreds of others individually making their way through state court systems.

And for that reason, Jacobi says, Purdue could be facing consequences much larger than those included in Healey’s complaint. Opioid manufacturers could face a situation similar to the 1998 Master Settlement Agreement with Big Tobacco, which forced five major manufacturers to pay out billions of dollars over cigarette marketing and promotional practices. (Mike Moore, the lawyer who orchestrated the Master Settlement Agreement, is now bringing a new suit against opioid distributors and manufacturers. He was not immediately available for comment to TIME.)

“Many people have suggested that the only way out of the thicket that all of these litigants find themselves in would be some sort of global settlement similar to what was achieved in the tobacco litigation, and I don’t think that’s a far-fetched suggestion,” Jacobi says. “All of those, at some point, will be gathered up and resolved.”

Khan agrees that the volume of lawsuits in the MDL could hold a major threat for opioid manufacturers. And the results of MDL cases set for trial later this year will likely set the tone for other individual suits, like Healey’s, filed around the country, he says.

“There becomes a point at which it becomes mathematically impossible for every one of those plaintiffs to receive what they’re seeking,” Khan says. “Some of these companies are not going to be equipped to survive. Purdue may or may not be differently situated.”

Bio.medical

via Healthland https://meson.in/2DLLW54

February 23, 2019 at 07:32AM

Encrypting DNS end-to-end

Encrypting DNS end-to-end

https://meson.in/2EUQnPb


Over the past few months, we have been running a pilot with Facebook to test the feasibility of securing the connection between 1.1.1.1 and Facebook’s authoritative name servers. Traditionally, the connection between a resolver and an authoritative name server is unencrypted i.e. over UDP.

In this pilot we tested how an encrypted connection using TLS impacts the end-to-end latency between 1.1.1.1 and Facebook’s authoritative name servers. Even though the initial connection adds some latency, the overhead is amortized over many queries. The resulting DNS latency between 1.1.1.1 and Facebook’s authoritative name servers is on par with the average UDP connections.

To learn more about how the pilot went, and to see more detailed results, check out the complete breakdown over on Code, Facebook’s Engineering blog.

Product.platform

via The Cloudflare Blog https://meson.in/2DaAAwa

December 22, 2018 at 01:02AM

Athenian Project Turns One: Are Election Websites Safer?

Athenian Project Turns One: Are Election Websites Safer?

https://meson.in/2ETIUAI


One year ago, Cloudflare launched the Athenian Project to provide free Enterprise-level service to election and voter registration websites run by state and local governments in the United States. Through this project, we have helped over 100 entities in 24 states protect their websites from denial of service attacks, SQL injection, and other malicious efforts aimed at undermining the integrity of their elections. With the end of the year approaching, and the November 6th US midterm elections behind us, we wanted to look back at the project and what we have learned as we move towards 2020.

US Midterm Election Day

The morning of November 6th was full of anticipation for the Athenian Project team with the policy, engineering and support teams ready as polls opened in the East. Early in the day, we were notified by our partner at the CDT that some elections websites were experiencing downtime. Mobilizing to help these groups, we reached out to the website administrators and, through the course of the day, on-boarded over 30 new county-level websites to the Athenian Project and helped them manage the unpredictably large amounts of legitimate traffic.

This last-minute effort would not have been possible without the help of the CDT and all of the other organizations working to maintain election integrity. Each organization brings their own strengths, and it took everyone working together, as well as preparation and diligence on the part of election officials, to make election day a success.

I Voted Stickers— Creative Commons Attribution Element5 Digital on Pexels

Civic Engagement Online

In looking at the aggregated election day data, the biggest story is one of engagement. In the month leading up to the November election, voter registration and election websites on the Athenian Project received nearly three times the number of requests as in September or any other month preceding it. Athenian Project websites received more requests in just the first seven days of November than in any other month except October.

When we first started the Athenian Project, we expected denial of service and other attacks to be the driving concern. However, we soon found that many state and local election websites experience large fluctuations in legitimate traffic on election day, especially in the event of a contested election, and appreciated having a CDN to help manage these events. As can be seen below, traffic levels, already higher than usual on election day, at times suddenly spiked to four times above the day’s average for certain websites.

Requests to Athenian Project websites on 11/6/18

Keeping a Lookout for Bad Actors

We are happy to report that we didn’t see any evidence of a coordinated set of attacks across the election websites on our service. There were, however, a variety of attacks stopped by rules within our Web Application Firewall (WAF). The prevented attacks included scans by malicious bots impersonating helpful bots. These scans enable malicious actors to check for vulnerabilities to exploit, and were stopped using fake user-agent rules which can identify the malicious bot’s attempt to spoof its identity. The WAF also stopped a variety of cross-site scripting attempts, forced login attempts, and SQL injection attacks aimed at gaining access to databases. The attacks appear to have been Internet-wide attacks targeting specific known vulnerabilities rather than election website specific attacks. This finding re-enforces our belief that improving cybersecurity is vital for everyone on the Internet every day, not just in response to large events.

Where We’re Going in 2019

Moving forward, we are hoping to continue improving the reach of the project. One year is a relatively short time, especially when considering code freezes around both the primaries and general elections, and we hope to continue education efforts and on-boardings in advance of the 2020 elections. One item we noticed was that, despite making it easy to obtain SSL certificates and use TLS on Cloudflare, not all of the requests to Athenian Project websites are encrypted. This happens either as a result of misconfiguration, or because Universal SSL has been disabled for the site and no non-Cloudflare certificates have been uploaded. As a result, we will strive to do a better job of encouraging SSL adoption and educating website administrators about the importance of encryption.

US Capital Building— Creative Commons Attribution on Pixabay

We would like to thank election officials and administrators across the country for their hard work in maintaining the integrity of our midterm elections. Election cybersecurity was not a story, and that is a testament to the commitment of these individuals.

With the midterm elections over, the Cloudflare Athenian Project team is setting our sights on 2020 and any special elections which may come before then as well as looking at opportunities to expand the Athenian Project into new areas. If you run a state or local election website and are interested in the Athenian Project, feel free to reach out through our web form at cloudflare.com/athenian.

Product.platform

via The Cloudflare Blog https://meson.in/2DaAAwa

December 22, 2018 at 04:06AM

Improving request debugging in Cloudflare Workers

Improving request debugging in Cloudflare Workers

https://meson.in/2EZPuGb


At Cloudflare, we are constantly looking into ways to improve development experience for Workers and make it the most convenient platform for writing serverless code.

As some of you might have already noticed either from our public release notes, on cloudflareworkers.com or in your Cloudflare Workers dashboard, there recently was a small but important change in the look of the inspector.

But before we go into figuring out what it is, let’s take a look at our standard example on cloudflareworkers.com:

The example worker code featured here acts as a transparent proxy, while printing requests / responses to the console.

Commonly, when debugging Workers, all you could see from the client-side devtools is the interaction between your browser and the Cloudflare Worker runtime. However, like in most other server-side runtimes, the interaction between your code and the actual origin has been hidden.

This is where console.log comes in. Although not the most convenient, printing random things out is a fairly popular debugging technique.

Unfortunately, its default output doesn’t help much with debugging network interactions. If you try to expand either of request or response objects, all you can see is just a bunch of lazy accessors:

You could expand them one-by-one, getting some properties back, but, when it comes to important parts like headers, that doesn’t help much either:

So, since the launch of Workers, what we have been able to suggest instead is certain JS tricks to convert headers to a more readable format:

This works somewhat better, but doesn’t scale well, especially if you’re trying to debug complex interactions between various requests on a page and subrequests coming from a worker. So we thought: how can we do better?

If you’re familiar with Chrome DevTools, you might have noticed before that we were already offering its trimmed-down version in our UI with basic Console and Sources panels. The obvious solution is: why not expose the existing Network panel in addition to these? And we did just* that.

* Unfortunately, this is easier said than done. If you’re already faimilar with the Network tab and are interested in the technical implementation details, feel free to skip the next section.

What can you do with the new panel?

You should be able to use most of the things available in regular Chrome DevTools Network panel, but instead of inspecting the interaction between browser and Cloudflare (which is as much as browser devtools can give you), you are now able to peek into the interaction between your Worker and the origin as well.

This means you’re able to view request and response headers, including both those internal to your worker and the ones provided by Cloudflare:

Check the original response to verify content modifications:

Same goes for raw responses:

You can also check the time it took worker to reach and get data from your website:

However, note that timings from a debugging service will be different than the ones in production in different locations, so it would make sense to compare these only with other requests on the same page or with the same request as you keep iterating on code of your Worker.

You can view the initiator of each request – this might come in handy if your worker contains complex routing handled by different paths, or if you want to simply check which requests on the page were intercepted and re-issued at all:

Basic features like filtering by type of content also work:

And, finally, you can copy or even export subrequests as HAR for further inspection:

How did we do this?

So far we have been using a built-in mode of the inspector which was specifically designed with JavaScript-only targets in mind. This allows it to avoid loading most of the components that would require a real browser (Chromium-based) backend, and instead leaves just the core that can be integrated directly with V8 in any embedder, whether it’s Node.js or, in our case, Cloudflare Workers.

Luckily, the DevTools Protocol itself is pretty well documented – chromedevtools.github.io/devtools-protocol/ – to facilitate third-party implementors.

While this is commonly used from client-side (for editor integration), there are some third-party implementors of the server-side too, even for non-JavaScript targets like Lua, Go, ClojureScript and even system-wide network debugging both on desktop and mobile: github.com/ChromeDevTools/awesome-chrome-devtools.

So there is nothing preventing us from providing our own implementation of Network domain that would give a native DevTools experience.

On the Workers backend side, we are already in charge of the network stack, which means we have access to all the necessary information to report and can wrap all the request/response handlers into own hooks to send it back to the inspector.

Communication between the inspector and the debugger backend is happening over WebSockets. So far we’ve been just receiving messages and passing them pretty much directly to V8 as-is. However, if we want to handle Network messages ourselves, that’s not going to work anymore and we need to actually parse the messages.

To do that in a standard way, V8 provides some build scripts to generate protocol handlers for any given list of domains. While these are used by Chromium, they require quite a bit of configuration and custom glue for different levels of message serialisation, deserialisation and error handling.

On the other hand, the protocol used for communication is essentially just JSON-RPC, and capnproto, which we’re already using in other places behind the scenes, provides JSON (de)serialisation support, so it’s easier to reuse it rather than build a separate glue layer for V8.

For example, to provide bindings for Runtime.callFrame we need to just define a capnp structure like this:

struct CallFrame {
  # Stack entry for runtime errors and assertions.
  functionName @0 :Text; # JavaScript function name.
  scriptId @1 :ScriptId; # JavaScript script id.
  url @2 :Text; # JavaScript script name or url.
  lineNumber @3 :Int32; # JavaScript script line number (0-based).
  columnNumber @4 :Int32; # JavaScript script column number (0-based).
}

Okay, so by combining these two we can now parse and handle supported Network inspector messages ourselves and pass the rest through to V8 as usual.

Now, we needed to make some changes to the frontend. Wait, you might ask, wasn’t the entire point of these changes to speak the same protocol as frontend already does? That’s true, but there are other challenges.

First of all, because Network tab was designed to be used in a browser, it relies on various components that are actually irrelevant to us and, if pulled in as-is, would not only make frontend code larger, but also require extra backend support too. Some of them are used for cross-tab integration (e.g. with Profiler), but some are part of the Network tab itself – for example, it doesn’t make much sense to use request blocking or mobile throttling when debugging server-side code. So we had some manual untangling to do here.

Another interesting challenge was in handling response bodies. Normally, when you click on a request in Network tab in the browser, and then ask to see its response body, devtools frontend sends a Network.getResponseBody message to the browser backend and then the browser sends it back.

What this means is that, as long as the Network tab is active, browser has to store all of the responses for all of the requests from the page in memory, not knowing which of them are actually going to be requested in the future or not. Such lazy handling makes perfect sense for local or even remote Chrome debugging, where you are commonly fully in charge of both sides.

However, for us it wouldn’t be ideal to have to store all of these responses from all of the users in memory on the debugging backend. After some forth and back on different solutions, we decided to deviate from the protocol and instead send original response bodies to the inspector frontend as they come through, and let frontend store them instead. This might seem not ideal either due to sending unnecessary data over the network during debugging sessions, but these tradeoffs make more sense for a shared debugging backend.

There were various smaller challenges and bug fixes to be made and upstreamed, but let them stay behind the scenes.

Is this feature useful to you? What other features would help you to debug and develop workers more efficiently? Or maybe you would like to work on Workers and tooling yourself?

Let us know!

P.S.: If you’re looking for a fun personal project for the holidays, this could be your chance to try out Workers, and play around with our new tools.

Product.platform

via The Cloudflare Blog https://meson.in/2DaAAwa

December 28, 2018 at 11:23PM

Real URLs for AMP Cached Content Using Cloudflare Workers

Real URLs for AMP Cached Content Using Cloudflare Workers

https://meson.in/2QjUmen


Today, we’re excited to announce our solution for arguably the biggest issue affecting Accelerated Mobile Pages (AMP): the inability to use real origin URLs when serving AMP-cached content. To allow AMP caches to serve content under its origin URL, we implemented HTTP signed exchanges, which extend authenticity and integrity to content cached and served on behalf of a publisher. This logic lives on Cloudflare Workers, meaning that adding HTTP signed exchanges to your content is just a simple Workers application away. Publishers on Cloudflare can now take advantage of AMP performance and have AMP caches serve content with their origin URLs. We’re thrilled to use Workers as a core component of this solution.

HTTP signed exchanges are a crucial component of the emerging Web Packaging standard, a set of protocols used to package websites for distribution through optimized delivery systems like Google AMP. This announcement comes just in time for Chrome Dev Summit 2018, where our colleague Rustam Lalkaka spoke about our efforts to advance the Web Packaging standard.

What is Web Packaging and Why Does it Matter?

You may already see the need for Web Packaging on a daily basis. On your smartphone, perhaps you’ve searched for Christmas greens, visited 1-800-Flowers directly from Google, and have been surprised to see content served under the URL https://google.com/amp/1800flowers.com/blog/flower-facts/types-of-christmas-greens/amp. This is an instance of AMP in action, where Google serves cached content so your desired web page loads faster.

Visiting 1-800 Flowers through AMP without HTTP signed exchange

Google cannot serve cached content under publisher URLs for clear security reasons. To securely present content from a URL, a TLS certificate for its domain is required. Google cannot provide 1-800-Flowers’ certificate on the vendor’s behalf, because it does not have the corresponding private key. Additionally, Google cannot, and should not be able to, sign content using the private key that corresponds to 1-800-Flowers’ certificate.

The inability to use original content URLs with AMP posed some serious issues. First, the google.com/amp URL prefix can strip URLs of their meaning. To the frustration of publishers, their content is no longer directly attributed to them by a URL (let alone a certificate). The publisher can no longer prove the integrity and authenticity of content served on their behalf.

Second, for web browsers the lack of a publisher’s URL can call the integrity and authenticity of a cached webpage into question. Namely, there’s no clear way to prove that this response is a cached version of an actual page published by 1-800-Flowers. Additionally, cookies are managed by third-party providers like Google instead of the publisher.

Enter Web Packaging, a collection of specifications for “packaging” website content with information like certificates and their validity. The HTTP signed exchanges specification allows third-party caches to cache and service HTTPS requests with proof of integrity and authenticity.

HTTP Signed Exchanges: Extending Trust with Cryptography

In the pre-AMP days, people expected to find a webpage’s content at one definitive URL. The publisher, who owns the domain of the definitive URL, would present a visitor with a certificate that corresponds to this domain and contains a public key.

The publisher would use the corresponding private key to sign a cryptographic handshake, which is used to derive shared symmetric keys that are used to encrypt the content and protect its integrity.

The visitor would then receive content encrypted and signed by the shared key.

The visitor’s browser then uses the shared key to verify the response’s signature and, in turn, the authenticity and integrity of the content received.

With services like AMP, however, online content may correspond to more than one URL. This introduces a problem: while only one domain actually corresponds to the webpage’s publisher, multiple domains can be responsible for serving a webpage. If a publisher allows AMP services to cache and serve their webpages, they must be able to sign their content even when AMP caches serve it for them. Only then can AMP-cached content prove its legitimacy.

HTTP signed exchanges directly address the problem of extending publisher signatures to services like AMP. This IETF draft specifies how publishers may sign an HTTP request/response pair (an exchange). With a signed exchange, the publisher can assure the integrity and authenticity of a response to a specific request even before the client makes the request. Given a signed exchange, the publisher authorizes intermediates (like Google’s AMP Cache) to forward the exchanges; the intermediate responds to a given request with the corresponding response in the signed HTTP request/response pair. A browser can then verify the exchange signature to assert the intermediate response’s integrity and authenticity.

This is like handing out an answer key to a quiz signed by the instructor. Having a signed answer sheet is just as good as getting the answer from the teacher in real time.

The Technical Details

An HTTP signed exchange is generated by the following steps.
First, the publisher uses MICE (Merkle Integrity Content Encoding) to provide a concise proof of integrity for the response included in the exchange. To start, the response is split into blocks of some record size bits long. Take, for example, a message ABCD, which is divided into record-size blocks A, B, C, and D. The first step to constructing a proof of integrity is to take the last block, D, and compute the following:

proof(D) = SHA-256(D || 0x0)

This produces proof(D). Then, all consequent proof values for blocks are computed as follows:

proof(C) = SHA-256(C || proof(D) || 0x1)
proof(B) = SHA-256(B || proof(C) || 0x1)
proof(A) = SHA-256(A || proof(B) || 0x1)

The generation of these proofs build the following tree:

      proof(A)
         /\
        /  \
       /    \
      A    proof(B)
            /\
           /  \
          /    \
         B    proof(C)
                /\
               /  \
              /    \
             C    proof(D)
                    |
                    |
                    D

As such, proof(A) is a 256-bit digest that a person who receives the real response should be able to recompute for themselves. If a recipient can recompute a tree head value identical to proof(A), they can verify the integrity of the response they received. In fact, this digest plays a similar role to the tree head of a Merkle Tree, which is recomputed and compared to the presented tree head to verify the membership of a particular node. The MICE-generated digest is stored in the Digest header of the response.

Next, the publisher serializes the headers and payloads of a request/response pair into CBOR (Concise Binary Object Representation). CBOR’s key-value storage is structurally similar to JSON, but creates smaller message sizes.

Finally, the publisher signs the CBOR-encoded request/response pair using the private key associated with the publisher’s certificate. This becomes the value of the sig parameter in the HTTP signed exchange.

The final HTTP signed exchange appears like the following:

sig=*MEUCIQDXlI2gN3RNBlgFiuRNFpZXcDIaUpX6HIEwcZEc0cZYLAIga9DsVOMM+g5YpwEBdGW3sS+bvnmAJJiSMwhuBdqp5UY=*;  
integrity="digest/mi-sha256";  
validity-url="https://example.com/resource.validity.1511128380";  
cert-url="https://example.com/oldcerts";  
cert-sha256=*W7uB969dFW3Mb5ZefPS9Tq5ZbH5iSmOILpjv2qEArmI=*;  
date=1511128380; expires=1511733180

Services like AMP can send signed exchanges by using a new HTTP response format that includes the signature above in addition to the original response.

When this signature is included in an AMP-cached response, a browser can verify the legitimacy of this response. First, the browser confirms that the certificate provided in cert-url corresponds to the request’s domain and is still valid. It next uses the certificate’s public key, as well as the headers and body values of request/response pair, to check the authenticity of the signature, sig. The browser then checks the integrity of the response using the given integrity algorithm, digest/mi-sha256 (aka MICE), and the contents of the Digest header. Now the browser can confirm that a response provided by a third party has the integrity and authenticity of the content’s original publisher.

After all this behind-the-scenes work, the browser can now present the original URL of the content instead of one prefixed by google.com/amp. Yippee to solving one of AMP’s most substantial pain points!

Generating HTTP Signed Exchanges with Workers

From the overview above, the process of generating an HTTP signed exchange is clearly involved. What if there were a way to automate the generation of HTTP signed exchanges and have services like AMP automatically pick them up? With Cloudflare Workers… we found a way you could have your HTTP origin exchange cake and eat it too!

We have already implemented HTTP signed exchanges for one of our customers, 1-800-Flowers. Code deployed in a Cloudflare Worker is responsible for fetching and generating information necessary to create this HTTP signed exchange.

This Worker works with Google AMP’s automatic caching. When Google’s search crawler crawls a site, it will ask for a signed exchange from the same URL if it initially responds with Vary: AMP-Cache-Transform. Our HTTP signed exchange Worker checks if we can generate a signed exchange and if the current document is valid AMP. If it is, that Vary header is returned. After Google’s crawler sees this Vary response, it will send another response with the following two headers:

AMP-Cache-Transform: google
Accept: application/signed-exchange;v=b2

When our implementation sees these header values, it will attempt to generate and return an HTTP response with Content-Type: application/signed-exchange;v=b2.

Now that Google has cached this page with the signed exchange produced by our Worker, the requested page will appear with the publisher’s URL instead of Google’s AMP Cache URL. Success!

If you’d like to see HTTP signed exchanges in action on 1-800-Flowers, follow these steps:

  1. Install/open Chrome Beta for Android. (It should be version 71+).
  2. Go to goo.gl/webpackagedemo.
  3. Search for “Christmas greens.”
  4. Click on the 1-800-Flowers link — it should be about 3 spots down with the AMP icon next to it. Along the way to getting there you should see a blue box that says “Results with the AMP icon use web packaging technology.” If you see a different message, double check that you are using the correct Chrome Beta.
    An example of AMP in action for 1-800-Flowers:

Visiting 1-800 Flowers through AMP with HTTP signed exchange

The Future: Deploying HTTP Signed Exchanges as a Worker App

Phew. There’s clearly a lot of infrastructure for publishers to build for distributing AMP content. Thankfully Cloudflare has one of the largest networks in the world, and we now have the ability to execute JavaScript at the edge with Cloudflare Workers. We have developed a prototype Worker that generates these exchanges, on the fly, for any domain. If you’d like to start experimenting with signed exchanges, we’d love to talk!

Soon, we will release this as a Cloudflare Worker application to our AMP customers. We’re excited to bring a better AMP experience to internet users and advance the Web Packaging standard. Stay tuned!

The Big Picture

Web Packaging is not simply a technology that helps fix the URL for AMP pages, it’s a fundamental shift in the way that publishing works online. For the entire history of the web up until this point, publishers have relied on transport layer security (TLS) to ensure that the content that they send to readers is authentic. TLS is great for protecting communication from attackers but it does not provide any public verifiability. This means that if a website serves a specific piece of content to a specific user, that user has no way of proving that to the outside world. This is problematic when it comes to efforts to archive the web.

Services like the Internet Archive crawl websites and keep a copy of what the website returns, but who’s to say they haven’t modified it? And who’s to say that the site didn’t serve a different version of the site to the crawler than it did to a set of readers? Web Packaging fixes this issue by allowing sites to digitally sign the actual content, not just the cryptographic keys used to transport data. This subtle change enables a profoundly new ability that we never knew we needed: the ability to record and archive content on the Internet in a trustworthy way. This ability is something that is lacking in the field of online publishing. If Web Packaging takes off as a general technology, it could be the first step in creating a trusted digital record for future generations to look back on.

Excited about the future of Web Packaging and AMP? Check out Cloudflare Ampersand to see how we’re implementing this future.

Product.platform

via The Cloudflare Blog https://meson.in/2DaAAwa

November 14, 2018 at 07:06AM

Fast Google Fonts with Cloudflare Workers

Fast Google Fonts with Cloudflare Workers

https://meson.in/2Ak00Da


Google Fonts is one of the most common third-party resources on the web, but carries with it significant user-facing performance issues. Cloudflare Workers running at the edge is a great solution for fixing these performance issues, without having to modify the publishing system for every site using Google Fonts.

This post walks through the implementation details for how to fix the performance of Google Fonts with Cloudflare Workers. More importantly, it also provides code for doing high-performance content modification at the edge using Cloudflare Workers.

Google fonts are SLOW

First, some background. Google Fonts provides a rich selection of royalty-free fonts for sites to use. You select the fonts you want to use, and end up with a simple stylesheet URL to include on your pages, as well as styles to use for applying the fonts to parts of the page:

<link href="https://fonts.googleapis.com/css?family=Open+Sans|Roboto+Slab"
      rel="stylesheet">
<style>
body {
 font-family: 'Open Sans', sans-serif;
}
h1 {
 font-family: 'Roboto Slab', serif;
}

Your visitor’s browser fetches the CSS file as soon as the HTML for the page is available. The browser will request the underlying font files when the browser does layout for the page and discovers that it needs fonts for different sections of text.

The way Google fonts are served, the CSS is on one domain (fonts.googleapis.com) and the font files are on a different domain (fonts.gstatic.com). Since they are on separate domains, the fetch for each resource takes a minimum of four round trips back and forth to the server: One each for the DNS lookup, establishing the socket connection, negotiating TLS encryption (for https) and a final round trip for the request itself.

Network waterfall diagram showing 4 round trips each for the font css and font file.
4 round trips each for the font css and font file.

The requests can’t be done in parallel because the fonts aren’t known about until after the CSS has been downloaded and the styles applied to the page. In a best-case scenario, this translates to eight round-trips before the text can be displayed (the text which was already available in the browser as soon as the HTML was available). On a slower 3G connection with a 300ms round-trip time, this adds up to a 2.4 second delay. Best case!

There is also a problem with resource prioritization. When you have all of the requests coming from the same domain on the same HTTP/2 connection they can be scheduled against each other. Critical resources (like CSS and fonts) can be pushed ahead in the queue and delivered before lower priority resources (like images). Since Google Fonts (and most third-party resources) are served from a different domain than the main page resources, they cannot be prioritized and end up competing with each other for download bandwidth. This can cause the actual fetch times to be much longer than the best-case of 8 round trips.

Network waterfall showing CSS and font download competing with low-priority images for bandwidth.
CSS and font download competing with low-priority images for bandwidth.

Users will see the images and skeleton for the page long before the actual text is displayed:

The page starts painting at 3.3 seconds with partial images and no text.
The page starts painting at 3.3 seconds with partial images and no text.
The text finally displays at 6.2 seconds into the page load.
The text finally displays at 6.2 seconds into the page load.

Fixing Google Fonts performance

The paths to fixing performance issues and making fonts lightning-fast is different for the CSS and the font files themselves. We can reduce the total number of round trips to one:

  1. Inline the CSS directly in the HTML.
  2. Proxy the Google Font files through the page origin.
Before and after diagram showing the inline CSS loading immediately as part of the HTML and the fonts loading before the images.
The inline CSS loads immediately as part of the HTML and the fonts load before the images.

Optimizing CSS delivery

For CSS, the easy answer is to just download the CSS file that Google hosts and either serve it yourself directly or place it into the HTML as an inline stylesheet. The problem with this is that Google fonts serves a browser-specific CSS file that is different for each browser so it can serve newer formats and use newer features, when supported, while still providing custom font support for older browsers.

With Cloudflare Workers, we can dynamically replace the external CSS reference with the browser-specific stylesheet content at fetch time when the HTML is requested by the browser. This way, the inlined CSS will always be up to date and support the capabilities of whichever browser is making the request. This completely eliminates any round trips for fetching the CSS for the fonts.

Optimizing font file delivery

For the font files themselves, we can eliminate all of the round trips except for the fetch itself by serving the font files directly from the same domain as the HTML. This brings with it the added benefit of serving fonts over the same HTTP/2 connection as the rest of the page resources, allowing them to be prioritized correctly and not compete for bandwidth.

Specifically, when we inline the CSS into the HTML response, we rewrite all of the font URLs to use the same domain as the HTML. When those rewritten requests arrive at the worker the URL is transformed back to the original URL served by Google and fetched by the worker (as a proxy). The worker fetches are routed through Cloudflare’s caching infrastructure so they will be cached automatically at the edge. The actual URL rewriting is pretty simple but effective. We take font URLs that look like this:

https://fonts.gstatic.com/s/...

and we just prepend the page domain to the front of the URL:

https://www.example.com/fonts.gstatic.com/s/...

That way, when they arrive at the edge, the worker can look at the path of a request and know right away that it is a proxy request for a Google font. At this point, rewriting the original URL is trivial. On the extremely odd chance that a page on your site actually has a path that starts with /fonts.gstatic.com/, those resources would break and something else should be appended to the URL to make sure they are unique.

Optimization Results

In practice, the results can be quite dramatic. On this test page for example, the wait time for fonts to become available dropped from 5.5 seconds to 1 second (an 81% improvement!):

Network waterfall diagram showing the fonts loading immediately after the HTML.
The fonts load immediately after the HTML.

Visually, the improvement to the user experience is also quite dramatic. Instead of seeing a skeleton page of images followed by the text eventually appearing, the text is available (and correctly styled) immediately and the user can start consuming the content right away:

The first paint happens much sooner at 2.5 seconds with all of the text displayed while the original page is still blank.
The first paint happens much sooner at 2.5 seconds with all of the text displayed while the original page is still blank.
At 3.3 seconds the original page finally starts to paint, displaying part of the images and no text.
At 3.3 seconds the original page finally starts to paint, displaying part of the images and no text.
At 4.4 seconds the optimized page is visibly complete while the original page still has not displayed any text.
At 4.4 seconds the optimized page is visibly complete while the original page still has not displayed any text.
At 6.2 seconds the original page finally displays the text content.
At 6.2 seconds the original page finally displays the text content.

One thing that I didn’t notice in the initial testing and only discovered when looking at the side-by-side video is that the fonts are only correctly styled in the optimized case. In the original case it took longer than Chrome’s 3-second timeout for the fonts to load and it fell back to the system fonts. Not only was the experience much faster; it was also styled correctly with the custom fonts right from the beginning.

Optimizing Google fonts with a Cloudflare worker

The full Cloudflare worker code for implementing the font optimization is available on GitHub here. Buckle-in because this is quite a bit more involved than most of the samples in the documentation.

At a high level this code:

  • Filters all requests and determines if a request is for a proxied font or HTML (and passes all other requests through unmodified).
  • Rewrites the URL and passes the fetch request through for Font requests.
  • For HTML requests:
    • Passes the request through unmodified to the origin server.
    • Returns non-UTF-8 content unmodified..
    • Processes the HTML response in streaming chunks as it is available.
    • Replaces Google font stylesheet link tags with the CSS inline with the font URLs rewritten to proxy through the origin.

The code here is slightly simplified to make it clearer to understand the flow. The full code on GitHub adds support for an in-memory worker cache for the CSS (in addition to the persistent cache API) and provides query parameters for toggling the HTML rewriting on and off (for testing).

The content modification is all done by operating on the HTML as strings (with a combination of regular expressions and string matches). This is much faster and lighter weight than parsing the HTML into a virtual DOM, operating on it and converting back to HTML. It also allows for incremental processing of the HTML as a stream.

Entry Point

The addEventListener(“fetch”) call is the main entry point for any worker and houses the JavaScript for intercepting inbound requests. If the handler does nothing, then the requests will be passed through normally and the worker will be out of the path in processing the response. Our goal it to minimize the amount of work that the worker has to do to determine if it is a request that it is interested in.

In the case of the proxied font requests, we can just look at the request URL and see that the path starts with /fonts.gstatic.com/. To identify requests for HTML content we can look at the “accept” header on the request. Every major browser I have tested includes text/html on the list of content types that it will accept when requesting a document. On the off chance that there is a browser that doesn’t include it as an accept header, the HTML will just be passed through and returned unmodified. The goal with everything here is to fail-safe and just return unoptimized content for any edge cases that aren’t covered. This way nothing breaks; it just doesn’t get the added performance boost.

addEventListener("fetch", event => {
 
 const url = new URL(event.request.url);
 const accept = event.request.headers.get('accept');
 if (event.request.method === 'GET' &&
     url.pathname.startsWith('/fonts.gstatic.com/')) {
 
   // Proxy the font file requests
   event.respondWith(proxyRequest('https:/' + url.pathname,
                                  event.request));
 
 } else if (accept && accept.indexOf("text/html") !== -1) {
 
   // Process the HTML
   event.respondWith(processHtmlRequest(event.request, event));
 
 }
})

Request Proxy

The proxying of the font requests is pretty straightforward. Since we are crossing origins it is generally a bad idea to just reuse the existing request object with a new URL. That can leak user data like cookies to a Third-party. Instead, we make a new request, clone a subset of the headers and pass the new fetch request back for the Worker runtime to handle.

The fetch path between workers and the outside Internet goes through the Cloudflare cache so the actual font files will only be fetched from Google if they aren’t already in the cache. Even in that case, the connection from Cloudflare’s edge to Google’s font servers is much faster (and more reliable) than the end-user’s connection from the browser. Even on a cache miss, it is an insignificant delay.

async function proxyRequest(url, request) {
 
 // Only pass through a subset of request headers
 let init = {
   method: request.method,
   headers: {}
 };
 const proxyHeaders = ["Accept",
                       "Accept-Encoding",
                       "Accept-Language",
                       "Referer",
                       "User-Agent"];
 for (let name of proxyHeaders) {
   let value = request.headers.get(name);
   if (value) {
     init.headers[name] = value;
   }
 }
 const clientAddr = request.headers.get('cf-connecting-ip');
 if (clientAddr) {
   init.headers['X-Forwarded-For'] = clientAddr;
 }
 
 // Only include a strict subset of response headers
 const response = await fetch(url, init);
 if (response) {
   const responseHeaders = ["Content-Type",
                            "Cache-Control",
                            "Expires",
                            "Accept-Ranges",
                            "Date",
                            "Last-Modified",
                            "ETag"];
   let responseInit = {status: response.status,
                       statusText: response.statusText,
                       headers: {}};
   for (let name of responseHeaders) {
     let value = response.headers.get(name);
     if (value) {
       responseInit.headers[name] = value;
     }
   }
   const newResponse = new Response(response.body, responseInit);
   return newResponse;
 }
 
 return response;
}

In addition to filtering the request headers we also filter the response headers sent back to the browser. If you’re not careful you could end up in a situation where a third-party is setting cookies on your origin or even turning on something like HTTP Strict Transport Security for your site.

Streaming HTML Processing

The HTML path is more complicated because we are going to intercept and modify the content itself.

In processing the HTML request, the first thing we want to do is make sure it is actually an HTML response. If it is something else, then we should get out of the way and let the response stream back to the browser as it does normally. It is very possible that a PDF document, file download, or even a directly opened image, has a Accept of text/html. It is critical to check the actual content that is being responded with to make sure it is something we want to inspect and possibly modify.

The easiest way to modify a response is to just wait for the response to be fully complete, process it as a single block of HTML, and then pass the modified HTML back to the browser:

 async function processHtmlRequest(request) {
 
 // Fetch from origin server.
 const response = await fetch(request)
 if (response.headers.get("content-type").indexOf("text/html") !== -1) {
  
   let content = await response.text();
   content = await modifyHtmlChunk(content, request);
 
   // Create a cloned response with our modified HTML
   return new Response(content, response);
 }
 return response;
}

This works reasonably well if you are sure that all of the HTML is UTF-8 (or ascii), and the server returns all of the HTML at once, but there are some pretty serious concerns with doing it this way:

  • The memory use can be unbounded and only limited by the size of the largest HTML response (possibly causing the worker to be killed for using too much memory).
  • Significant delay will be added to any pages where the server flushes the initial content early and then does some expensive/slow work before returning the rest of the HTML.
  • This only works if the text content uses an encoding that JavaScript can decode directly as UTF-8. Any other character encodings will fail to decode.

For our worker we are going to process the HTML stream incrementally as it arrives from the server and pass it through to the browser as soon as possible (and pass-through any content that isn’t utf-8 unmodified).

Processing HTML as a stream

First we are going to look at what it takes to process the HTML stream incrementally. We will leave the character encoding changes out for now to keep things (relatively) simple.

To process the stream incrementally, we need to generate a new fetch response to pass back from the worker that uses a TransformStream for its content. That will allow us to pass the response itself back immediately and write to the stream as soon as we have content to add. We pass all of the other headers through unmodified.

async function processHtmlRequest(request) {
 
 // Fetch from origin server.
 const response = await fetch(request)
 if (response.headers.get("content-type").indexOf("text/html") !== -1) {
  
   // Create an identity TransformStream (a.k.a. a pipe).
   // The readable side will become our new response body.
   const { readable, writable } = new TransformStream();
 
   // Create a cloned response with our modified stream
   const newResponse = new Response(readable, response);
 
   // Start the async processing of the response stream (NO await!)
   modifyHtmlStream(response.body, writable, request);
 
   // Return the in-process response so it can be streamed.
   return newResponse;
 }
 return response;
}

The key thing here is to not wait for the async modifyHtmlStream async function to complete before passing the new response back from the worker. This way the initial headers can be sent immediately and the response will continue to stream anything written into the TransformStream until it is closed.

Processing the HTML stream in chunks as it arrives is a little tricky. The HTML stream will arrive in chunks of arbitrary sizes as strings. We need to add some protection to make sure that a chunk boundary doesn’t split a link tag. If it does, and we don’t account for it, we can miss a stylesheet link (or worse, process a partial link URL with the wrong style type). To make sure we don’t split link tags, we search from the end of the string for “<link “ and from the start of the last link tag we search forward for a closing “>” tag. If we don’t find one, then there is a partial link tag and we split the string just before the link tag starts. We process everything up to the split link tag and keep the partial tag to prepend it to the next chunk of data that arrives.

An alternative would be to keep accumulating data and only process it when there is no split link tag at the end, but this way we can return more data to the browser sooner.

When the incoming stream is complete, we process any partial data left over from the previous chunk and close the output stream (ending the response to the browser).

async function modifyHtmlStream(readable, writable, request) {
 const reader = readable.getReader();
 const writer = writable.getWriter();
 const encoder = new TextEncoder();
 let decoder = new TextDecoder();
 
 let partial = '';
 let content = '';
 
 for(;;) {
   // Read the next chunk of HTML from the fetch request.
   const { done, value } = await reader.read()
 
   if (done) {
 
     // Send any remaining fragment and complete the request.
     if (partial.length) {
       partial = await modifyHtmlChunk(partial, request);
       await writer.write(encoder.encode(partial));
       partial = '';
     }
     break;
 
   }
  
   try {
     let chunk = decoder.decode(value, {stream:true});
 
     // Add the inbound chunk to the the remaining partial chunk
     // from the previous read.
     content = partial + chunk;
     partial = '';
 
     // See if there is an unclosed link tag at the end (and if so,
     // carve it out to complete when the remainder comes in).
     const linkPos = content.lastIndexOf('<link');
     if (linkPos >= 0) {
       const linkClose = content.indexOf('/>', linkPos);
       if (linkClose === -1) {
         partial = content.slice(linkPos);
         content = content.slice(0, linkPos);
       }
     }
 
     if (content.length) {
       // Do the actual HTML modifications on the current chunk.
       content = await modifyHtmlChunk(content, request);
     }
   } catch (e) {
     // Ignore the exception
   }
 
   // Send the processed HTML chunk to the requesting browser.
   if (content.length) {
     await writer.write(encoder.encode(content));
     content = '';
   }
 }
 
 await writer.close()
}

One thing I was initially worried about was having to modify the “content-length” response header from the original response since we are modifying the content. Luckily, the worker takes care of that automatically and it isn’t something you have to implement.

There is a try/catch handler around the processing in case something goes horribly wrong with the decode.

The actual HTML rewriting is handled in “modifyHtmlChunk”. This is just the logic for processing the incoming data as incremental chunks.

Dealing with character encodings other than UTF-8

We intentionally skipped over handling character encodings other than UTF-8 up until now. To handle arbitrary pages you will need to be able to process other character encodings. The Worker runtime only supports decoding UTF-8 but we need to make sure that we don’t break any content that isn’t UTF-8 (or similar). To do this, we detect the current encoding if it is specified and anything that isn’t UTF-8 is passed through unmodified. In the case that the content type can not be detected we also detect decode errors and pass content through unmodified when they occur.

The HTML charset can be specified in the content-type response header or as a <meta charset> tag in the HTML itself.

For the response headers it is pretty simple. When we get the original response, see if there is a charset in the content-type header. If there is, extract the current value and if it isn’t a supported charset just pass the response through unmodified.

   // Workers can only decode utf-8. If it is anything else, pass the
   // response through unmodified
   const VALID_CHARSETS = ['utf-8', 'utf8', 'iso-8859-1', 'us-ascii'];
   const charsetRegex = /charset\s*=\s*([^\s;]+)/mgi;
   const match = charsetRegex.exec(contentType);
   if (match !== null) {
     let charset = match[1].toLowerCase();
     if (!VALID_CHARSETS.includes(charset)) {
       return response;
     }
   }
  
   // Create an identity TransformStream (a.k.a. a pipe).
   // The readable side will become our new response body.
   const { readable, writable } = new TransformStream();
 
   // Create a cloned response with our modified stream
   const newResponse = new Response(readable, response);
 
   // Start the async processing of the response stream
   modifyHtmlStream(response.body, writable, request, event);

For the cases where there is a “” tag in the HTML (and possibly no header) things get a bit more complicated. If at any point an unsupported charset is detected then we pipe the incoming byte stream directly into the output stream unmodified. We first decode the first chunk of HTML response using the default decoder. Then, if a ” tag is found in the html we extract the charset. If it isn’t a supported charset then we enter passthrough mode. If at any point the input stream can’t be decoded (likely because of an invalid charset) we also enter passthrough mode and pipe the remaining content through unprocessed.

async function modifyHtmlStream(readable, writable, request, event) {
 const reader = readable.getReader();
 const writer = writable.getWriter();
 const encoder = new TextEncoder();
 let decoder = new TextDecoder("utf-8", {fatal: true});
 
 let firstChunk = true;
 let unsupportedCharset = false;
 
 let partial = '';
 let content = '';
 
 try {
   for(;;) {
     const { done, value } = await reader.read();
     if (done) {
       if (partial.length) {
         partial = await modifyHtmlChunk(partial, request, event);
         await writer.write(encoder.encode(partial));
         partial = '';
       }
       break;
     }
 
     let chunk = null;
     if (unsupportedCharset) {
       // Pass the data straight through
       await writer.write(value);
       continue;
     } else {
       try {
         chunk = decoder.decode(value, {stream:true});
       } catch (e) {
         // Decoding failed, switch to passthrough
         unsupportedCharset = true;
         if (partial.length) {
           await writer.write(encoder.encode(partial));
           partial = '';
         }
         await writer.write(value);
         continue;
       }
     }
 
     try {
       // Look inside of the first chunk for a HTML charset or
       // content-type meta tag.
       if (firstChunk) {
         firstChunk = false;
         if (chunkContainsInvalidCharset(chunk)) {
           // switch to passthrough
           unsupportedCharset = true;
           if (partial.length) {
             await writer.write(encoder.encode(partial));
             partial = '';
           }
           await writer.write(value);
           continue;
         }
       }
 
       content = partial + chunk;
       partial = '';
 
       // See if there is an unclosed link tag at the end (and if so,
       // carve it out to complete when the remainder comes in).
       const linkPos = content.lastIndexOf('<link');
       if (linkPos >= 0) {
         const linkClose = content.indexOf('/>', linkPos);
         if (linkClose === -1) {
           partial = content.slice(linkPos);
           content = content.slice(0, linkPos);
         }
       }
 
       if (content.length) {
         content = await modifyHtmlChunk(content, request, event);
       }
     } catch (e) {
       // Ignore the exception
     }
     if (content.length) {
       await writer.write(encoder.encode(content));
       content = '';
     }
   }
 } catch(e) {
   // Ignore the exception
 }
 
 try {
   await writer.close();
 } catch(e) {
   // Ignore the exception
 }
}

There is a helper that scans for the charset in both meta tags that support setting the charset:

function chunkContainsInvalidCharset(chunk) {
 let invalid = false;
 const VALID_CHARSETS = ['utf-8', 'utf8', 'iso-8859-1', 'us-ascii'];
 
 // meta charset
 const charsetRegex = /<\s*meta[^>]+charset\s*=\s*['"]([^'"]*)['"][^>]*>/mgi;
 const charsetMatch = charsetRegex.exec(chunk);
 if (charsetMatch) {
   const docCharset = charsetMatch[1].toLowerCase();
   if (!VALID_CHARSETS.includes(docCharset)) {
     invalid = true;
   }
 }
 // content-type
 const contentTypeRegex = /<\s*meta[^>]+http-equiv\s*=\s*['"]\s*content-type[^>]*>/mgi;
 const contentTypeMatch = contentTypeRegex.exec(chunk);
 if (contentTypeMatch) {
   const metaTag = contentTypeMatch[0];
   const metaRegex = /charset\s*=\s*([^\s"]*)/mgi;
   const metaMatch = metaRegex.exec(metaTag);
   if (metaMatch) {
     const charset = metaMatch[1].toLowerCase();
     if (!VALID_CHARSETS.includes(charset)) {
       invalid = true;
     }
   }
 }
 return invalid;
}

HTML Business Logic

Finally, we can start the actual logic for inlining the font CSS. The basic logic is:

  • Use a regex to find link tags for Google fonts css.
  • Fetch the browser-specific version of the CSS (from cache if possible).
    • The fetch logic (discussed later) modifies the font URLs in the CSS to proxy through the worker.
  • Replace the link tag with an inline style block with the CSS.
async function modifyHtmlChunk(content, request, event) {
 const fontCSSRegex = /<link\s+[^>]*href\s*=\s*['"]((https?:)?\/\/fonts.googleapis.com\/css[^'"]+)[^>]*>/mgi;
 let match = fontCSSRegex.exec(content);
 while (match !== null) {
   const matchString = match[0];
   if (matchString.indexOf('stylesheet') >= 0) {
     const fontCSS = await fetchCSS(match[1], request, event);
     if (fontCSS.length) {
       // See if there is a media type on the link tag
       let mediaStr = '';
       const mediaMatch = matchString.match(/media\s*=\s*['"][^'"]*['"]/mig);
       if (mediaMatch) {
         mediaStr = ' ' + mediaMatch[0];
       }
       // Replace the actual css
       let cssString = "<style" + mediaStr + ">\n";
       cssString += fontCSS;
       cssString += "\n</style>\n";
       content = content.split(matchString).join(cssString);
     }
     match = fontCSSRegex.exec(content);
   }
 }
 
 return content;
}

The fetching (and modifying) of the CSS is a little more complicated than a straight passthrough because we want to cache the result when possible. We cache the responses locally using the worker’s Cache API. Since the response is browser-specific, and we don’t want to fragment the cache too crazily, we create a custom cache key based on the browser user agent string that is basically browser+version+mobile.

Some plans have access to named cache storage, but to work with all plans it is easiest if we just modify the font URL that gets stored in cache and append the cache key to the end of the URL as a query parameter. The cache URL never gets sent to a server but is useful for local caching of different content that shares the same URL. For example:

https://fonts.googleapis.com/css?family=Roboto&chrome71

If the CSS isn’t available in the cache then we create a fetch request for the original URL from the Google servers, passing through the HTML url as the referer, the correct browser user agent string and the client’s IP address in a standard proxy X-Forwarded-For header. Once the response is available we store it in the cache for future requests.

For browsers that can’t be identified by user agent string a generic request for css is sent with the user agent string from Internet Explorer 8 to get the lowest common denominator fallback CSS.

The actual modification of the CSS just uses a regex to look for font URLs, replaces them with the HTML origin as a prefix.

async function fetchCSS(url, request) {
 let fontCSS = "";
 const userAgent = request.headers.get('user-agent');
 const clientAddr = request.headers.get('cf-connecting-ip');
 const browser = getCacheKey(userAgent);
 const cacheKey = browser ? url + '&' + browser : url;
 const cacheKeyRequest = new Request(cacheKey);
 let cache = null;
 
 let foundInCache = false;
 // Try pulling it from the cache API (wrap it in case it's not implemented)
 try {
   cache = caches.default;
   let response = await cache.match(cacheKeyRequest);
   if (response) {
     fontCSS = response.text();
     foundInCache = true;
   }
 } catch(e) {
   // Ignore the exception
 }
 
 if (!foundInCache) {
   let headers = {'Referer': request.url};
   if (browser) {
     headers['User-Agent'] = userAgent;
   } else {
     headers['User-Agent'] =
       "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)";
   }
   if (clientAddr) {
     headers['X-Forwarded-For'] = clientAddr;
   }
 
   try {
     const response = await fetch(url, {headers: headers});
     fontCSS = await response.text();
 
     // Rewrite all of the font URLs to come through the worker
     fontCSS = fontCSS.replace(/(https?:)?\/\/fonts\.gstatic\.com\//mgi,
                               '/fonts.gstatic.com/');
 
     // Add the css info to the font caches
     FONT_CACHE[cacheKey] = fontCSS;
     try {
       if (cache) {
         const cacheResponse = new Response(fontCSS, {ttl: 86400});
         event.waitUntil(cache.put(cacheKeyRequest, cacheResponse));
       }
     } catch(e) {
       // Ignore the exception
     }
   } catch(e) {
     // Ignore the exception
   }
 }
 
 return fontCSS;
}

Generating the browser-specific cache key is a little sensitive since browsers tend to clone each other’s user agent strings and add their own information to them. For example, Edge includes a Chrome identifier and Chrome includes a Safari identifier, etc. We don’t necessarily have to handle every browser string since it will fallback to the least common denominator (ttf files without unicode range support) but it is helpful to catch as many of the large mainstream browser engines as possible.

function getCacheKey(userAgent) {
 let os = '';
 const osRegex = /^[^(]*\(\s*(\w+)/mgi;
 let match = osRegex.exec(userAgent);
 if (match) {
   os = match[1];
 }
 
 let mobile = '';
 if (userAgent.match(/Mobile/mgi)) {
   mobile = 'Mobile';
 }
 
 // Detect Edge first since it includes Chrome and Safari
 const edgeRegex = /\s+Edge\/(\d+)/mgi;
 match = edgeRegex.exec(userAgent);
 if (match) {
   return 'Edge' + match[1] + os + mobile;
 }
 
 // Detect Chrome next (and browsers using the Chrome UA/engine)
 const chromeRegex = /\s+Chrome\/(\d+)/mgi;
 match = chromeRegex.exec(userAgent);
 if (match) {
   return 'Chrome' + match[1] + os + mobile;
 }
 
 // Detect Safari and Webview next
 const webkitRegex = /\s+AppleWebKit\/(\d+)/mgi;
 match = webkitRegex.exec(userAgent.match);
 if (match) {
   return 'WebKit' + match[1] + os + mobile;
 }
 
 // Detect Firefox
 const firefoxRegex = /\s+Firefox\/(\d+)/mgi;
 match = firefoxRegex.exec(userAgent);
 if (match) {
   return 'Firefox' + match[1] + os + mobile;
 }
  return null;
}

Profit!

Any site served through Cloudflare can implement workers to rewrite their content but for something like Google fonts or other third-party resources it gets much more interesting when someone implements it once and everyone else can benefit. With Cloudflare Apps’ new worker support you can bundle up and deliver complex worker logic for anyone else to consume and publish it to the Apps marketplace.

If you are a third-party content provider for sites, think about what you might be able to do to leverage workers for your content for sites that are served through Cloudflare.

I get excited thinking about the performance implications of something like a tag manager running entirely on the edge without the sites having to change their published pages and without browsers having to fetch heavy JavaScript to do the page modifications. It can be done dynamically for every request directly on the edge!

Product.platform

via The Cloudflare Blog https://meson.in/2DaAAwa

November 22, 2018 at 08:32PM

Every 7.8μs your computer’s memory has a hiccup

Every 7.8μs your computer’s memory has a hiccup

https://meson.in/2Bx8OaC

Every 7.8μs your computer’s memory has a hiccup

Every 7.8μs your computer’s memory has a hiccup
Modern DDR3 SDRAM. Source: BY-SA/4.0 by Kjerish

During my recent visit to the Computer History Museum in Mountain View, I found myself staring at some ancient magnetic core memory.

Every 7.8μs your computer’s memory has a hiccup
Source: BY-SA/3.0 by Konstantin Lanzet

I promptly concluded I had absolutely no idea on how these things could ever work. I wondered if the rings rotate (they don’t), and why each ring has three wires woven through it (and I still don’t understand exactly how these work). More importantly, I realized I have very little understanding on how the modern computer memory – dynamic RAM – works!

Every 7.8μs your computer’s memory has a hiccup
Source: Ulrich Drepper’s series about memory

I was particularly interested in one of the consequences of how dynamic RAM works. You see, each bit of data is stored by the charge (or lack of it) on a tiny capacitor within the RAM chip. But these capacitors gradually lose their charge over time. To avoid losing the stored data, they must regularly get refreshed to restore the charge (if present) to its original level. This refresh process involves reading the value of every bit and then writing it back. During this "refresh" time, the memory is busy and it can’t perform normal operations like loading or storing bits.

This has bothered me for quite some time and I wondered… is it possible to notice the refresh delay in software?

Dynamic RAM refresh primer

Each DIMM module is composed of "cells" and "rows", "columns", "sides" and/or "ranks". This presentation from the University of Utah explains the nomenclature. You can check the configuration of memory in your computer with decode-dimms command. Here’s an example:

$ decode-dimms
Size                                       4096 MB
Banks x Rows x Columns x Bits              8 x 15 x 10 x 64
Ranks                                      2

For today we don’t need to get into the whole DDR DIMM layout, we just need to understand a single memory cell, storing one bit of information. Specifically we are only interested in how the refresh process is performed.

Let’s review two sources:

Each bit stored in dynamic memory must be refreshed, typically every 64ms (called Static Refresh). This is a rather costly operation. To avoid one major stall every 64ms, this process is divided into 8192 smaller refresh operations. In each operation, the computer’s memory controller sends refresh commands to the DRAM chips. After receiving the instruction a chip will refresh 1/8192 of its cells. Doing the math – 64ms / 8192 = 7812.5 ns or 7.81 μs. This means:

  • A refresh command is issued every 7812.5 ns. This is called tREFI.
  • It takes some time for the chip to perform the refresh and recover so it can perform normal read and write operations again. This time, called tRFC is either 75ns or 120ns (as per the mentioned Micron datasheet).

When the memory is hot (>85C) the memory retention time drops and the static refresh time halves to 32ms, and tREFI falls to 3906.25 ns.

A typical memory chip is busy with refreshes for a significant fraction of its running time – between 0.4% to 5%. Furthermore, memory chips are responsible for a nontrivial share of typical computer’s power draw, and large chunk of that power is spent on performing the refreshes.

For the duration of the refresh action, the whole memory chip is blocked. This means each and every bit in memory is blocked for more than 75ns every 7812ns. Let’s measure this.

Preparing an experiment

To measure operations with nanosecond granularity we must write a tight loop, perhaps in C. It looks like this:

    for (i = 0; i < ...; i++) {
        __m128i x;
        // Perform data load. MOVNTDQA is here only for kicks,
        // for writeback memory it makes zero difference, the
        // caches are still polluted.
        asm volatile("movntdqa (%1), %0" :  "=x" (x) : "a" (one_global_var));


        // Lie to compiler to prevent optimizing this code. Claim that 'x' is referred to.
      asm volatile("" : : "x" ( x ));

              // Flush the value from CPU caches, to force memory fetch next time.
        _mm_clflush(one_global_var);

             // Measure and record time
       clock_gettime(CLOCK_MONOTONIC, &ts);
    }

Full code is available on Github.

The code is really straightforward. Perform a memory read. Flush data from CPU caches. Measure time.

I’m attempting to use MOVNTDQA to perform the data load, but this is not important. In fact, current hardware ignores the non-temporary load hint on writeback memory. In practice any memory load instruction would do.

On my computer it generates data like this:

# timestamp, loop duration
3101895733,     134
3101895865,     132
3101896002,     137
3101896134,     132
3101896268,     134
3101896403,     135
3101896762,     359
3101896901,     139
3101897038,     137

Typically I get ~140ns per loop, periodically the loop duration jumps to ~360ns. Sometimes I get odd readings longer than 3200ns.

Unfortunately, the data turns out to be very noisy. It’s very hard to see if there is a noticeable delay related to the refresh cycles.

Fast Fourier Transform

At some point it clicked. Since we want to find a fixed-interval event, we can feed the data into the FFT (fast fourier transform) algorithm, which deciphers the underlying frequencies.

I’m not the first one to think about this – Mark Seaborn of Rowhammer fame implemented this very technique back in 2015. Even after peeking at Mark’s code, getting FFT to work turned out to be harder than I anticipated. But finally I got all the pieces together.

First we need to prepare the data. FFT requires input data to be sampled with a constant sampling interval. We also want to crop the data to reduce noise. By trial and error I found the best results are when data is preprocessed:

  • Small (smaller than average * 1.8) values of loop iterations are cut out, ignored, and replaced with readings of "0". We really don’t want to feed the noise into the algorithm.
  • All the remaining readings are replaced with "1", since we really don’t care about the amplitude of the delay caused by some noise.
  • I settled on sampling interval of 100ns, but any number up to a Nyquist value (double expected frequency) also work fine.
  • The data needs to be sampled with fixed timings before feeding to FFT. All reasonable sampling methods work ok, I ended up doing basic linear interpolation.

The algorithm is roughly:

UNIT=100ns
A = [(timestamp, loop_duration),...] 
p = 1
for curr_ts in frange(fist_ts, last_ts, UNIT):
    while not(A[p-1].timestamp <= curr_ts < A[p].timestamp):
        p += 1
    v1 = 1 if avg*1.8 <= A[p-1].duration <= avg*4 else 0
    v2 = 1 if avg*1.8 <= A[p].duration <= avg*4 else 0
    v = estimate_linear(v1, v2, A[p-1].timestamp, curr_ts, A[p].timestamp)
    B.append( v )

Which on my data produces fairly boring vector like this:

[0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0,
 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...]

The vector is pretty long though, typically about ~200k data points. With data prepared like this, we are ready to feed it into FFT!

C = numpy.fft.fft(B)
C = numpy.abs(C)
F = numpy.fft.fftfreq(len(B)) * (1000000000/UNIT)

Pretty simple, right? This produces two vectors:

  • C contains complex numbers of the frequency components. We are not interested in complex numbers and we can flatten them out by calling abs().
  • F contains labels to what frequency bin lies in which place in vector C. We need to normalize it to Hz – by multiplying it by the input vector sampling frequency.

The result can be charted:

Every 7.8μs your computer’s memory has a hiccup

Y axis is unit-less since we normalized the delay times. Even though, it clearly shows spikes at some fixed frequency intervals. Let’s zoom in:

Every 7.8μs your computer’s memory has a hiccup

We can clearly see first three spikes. After a bit of wishy-washy arithmetic, involving filtering the reading at least 10 times the size of average, we can extract the underlying frequencies:

127850.0
127900.0
127950.0
255700.0
255750.0
255800.0
255850.0
255900.0
255950.0
383600.0
383650.0

Doing the math: 1000000000 (ns/s) / 127900 (Hz) = 7818.6 ns

Hurray! The first frequency spike is indeed what we were looking for, and indeed does correlate with the refresh times.

The other spikes at 256kHz, 384kHz, 512kHz and so on, are multiplies of our base frequency of 128kHz called harmonics. These are a side effect of performing FFT on something like a square wave and totally expected.

For easier experimentation, we prepared a command line version of this tool. You can run the code yourself. Here is a sample run on my server:

~/2018-11-memory-refresh$ make
gcc -msse4.1 -ggdb -O3 -Wall -Wextra measure-dram.c -o measure-dram
objdump -dx measure-dram | egrep 'movntdqa' -A 2 -B 2
 a29:    0f 1f 80 00 00 00 00     nopl   0x0(%rax)
 a30:    4c 89 e0                 mov    %r12,%rax
 a33:    66 0f 38 2a 00           movntdqa (%rax),%xmm0
 a38:    41 0f ae 3c 24           clflush (%r12)
 a3d:    0f ae f0                 mfence
./measure-dram | python3 ./analyze-dram.py
[*] Verifying ASLR: main=0x555555554890 stack=0x7fffffefe2ec
[ ] Fun fact. I did 40663553 clock_gettime()'s per second
[*] Measuring MOVNTDQA + CLFLUSH time. Running 131072 iterations.
[*] Writing out data
[*] Input data: min=117 avg=176 med=167 max=8172 items=131072
[*] Cutoff range 212-inf
[ ] 127849 items below cutoff, 0 items above cutoff, 3223 items non-zero
[*] Running FFT
[*] Top frequency above 2kHz below 250kHz has magnitude of 7716
[+] Top frequency spikes above 2kHZ are at:
127906Hz    7716
255813Hz    7947
383720Hz    7460
511626Hz    7141

I must admit the code is not perfectly stable. In case of trouble, consider disabling Turbo Boost, CPU frequency scaling and tuning for performance.

Finale

There are two major takeaways from this work.

We’ve seen that the low level data is pretty hard to analyze and seem to be pretty noisy. Instead of trying to figure something out with naked eye, we can always employ good old FFT. Some wishful thinking is needed when preparing the data.

Most importantly, we showed it’s often possible to measure subtle hardware behaviours from a simple userspace process. This kind of thinking led to the discovery of the original Rowhammer vulnerability, was used in Meltdown/Spectre attacks and showed again in recent ECC-defeating reincarnation of Rowhammer.

There is so much more to be said. We barely scratched the surface of the inner workings of the memory subsystem. I recommend further reading:

Finally, here is a good reading about the old magnetic core memory:

Product.platform

via The Cloudflare Blog https://meson.in/2DaAAwa

November 24, 2018 at 12:56AM

Corporate debt – the IMF gets worried

Corporate debt – the IMF gets worried

https://meson.in/2QmerAA


The IMF does not pull any punches in its latest post on the IMF blog.  It is really worried that so-called ‘leveraged loans’ are reaching dangerous levels globally.  These loans, usually arranged by a syndicate of banks, are made to companies that are heavily indebted or have weak credit ratings. They are called “leveraged” because the ratio of the borrower’s debt to assets or earnings significantly exceeds industry norms.  The level of these loans globally now stands at $1.3trn and annual issuance is now matching the pre-crash year of 2007.

With interest rates extremely low for years and with ample money flowing though the financial system, yield-hungry investors are tolerating ever-higher levels of risk and betting on financial instruments that, in less speculative times, they might sensibly shun.” says the IMF.  About 70% of these loans are in the US; so that is where the risk of a credit crunch is greatest.  And more than half of this year’s total involves money borrowed to fund mergers and acquisitions and leveraged buyouts (LBOs), pay dividends, and buy back shares from investor—in other words, for financial risk-taking rather than productive investment.

And even though corporate earnings in the US have risen sharply in 2018, the share of companies that have raised their debt to earnings above five times has reached a higher level than in 2007.

New deals also include fewer investor protections, known as covenants, and lower loss-absorption capacity. This year, so-called covenant-lite loans account for up 80% of new loans arranged for non-bank lenders (so-called “institutional investors”), up from about 30% in 2007.

With rising leverage, weakening investor protections and eroding debt cushions, average recovery rates for defaulted loans have fallen to 69% from the pre-crisis average of 82%. So any sizeable defaults would hit the ‘real’ economy hard.

Back in 2007, the debt crunch was exacerbated by the phenomenal growth in credit derivatives issued by non-banks, the so-called ‘shadow banks’, not subject to central bank controls.  Now again, it is in the shadow bank area that a debt crisis is looming.  These institutions now hold about $1.1 trillion of leveraged loans in the US, almost double the pre-crisis level.  On top of that are $1.2 trillion in high yield, or junk bonds, outstanding. The non-bank institutions include loan mutual funds, insurance companies, pension funds, and collateralized loan obligations (CLOs), which package loans and then resell them to still other investors. CLOs buy more than half of overall leveraged loan issuance. Mutual funds (which are usually bought by average savers through their banks) that invest in leveraged loans have grown from roughly $20 billion in assets in 2006 to about $200 billion this year, accounting for over 20% of loans outstanding.

All this debt can be serviced as long as earnings pour into companies and the interest rate on the debt does not rise too much.  Corporate earnings appear to be strong at least in the US.  In the latest quarter of reported company earnings, with 85-90% of companies having reported, US corporate earnings are up nearly 27% from the same period last year (although sales revenues are up only 8%).  US sales revenue growth is about 20% higher than in Europe and Japan but earnings growth is two to three times larger.  That tells you US earnings are being inflated by the one-off Trump corporate tax cuts etc.

Moreover it is earnings in the energy/oil sector that have led the way, as oil prices rose through the last year.  Recently, the oil price has taken a serious plunge as supply (production in the US) has rocketed. That’s going to reduce the contribution of the large energy sector to earnings growth.

Anyway, the reported earnings by companies in their accounts are really smoke and mirrors.  The real level of profits is better revealed by the wider data provided in the official national accounts.  And the discrepancy between the rise in profits as recorded there and the company earnings reports has not been as high since the dot.com bust of 2000, which eventually presaged the mild economic slump of 2001.  Reported US corporate earnings per share are rising fast, but ‘whole economy’ profits are basically flat.

The other moving part is the cost of borrowing.  The decade of low interest rates is over as the US Fed continues with its policy of hiking its policy rate.

The Fed policy sets the floor for all borrowing rates, not only in the US economy, but also abroad whenever borrowing dollars.

As I have explained in a number of posts, the Fed’s hiking policy will add to the burden of servicing corporate debt, particularly for those companies that have resorted to leveraged loans and junk bonds.  Herein lies the kernel of a future slump.


This entry was posted on November 16, 2018 at 3:19 pm and is filed under capitalism, economics, Profitability. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.

Econo.Progressive

via Michael Roberts Blog https://meson.in/2ErIaA0

November 17, 2018 at 12:33AM

A new DNA sequencing service wants to reward you for sharing your data

A new DNA sequencing service wants to reward you for sharing your data

https://meson.in/2AiJPGg

George Church's firm wants to put genomes on blockchains

George Church’s firm wants to put genomes on blockchains

Deanne Fitzmaurice/NGS/Alamy

By Alice Klein

A new company co-founded by Harvard geneticist George Church wants to let people learn about their genome for free. The blockchain-secured service will sequence people’s genomes without charging a fee, and offer rewards to those who let third parties access their data.

Consumers can already get their DNA sequenced by companies like 23andMe and Helix, but it typically costs around $200. If a user consents, their results can be sold to third parties such as pharmaceutical firms, but users don’t receive any compensation or reward for this.

Nebula Genomics, which launches today, is intended to connect consumers interested in exploring their genetics with researchers looking for large DNA datasets to assist with drug development. Co-founder George Church is a well-known pioneering geneticist at Harvard University, whose other ambitions include bringing back the woolly mammoth, editing our genes to fix diseases, and reversing ageing.

Advertisement

The service offers to sequence your entire genome for free and secure it using a blockchain. You can then share your genomic data anonymously with companies or research institutes of your choosing, in return for “Nebula credits”.

Collecting credits

Users will also receive credits for sharing other health information, like whether they have diabetes or arthritis. This should help researchers identify patterns between certain gene variants and diseases, and potentially identify novel drug targets, says the company’s co-founder Dennis Grishin.

The credits earned by users will allow them to access other services on the Nebula platform. For example, they could be used to purchase information about a user’s risks of developing certain diseases like breast cancer or Alzheimer’s disease based on their genomic data.

For the time being, users won’t be able to convert the credits into hard cash. Pharmaceutical companies are currently prepared to pay upwards of $1000 for individual genomes, but Nebula hasn’t decided yet whether to allow cash transactions, says Grishin.

Jacqueline Savard at Deakin University in Australia warns that some drug companies may use people’s data in ways they aren’t aware of or can’t foresee. “People should be free to do what they want with their genomic data,” she says. “But they need to be given clear information about how it will be used so they can make the right decisions.”

More on these topics:

Bio.medical

via New Scientist – Health https://meson.in/2AA4I2U

November 18, 2018 at 05:11AM