Chromium Blog: November 2008

185 captures

29 Apr 2017 - 19 Jul 2026

Sep	OCT	Nov
	15
2019	2020	2021

success

fail

About this capture

COLLECTED BY

Collection: Wikipedia Eventstream

TIMESTAMPS

The Wayback Machine - https://web.archive.org/web/20201015191616/https://blog.chromium.org/2008/11/

Chromium Blog

News and developments from the open source browser project

Understanding Phishing and Malware Protection in Google Chrome

Friday, November 14, 2008

Google Chrome includes features to help protect users against phishing and malware attacks. If you have ever hit a red page with the title "Warning: Visiting this site may harm your computer!" (such as our test page) or "Warning: Suspected phishing site!" then you have already seen these features in action. While we try to provide an explanation of what's happening on that warning page, a number of people have asked for more information about how this feature works, in terms of where the data behind those warnings come from, how that data gets to the computer, and what privacy implications the feature has.
Where does the phishing and malware data come from?
Google is constantly crawling and re-crawling the web, all the while finding new and changed websites. These websites are found by following links from other websites, crawling URLs submitted by webmasters and users, and so forth. Sometimes, during that process, we discover a website where something doesn't seem right. A website may look like a phishing website, designed to steal your personal information, or it may contain signs of potentially malicious activity that would install malware onto your computer without your consent. If we find a website that looks like it's a phishing page, it gets added to a list of suspected phishing websites. If we find a website that contains signs of potentially malicious activity, we start up a virtual machine, browse to that website, and watch what happens. If we see certain activities happen on that virtual machines (such as viruses being downloaded and installed), we add that website to a list of suspected malware-infected websites. The process for discovering suspected malware-infected websites is described in more detail in a paper written by Niels Provos and colleagues from Google's anti-malware team.
How does this data get to my computer?
If you have phishing and malware protection enabled, then Google Chrome will contact servers at Google within five minutes of startup, and approximately every half hour thereafter, to download updated lists of suspected phishing and malware websites. These lists are then stored on your computer, so that as you browse the web, each page can be checked against the list of suspected phishing and malware websites locally, without sending the address of each webpage you visit to Google. This is designed to offer both performance (by not having to wait on a round-trip request to Google's servers) and privacy (by not sending a record of your browsing session to Google).
As the lists are large (hundreds of thousands of entries), we looked for ways to reduce the amount of information that had to be sent to and stored on users' computers, to reduce the amount of bandwidth and storage space consumed. One way we achieve this is by using partial hashes of URLs in the lists downloaded by the computer. What this means is that rather sending down the full URL of each website, we do the following. First, we hash the URL using SHA-256. Then, we send add the first 32 bits of that 256-bit hash into the list of phishing or malware websites. Those lists of 32-bit hash prefixes are then downloaded by Google Chrome in the background as described earlier.
How is this data used, and what is sent back to Google?
When you browse the web using Google Chrome, the hash of each URL is computed, and the first 32 bits of that URL's hash is compared against the list of suspected phishing and malware websites. This includes the URL of the website you are visiting, as well as the URL of any included resources (such as included JavaScript or Adobe Flash movies). If the first 32 bits of the hash match an entry in the list, it is likely that the URL is on the list of suspected phishing or malware websites. At this point, we can only say likely, because there is still a reasonable chance of hash collisions in the 32-bit space - two distinct URLs with distinct 256-bit hashes where the first 32 bits of those hashes are the same. To confirm that the URL is suspected as a phishing or malware website, and not just a 32-bit hash collision, the 32-bit hash is sent to Google. Google then returns the full 256-bit hashes suspected of being phishing or malware and starting with those 32 bits. The full 256-bit hash of the URL in question can then be compared against the 256-bit hash(es) returned by Google, to make a determination of whether in fact the URL in question is or is not on the list of suspected phishing or malware websites. Using this scheme, Google Chrome is able to quickly check the website and its resources against a local database, and only sends information back to Google when the site matches an entry on the locally stored lists. In the case where information is sent to Google to verify such a suspicion, that information consists only of a part of the hash of a URL, not the URL itself. As such, Google never gets information that would definitively indicate whether a user has visited a particular website or not. The end result is a low-overhead efficient mechanism to help protect against phishing and malware, while also helping to protect users' privacy.
Posted by Ian Fette, Product Manager

Putting It to the Test

Friday, November 7, 2008

Historically, testing hasn't gotten much respect in the world of software development. As the old saying goes, "It compiles! Ship it!" Only a joke — but like most jokes, it hides a grain of truth.

Not so for the Chromium project. Our philosophy is to test everything we possibly can, in as many ways as we can think of.
Test drive: why test?
It's easy to find arguments against testing. Writing tests takes time that developers could be using to write features, and keeping the test hardware and software infrastructure running smoothly isn't trivial. (I'm one of the people largely responsible for the latter for Chromium, along with Nicolas Sylvain, so I know how time-consuming it can be.) But in the long run, it's a big win, for at least two reasons.
A well-established set of tests that developers are expected to run before sending changes in makes it a lot easier to avoid causing problems, which lets other developers stay productive rather than chasing down regressions. And testing submitted changes promptly keeps the code building cleanly and minimizes trouble in the longer term.
But even more importantly, an extensive set of automated tests gives us more confidence that Chromium is reliable, stable, and correct. We're not afraid to rewrite major portions of the code, because verifying correctness afterward is easier. And we have the flexibility to iterate faster and produce releases more often, because we don't need a 6-month QA cycle before each one.
The test of time: performance testing
We run a lot of different tests. Tests of security. Tests of UI functionality. Tests of startup time, page-load speed, DOM manipulation, memory usage. Tests for memory errors using Rational Purify. WebKit's suite of layout tests. Hundreds of unit tests to make sure that individual methods are still doing what they should. At last count, we run more than 9100 individual tests, typically 30-40 times every weekday.^[1] You can find the full list in the developer documentation, but I'll talk more about one broad category here: performance testing.
With every change made in the tree, we keep track of Chromium's page-load time, memory usage, startup time, the time to open a new tab or switch to one, and more. All these data points are available in graphs like this one:

Here the top, gold trace shows the startup time on XP for the tip-of-tree build; the green, bottom trace shows the startup time for a reference build so we can discount variation in the test conditions; and the blue, middle trace shows the startup time along a different code path that includes loading gears.dll. The light blue horizontal line is a reference marker. As you can see, whatever changed between the previous build and r3693, it increased the startup time (gold trace) by more than 8%. The developer responsible was able to see that and fix the problem a few builds later.
This graph also shows the usefulness of running a reference build. The spike in startup time that lasted only a single build also shows up in the reference-build time (the green trace). We can assume that it was something temporarily affecting the build machine, rather than a code change. (The problem must have cleared up by the time the Gears startup test ran.)
With so many performance graphs, it can be hard to watch them all, so there's also a summary page.
One final note about Chromium's performance graphs: they're written in HTML and JavaScript, and we're looking for someone to make them easier to use. If you're interested, grab the code and start hacking!
Test bed: the Chromium buildbot
Nearly all of this testing is controlled by Chromium's buildbot, which automates the build/test cycle. Every time a change is submitted, the buildbot master builds the tree, runs the tests on all the different platforms, and displays the results. For a complete guide to the buildbot and its "waterfall" result page, see the Tour of the Chromium Buildbot in the developer docs.
Pro-test
Of course, once you have lots of tests running, the second important aspect of good tree hygiene is to keep them all passing. But that's a subject for another post.
[1] It's hard to put a single number on it, because certain tests only apply to some parts of the code. But however you count it, it's a lot of tests.
Posted by Pamela Greene, Software Engineer

Labels

Archive

2020
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2019
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2018
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2017
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2016
- Dec
- Nov
- Oct
- Sep
- Aug
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2015
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2014
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2013
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2012
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2011
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2010
- Dec
- Nov
- Oct
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2009
- Dec
- Nov
- Sep
- Aug
- Jul
- Jun
- May
- Apr
- Mar
- Feb
- Jan

2008
- Dec
- Nov
- Oct
- Sep

Feed

Give us feedback in our Product Forums.

Google
Privacy
Terms