azakai

Emscripten Standard Library Support, Now With More C++

2012-01-23T11:28:00.000-08:00

I just landed much more comprehensive support for the C++ standard library in Emscripten, which now allows you to compile pretty much any C++ code using the standard C++ library to JavaScript. So I figured it was a good time to write up an overview of how Emscripten handles standard libraries.

As background, one of the initial design decisions in Emscripten was to focus on generating good code, even when that has some potential downsides elsewhere. Good code means both fast code and small code, both of which are particularly important on the web: While fast code is important everywhere, JavaScript is not yet as fast as native code, so to counter that we need to really focus on generating efficient code, and regarding code size, you might not care much about linking to a 5MB shared library on your desktop, but downloading and parsing a 5MB script is something significant.

For that reason, it didn't seem like a good idea to build C and C++ standard libraries, ship them with your code, and link them at runtime: The standard libraries are quite large. Furthermore, Emscripten doesn't have a single ABI, it has several code generation modes, as one example there are two typed array modes and one mode without typed arrays, and code compiled with one is not interface-compatible with another. Not having a stable ABI lets us generate more specialized and efficient code, but it is another reason for not shipping separate linkable standard libraries.

Instead, when compiling code with Emscripten we build the standard library along with your project's code. Everything is then shipped as a single file. This gives the advantages mentioned before: Smaller code size since we know which parts of the standard library you actually need, and faster code since we can specialize both the standard library and your own code, not just your own.

However, there are two disadvantages of this approach. The first is that combining the standard libraries with your own code means they form a single "unit". When normally you build your code and then link to the LGPL-licensed GNU standard libraries, it's clear that your code does not need to comply with the LGPL (you just need to comply with the LGPL regarding the LGPL'd library itself). But if you build your project together with the standard library, intertwining them in an optimized way, it is less clear how the LGPL applies here. I actually don't think there is a problem - it seems equivalent to the former "normal" case to me, despite the differences between them - however, I am not a lawyer, and also it is better to avoid any possible confusion and concern. In addition, even if the LGPL still applies just to the library, you would be shipping the library yourself, meaning you need to comply with the LGPL for it (which I don't think is a problem myself, but it is a concern for other people). For those reasons, Emscripten doesn't include any LGPL code. That ruled out using the GNU standard C and C++ libraries, which would otherwise be the first choice because of their familiarity and compatibility with existing code.

Given that decision, I looked at the other options and decided to use the Newlib C library. There are then two options: Use just the Newlib headers, or use the existing Newlib implementation code, porting it to the new platform. I decided to use just the headers, because (1) Newlib is already not 100% compatible with the GNU C library, so there would anyhow be inconsistencies and missing parts we would need to work around, and easier to do so in our own new code, (2) By implementing the C standard library in JavaScript, we can optimize it using the existing capabilities of the web platform (for example, we use JavaScript's sort inside qsort), and (3) Porting Newlib to the web would mean writing new C code inside Newlib, and interfacing that with JavaScript code that hooks into the web platform APIs themselves, which is a little less convenient than writing just JavaScript, and finally (4) Porting a C standard library means working with the internals of that library, as opposed to implementing the familiar C standard library interface which is higher-level. So, Emscripten has an implementation of the C standard library written in JavaScript, primarily using the Newlib headers. (The only exception is malloc and free, which we compile from dlmalloc, because writing an effective malloc implementation is not trivial. However, we should implement malloc and free in JavaScript eventually since we could optimize it quite a bit.)

For C++, again we couldn't use the GNU C++ standard library. Instead we started out with the libc++ headers. Just using the headers was enough to get a lot of code to run, because a lot of the functionality is in the headers themselves. We did need to introduce some ugly hacks in the headers though, as well as implement some bits in JavaScript. This was enough for a lot of projects to work, but was still missing a lot of stuff, almost everything that wasn't implemented in a header.

That problem is what I worked on fixing last week: As of Saturday, we will build the libc++ sources, if they are needed by your project, and include them. To get this to be efficient, I also enabled LLVM's global dead code elimination, so that while we link in the entire C++ standard library, we immediately eliminate all the parts you don't actually need before proceeding to compile to JavaScript. This helps quite a lot with the size of the generated code (and also is nice for compilation times). Aside from that, the main challenge here was getting libc++ to build using the Newlib C standard library headers. The end result is that all of our hacks in the libc++ headers are now removed, we now build stock libc++ and include it if necessary, which means pretty much any C++ program that uses the C++ standard library should work. (With the obvious caveats of no multithreading with shared state and so forth, which are general limitations of compiling to JavaScript.)

I mentioned before that there were two downsides to building the standard libraries with your project, the first of which was licensing, which led us to avoid LGPL code. The other disadvantage is build time: You compile the standard library into JavaScript when you build your project, as opposed to just building your own project and linking it with prebuilt standard libraries. While I believe this is definitely worth the advantages of the approach (faster and smaller generated code), it is a concern. We get around a lot of the problem by (1) having the C standard library implemented in JavaScript, so there is no compilation time for it, (2) using LLVM's dead code elimination to quickly get rid of parts of the C++ standard library (and dlmalloc, as mentioned before that we also build, if it is used) early in the compilation process, (3) only linking in the C++ standard library (and dlmalloc) if they are actually used, and (4) caching the bitcode result of compiling the C++ standard library (and dlmalloc), so that we only compile it once to bitcode. With these in place, while emcc is still slower than gcc, it isn't very significant, except for the very first time you compile libc++ from source into bitcode (which as mentioned before, is done once and then cached).

New Emscripten tutorial: C/C++ to JavaScript now easier than ever with "emcc"

2011-12-20T11:47:00.000-08:00

A new compiler frontend for Emscripten, emcc, has landed recently. emcc can be used basically as a drop-in replacement for gcc, making it much easier to compile C and C++ into JavaScript. For example,

emcc src.cpp

will generate a.out.js, and

emcc src.cpp -o src.html

will generate a complete HTML file with the compiled code as embedded JavaScript, including SDL support so the code can render to a Canvas element. Optimizing code is now easy as well,

emcc -O2 src.cpp

will generate optimized code (optimizing in LLVM, the Emscripten compiler itself, the Closure Compiler, and the Emscripten JavaScript optimizer). (Note that there is an even faster setting, -O3, see the docs for more.)

emcc is presented in more detail in the new Emscripten Tutorial. Check it out! Feedback is welcome :)

Typed Arrays by Default in Emscripten

2011-12-10T20:36:00.000-08:00

Emscripten has several ways of compiling code into JavaScript, for example, it can use typed arrays or not (for more, see Code Generation Modes). I merged the 'ta2 by default' branch into master in Emscripten just now, which makes one of the typed array modes the default. I'll explain here the reason for that, and the results of it.

Originally Emscripten did not use typed arrays. When I began to write it, typed arrays were supported only in Firefox and Chrome, and even there they were of limited benefit due to lack of optimization and incomplete implementation. Perhaps more importantly, it was not clear whether they would ever be universally supported in all browsers. So to generate code that truly runs everywhere, Emscripten did not use typed arrays, it generated "plain vanilla" JavaScript.

However, that has changed. Firefox and Chrome now have mature and well-performing implementations of typed arrays, and Opera and Safari are very close to the same. Importantly, Microsoft has said that IE10 will support typed arrays. So typed arrays are becoming ubiquitous, and have a bright future.

The main benefits of using typed arrays are speed and code compatibility. Speed is simply a cause of JS engines being able to optimize typed arrays better than normal ones, both in how they are laid out in memory and how they are accessed. Compatibility stems from the fact that by using typed arrays with a shared buffer, you can get the same memory behavior as C has, for example, you can read an 8-bit byte from the middle of a 32-bit int and get the same result C would get. It's possible to do that without typed arrays, but it would be much, much slower. (There is however a downside to such C-like memory access: Your code, if it was not 100% portable in the first place, may depend on the CPU endianness.)

Because of those benefits, I worked towards using typed arrays by default. To get there, I had to fix various problems with accessing 64-bit values, which are only a problem when doing C-like memory access, because unaligned 64-bit reads and writes do not work (due to how the typed arrays API is structured). The settings I64_MODE and DOUBLE_MODE control reading those 64-bit values: If set to 1, reads and writes will be in two 32-bit parts, in a safe way.

Another complication is that typed arrays cannot be resized. So when sbrk() is called to a value that is larger than the max size, we can't easily enlarge the typed arrays we are using. The current implementation will create new typed arrays and copy the old values into them, which will work but is potentially slow.

Typed arrays have already worked in Emscripten for a long time (in two modes, even, shared and non-shared buffers), but the issues mentioned in the previous two paragraphs limited their use in some areas. So the recent work has been to smooth over all the missing pieces, to make typed arrays ready as the default mode.

The current default in Emscripten, after the merge, is to use typed arrays (in mode 2, with a shared buffer, that is, C-like memory access), and all the other settings are set to safe values (I64_MODE and DOUBLE_MODE are both 1), etc. This means that all the code that worked out of the box before will continue to work, and additional code will now work out of the box as well. Note that this is just the defaults: If your makefile sets all the Emscripten settings itself (like defining whether to use typed arrays or not, etc.), then nothing will change.

The only thing to keep in mind with this change is that by default, you will need typed arrays to run the generated code. If you want your code, right now, to run in the most places, you should set USE_TYPED_ARRAYS to 0 to disable typed arrays. Another possible issue is that not all JS console environments support typed arrays: Recent versions of SpiderMonkey and Node.js do, but the V8 shell has some issues (note that this is just a problem in the commandline shell, not in Chrome), so if you test your generated code using d8 then it will not work. Instead, you can test it in a browser, or by using Node.js or the SpiderMonkey shell for now.

Emscripten in node.js and on the web

2011-12-05T15:17:00.000-08:00

Until now, to use Emscripten to compile LLVM to JavaScript you had to install a JavaScript engine shell (like SpiderMonkey's or V8's), both to run Emscripten itself and to run the generated code. This meant you had to get the latest source code of one of those shells and build it, which isn't hard but isn't super convenient either. So over the weekend I landed support for running Emscripten itself in node.js and in web browsers, as well as support for running the generated code in node.js (it always ran in browsers).

What this means is that if you have node.js, Python and Clang, you have everything you need to use Emscripten. For more, see the updated Getting Started page. (Regarding running Emscripten itself in a web browser, see src/compiler.html. This isn't really intended as a serious way to use it, but there are some interesting use cases for it, or will be.)

It is still strongly recommended to install the JavaScript engine shells themselves, though. One reason is the trunk engine shells are the very latest code, so to see the maximum speed code can run you should use them. Also, some tests require the SpiderMonkey shell because the others do not yet fully support the latest typed arrays spec. But, if you already have node.js installed anyhow, it is now easier to use Emscripten because you can just use that.

Code Size When Compiling to JavaScript

2011-11-15T11:10:00.000-08:00

When compiling code to JavaScript from some other language, one of the questions is how big the code will be. This is interesting because code must be downloaded on the web, and large downloads are obviously bad. So I wanted to investigate this, to see where we stand and what we need to do (either in current compilers, or in future versions of the JavaScript language - being a better compiler target is one of the goals there).

The following is some preliminary data from two real-world codebases, the Bullet physics library (compiled to JavaScript in the ammo.js project) and Android's H264 decoder (compiled to JavaScript in the Broadway project):

Bullet

.js 19.2 MB
.js.cc 3.0 MB
.js.cc.gz 0.48 MB

.o    1.9 MB
.o.gz 0.56 MB

Android H264

.js 2,493 KB
.js.cc 265 KB
.js.cc.gz 61 KB

.o      110 KB
.o.gz    53 KB

Terms used:

.js Raw JS file compiled by Emscripten from LLVM bitcode

.js.cc    JS file with Closure Compiler simple opts
.js.cc.gz JS file with Closure, gzipped

.o    Native code object file
.o.gz       Native code object file, gzipped

Notes on methodology:

Native code was generated with -O2. This leads to smaller code than without optimizations in both cases.
Closure Compiler advanced optimizations generate smaller JS code in these two cases, but not by much. While it optimizes better for size, it also does inlining which increases code size. In any case it is potentially misleading since its dead code elimination rationale is different from the one used for LLVM and native code, so I used simple opts instead.
gzip makes sense here because you can compress your scripts on the web using it (and probably should). You can even do gzip compression in JS itself (by compiling the decompressor).
Debug info was not left in any of the files compared here.
This calculation overstates the size of the JS files, because they have the relevant parts of Emscripten's libc implementation statically linked in. But, it isn't that much.
LLVM and clang 3.0-pre are used (rev 141881), Emscripten and Closure Compiler are latest trunk as of today.

Analysis

At least in these two cases it looks like compiled, optimized and gzipped JavaScript is very close to (also gzipped) native object files. In other words, the effective size of the compiled code is pretty much the same as you would get when compiling natively. This was a little surprising, I was expecting to see the size be bigger, and to then proceed to investigate what could be improved.

Now, the raw compiled JS is in fact very large. But that is mostly because the original variable names appear there, which is basically fixed by running Closure. After Closure, the main reason the code is large is because it's in string format, not an efficient binary format, so there are things like JavaScript keywords ('while', for example) that take a lot of space. That is basically fixed by running gzip since the same keywords repeat a lot. At that point, the size is comparable to a native binary.

Another comparison we can make is to LLVM bitcode. This isn't an apples-to-apples comparison of course, since LLVM bitcode is a compiler IR: It isn't designed as a way to actually store code in a compact way, instead it's a form that is useful for code analysis. But, it is another representation of the same code, so here are those numbers:

Bullet

.bc 3.9 MB
.bc.gz 2.2 MB

Android H264

.bc 365 KB
.bc.gz 258 KB

LLVM bitcode is fairly large, even with gzip: gzipped bitcode is over 4x larger than either gzipped native code or JS. I am not sure, but I believe the main reason why LLVM bitcode is so large here is because it is strongly and explicitly typed. Because of that, each instruction has explicit types for the expressions it operates on, and elements of different types must be explicitly converted. For example, in both native code and compiled JS, taking a pointer of one type and converting it to another is a simple assignment (which can even be eliminated depending on where it is later used), but in LLVM bitcode the pointer must be explicitly cast to the new type which takes an instruction.

So, JS and native code are similar in their lack of explicit types, and in their gzipped sizes. This is a little ironic since JS is a high level language and native code is the exact opposite. But both JS and native code are pretty space-efficient it turns out, while something that seems to be in between them - LLVM bitcode, which is higher than native code but lower than JS - ends up being much larger. But again, this actually makes sense since native code and JS are designed to simply execute, while LLVM bitcode is designed for analysis, so it really isn't in between those two.

(Note that this is in no way a criticism of LLVM bitcode! LLVM bitcode is an awesome compiler IR, which is why Emscripten and many other projects use it. It is not optimized for size, because that isn't what it is meant for, as mentioned above, it's a form that is useful for analysis, not compression. The reason I included those numbers here is that I think it's interesting seeing the size of another representation of the same compiled code.)

In summary, it looks like JavaScript is a good compilation target in terms of size, at least in these two projects. But as mentioned before, this is just a preliminary analysis (for example, it would be interesting to investigate specific compression techniques for each type of code, and not just generic gzip). If anyone has additional information about this topic, it would be much appreciated :)

JSConf.eu, Slides, SQLite on the Web

2011-10-07T13:38:00.000-07:00

I got back from JSConf.eu a few days ago. I had never been to JSConf before, and it was very interesting! Lots of talks about important and cool stuff. The location, Berlin, was also very interesting (the mixture of new and old architecture in particular). Overall it was a very intensive two days, and the organizers deserve a ton of credit for running everything smoothly and successfully.

I was invited to give a talk about Emscripten, the LLVM to JavaScript compiler I've been working on as a side project over the last year. Here are my slides from the talk, links to demos are in them. There was also a fourth unplanned demo which isn't in the slides, here is a link to it.

If you've seen the previous Emscripten demos, then some of what I showed had new elements, like the Bullet/ammo.js demo which shows the new bindings generator which lets you use the compiled C++ objects directly from JS in a natural way. One demo was entirely new though, SQLite ported to JS. I haven't had time to do any rigorous testing of the port or to optimize it for speed. However it appears to work properly in all the basic tests I tried: creating tables, doing selects, joins, etc. With WebSQL not moving forward as a web standard, compiling SQLite to JS directly might be the best way to get SQL on the web. The demo is just a proof of concept, but I think it shows the approach is feasible.

Long-term support for Firefox?

2011-06-25T22:13:00.000-07:00

A debate is currently ongoing regarding long-term support of Firefox, with some surprised that Firefox 4 is being EOL'ed, and what that means for the viability of Firefox in the enterprise, if Mozilla is going to release a new version every 6 weeks.

Without stating an opinion either way (since I don't have one), there is one potential solution here that I haven't seen mentioned enough. The whole point of open source is that anyone, anywhere, can take the source code and customize it for any purpose they see fit. If there is indeed a need for long-term support for Firefox in the enterprise (and again, I am voicing no opinion on the matter), then anyone can do that. It doesn't need to be Mozilla. Anyone can form a new company, do the work to backport security fixes and QA them, and sell long-term support for Firefox (or this could be done in various other ways).

This is possible since Firefox is 100% open source. It isn't possible with proprietary software like IE, and isn't possible with software that mixes the two (like Chrome: you can sell long-term support for Chromium, but it will not have Chrome-only features like print preview, etc., so it would be a different product).

I want to stress the fact that you can sell open source software. Complying with Firefox's license doesn't preclude that. I won't get into the details, but this model works fine, just ask Red Hat. In fact I assume Red Hat already does exactly this, it sells supports for old versions of Firefox as part of Red Hat Enterprise Linux for a very long time.

Mozilla is driving Firefox forward as fast as possible, but Mozilla doesn't 'own' Firefox in the sense that it is the only party that can do things with it. Anyone can. If there is a business need for long-term support for Firefox, anyone can serve that need by selling that support.

Note that this is not a case of me saying "if you want some feature, you can always do it yourself". The situation we are talking about here is enterprise users. Big corporations do pay for support for the software they use, it is worthwhile for them to pay for such support. My point is that open source software like Firefox fits into this model perfectly - if there is a business need, anyone can step in and fill that need.

Again, I don't have an opinion myself as to whether such support is important - I don't know enough about the enterprise software market to have such an opinion. I also have nothing to do with Mozilla policy and planning, I just write code. My only point is that Firefox is open source, and that means it is business-friendly in the sense that you are not under the control of a single vendor selling you a product: You can customize the software yourself, or you can get someone else to do it for you. As an open source advocate, I wanted to point this out, since Firefox is the only major browser that is 100% open source, and that means there are solutions to a situation where people say there is a need for something, but Mozilla does not currently do that thing.

Device Orientation API Changes

2011-06-08T11:47:00.000-07:00

We recently moved from our temporary Mozilla-specific onMozOrientation API for device orientation events - which let you detect what angle you are holding your mobile phone, for example - to the W3C DeviceOrientation spec.

The main difference in the orientation data is that onMozOrientation returned an (x,y,z) vector, whereas the W3C spec returns (alpha,beta,gamma) Euler angles.

The easiest way to understand what those mean is to look at the examples in the W3C spec linked to before (search for "The following code extracts illustrate basic use of the events"). For now on Android we don't give the alpha value, which is the compass heading (azimuth). You can use the beta and gamma values though. Basically, if the user holds the device flat on a surface pointing up, then tilting it towards the user or away changes beta, and tilting to the right or to the left changes gamma.

We hope to implement the rest of the spec soon.

No more stackQuota()

2011-06-03T20:52:00.000-07:00

If you use SpiderMonkey, and have had to run code like this:

js -e "stackQuota(100000000)" myFile.js

Then as of Bug 644241 landing on tracemonkey, calling stackQuota is no longer necessary (it will cause an error, as stackQuota no longer exists).

For more details see the bug. In brief summary, the script stack quota originally limited the amount of memory scripts could use, but its effectiveness diminished over time, while it has been causing more problems as people run larger JavaScript files and keep hitting the quota (we had a bug in Firefox, then another bug for worker threads, and as mentioned above it was necessary to call stackQuota in the console). So the script stack quota has been removed. A followup bug has plans to introduce a new way to limit the amount of memory scripts can use.

Rendering PDFs in JavaScript...?

2011-04-11T16:33:00.000-07:00

I released Emscripten 1.0 over the weekend, which came with a demo of rendering PDFs entirely in JavaScript (warning: >12MB will be downloaded for that page). Emscripten is an LLVM-to-JavaScript compiler which allows running code written in C or C++ on the web. In the linked demo, Poppler and FreeType were compiled to JavaScript from C++.

The goal of the demo was to show Emscripten's capabilities. Over the last year it has gotten very usable, and can probably compile most reasonable C/C++ codebases (albeit with some manual intervention in some cases). It is my hope that Emscripten can help against the tendency to write non-web applications, such as native mobile applications (for iOS, Android, etc.) or using plugins on the web (Flash, NaCl, etc.). Simply put, the web is competing with these platforms. Emscripten can make the web a more attractive platform for developers, by letting them use their languages of choice, such as C, C++ or Python (without necessarily compromising on speed: the code generated by Emscripten can be optimized very well, and it is my hope that things like type inference will make it very fast eventually).

Meanwhile, getting back to the PDF rendering demo, I was thinking: How about making a Firefox plugin with it, that is, that when a PDF is clicked in Firefox it is shown in an internal PDF viewer? Aside from the novelty, I think this would be cool to do because it would be an extremely secure PDF viewer (since it would be entirely in JavaScript). If you are a plugin or frontend hacker and think it's a cool idea too, please get in touch and let's make it happen! :)

massdiff - Diff for Massif Snapshots

2011-03-18T13:46:00.000-07:00

Massif (part of Valgrind) is a super-useful tool to find out how a program allocates memory. It gives very detailed graphs about exactly which lines of code do allocations that are not free'd later. For example,

96.42% (22,771,482B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->39.75% (9,388,032B) 0x8058AC1: arena_malloc_large (jemalloc.c:3831)
| ->39.75% (9,388,032B) 0x8058DB9: arena_malloc (jemalloc.c:3856)
|   ->34.74% (8,204,288B) 0x8058E98: imalloc (jemalloc.c:3866)
|   | ->34.27% (8,093,696B) 0x805D833: malloc (jemalloc.c:5882)
|   | | ->12.23% (2,887,680B) 0x6BBB63F: sqlite3MemMalloc (sqlite3.c:14221)

...

This shows a snapshot at a particular point in time, during which the whole program allocated ~22MB of memory, of which almost 3MB was due to SQLite.

One limitation though is you can't easily see what changes from one snapshot to another, which is important in order to see if there are any gradual memory increases (these may or may not be actual leaks). So I wrote a small python script, massdiff, which diffs two Massif snapshots. Here is an example of the output:

Diffing snapshots 30 50 

- (heap allocation functions) malloc/new/new[], --alloc-fns, etc. - 22,286,738
    [ diff: +502,088 ]
+ (heap allocation functions) malloc/new/new[], --alloc-fns, etc. - 22,788,826 

-   0x8058927: arena_malloc_small (jemalloc.c:3794) -  7,494,594
      [ diff: +499,552 ]
+   0x8058927: arena_malloc_small (jemalloc.c:3794) -  7,994,146 

-     0x8058D9D: arena_malloc (jemalloc.c:3854) -  7,494,594
        [ diff: +499,552 ]
+     0x8058D9D: arena_malloc (jemalloc.c:3854) -  7,994,146 

-       0x8058E98: imalloc (jemalloc.c:3866) -  6,055,702
          [ diff: +499,232 ]
+       0x8058E98: imalloc (jemalloc.c:3866) -  6,554,934 

-         0x805D833: malloc (jemalloc.c:5882) -  5,552,542
            [ diff: +499,232 ]
+         0x805D833: malloc (jemalloc.c:5882) -  6,051,774 

-           0x52D2FA3: js_malloc (jsutil.h:213) -  1,635,252
              [ diff: +432,064 ]
+           0x52D2FA3: js_malloc (jsutil.h:213) -  2,067,316 

-             0x52D9D27: JSRuntime::malloc(unsigned int, JSContext*) (jscntxt.h:1358) -  1,635,252
                [ diff: +432,064 ]
+             0x52D9D27: JSRuntime::malloc(unsigned int, JSContext*) (jscntxt.h:1358) -  2,067,316 

-               0x52D9DEC: JSContext::malloc(unsigned int) (jscntxt.h:2027) -  1,600,532
                  [ diff: +431,904 ]
+               0x52D9DEC: JSContext::malloc(unsigned int) (jscntxt.h:2027) -  2,032,436 

-                 0x5D5B41D: JSObject::allocSlots(JSContext*, unsigned int) (jsobj.cpp:4032) -    161,456
                    [ diff: +344,832 ]
+                 0x5D5B41D: JSObject::allocSlots(JSContext*, unsigned int) (jsobj.cpp:4032) -    506,288 

-                   0x5D5B594: JSObject::growSlots(JSContext*, unsigned int) (jsobj.cpp:4078) -    161,456
                      [ diff: +344,832 ]
+                   0x5D5B594: JSObject::growSlots(JSContext*, unsigned int) (jsobj.cpp:4078) -    506,288 

The diff only shows what changed between the two revisions, and shows that in in tree format, just like Massif's ms_print does. Each group of three lines shows a line from the first snapshot on top, the corresponding line from the later snapshot on bottom, and between them the difference.

The output here is from loading about:blank 500 times in Fennec. There is overall ~500K of additional allocation (so, ~1K per page load), of which JSObject::growSlots is responsible for ~340K (this is later all deallocated at once, presumably due to GC being run).

So far this has been useful in helping discover one specific case, bug 641663, and I'm still investigating some additional issues. Hopefully it can be a useful tool for other people too. To use it,

Run Massif on your program, see here and here
Run ms_print on the output
Run massdiff on that file, with the two snapshot numbers you want to diff as parameters

High-Level Fennec Profiling

2011-02-16T10:32:00.000-08:00

I've been working on a series of patches to let us do high-level profiling on Fennec (and Firefox). The goal is to get a "big picture" view of what processing happens in Fennec, so we can investigate what should be optimized. More specifically, the idea is to see what events, runnables, and IPC messages are run, and how much time is spent on each.

Here is some example data, from ~9 seconds of panning and zooming on the Mozilla crash stats page, in the parent process. Notes:

Event 10 is a mouse event, and 37 is a simple gesture. So these are events that are triggered by the panning and zooming actions. A lot of these events happen, and they can take up to 1/20th of a second to process.
I am not sure why the RefreshDriver is called here (since this is the parent process). Perhaps worth looking into.
Layers Update IPC messages are working very well, with a mean processing time of 0.0004 seconds.

And here is some data from loading google.com (nonmobile). Notes:

AsyncAssociateIconToPage and InsertVisitedURIs take a while, but they are at least on a side thread.
RecvGetValue is web storage. We are considering what to do with that in bug 627635.
nsGlobalWindow.cpp:8834 is likely a setTimeout or Interval that google.com created.
We receive 39 RecvHTMLDNSPrefetch IPC messages. Offhand I don't know why so many would occur on a blank search page. Filed bug 634653.

So, the data shows on a high level what code is run, unlike a low-level profiler (like oprofile) that shows how much individual functions or lines of code take. Low-level profilers are very useful of course, but sometimes it is also good to get higher-level data (and it isn't always possible or easy to deduce that from the low-level data). Also, running low-level profilers depends on the OS, so you need one to be available for your OS (a problem on Android - but see the very cool piranha), and even if you do have one, your data is for that OS only, and not necessarily directly comparable to data from a different low-level profiler on a different OS (in other words, you are changing both profiler and OS, and not just OS).

Of course this approach has limitations as well, so a combination of both approaches is the best thing.

The patches are as follows:

Timer logger: A script that automatically rewrites the Fennec source code to add identifying strings to all timers, and code to generate log output for that. This is an improvement of my previous IPC profiling patch.
Runnables and events logger: Another rewriting script, that does something similar for Runnables. Also adds log output for Runnables and Events.
IPC traffic logger: A few hooks to generate log output for each IPC message.
Android log processer: Tiny script that splits an android adb logcat dump into separate files for each process, that are ready for further processing. This is only needed on Android.
Data analyzer: Processes the generated logs and creates readable output (like the data shown above)

To use the patches, apply them, then run timer_rewriter.py and runnable_rewriter.py. Build Fennec (or Firefox), then run it, and capture the log output into a file. You should have a separate file for each process; on Android, use the android_prettifier.py script mentioned above. Then run the analyze_profiling_output.py script on the data to get the summary.

Visualizing IPC Messages in Fennec

2010-09-23T15:46:00.000-07:00

More progress on IPC profiling tools, this time a visualization generator. Here is what happens during loading of cnn.com:

On the left you see Fennec before dougt's prefs optimizing patch, and on the right after it. On both sides, the data is as follows:

Each line is a single IPC message. Messages are ordered by send time. The width of the line is how long it took until the message was received. The X axis is linear in time.
The Y axis also progresses forward in time (since each line is one message, and they are ordered). But it isn't linear (a lot of short messages will take a lot of vertical space, even though little time passed). The numbers show how much time actually passed. So the Y axis really indicates how many messages have been sent.
Colors indicate the type of message: green = http, red = prefs, blue = cookies, black = other.

That patch basically removes all the prefs messages, and the result is a 12% speedup (yay!) There were a lot of prefs messages, and even though they were very short and fast to process, they were synchronous (i.e., blocking).

The visualization tool can also show messages for the child and parent process separately (in the image above, they are all shown together).

Instructions for using it:

Apply the patch.
Run Fennec with something like this (from $objdir/dist/bin):

MOZ_IPC_MESSAGE_LOG=1 MOZ_REDIRECT_CHILD_STDOUT=co MOZ_REDIRECT_CHILD_STDERR=ce ./fennec www.cnn.com &> o

Run the visualizer with something like this (you need the Python Imaging Library installed):

python ../../../mozilla-central/tools/ipcprof/profiler.py o ce

Images should have been created, parent.png, child.png, both.png.

A perhaps more easily viewable image can be found here.

Profiling IPC Messages in Fennec

2010-09-15T14:36:00.000-07:00

Bug 596725 has a Python script that can profile the IPC messages log. Here's the output when run on icanhascheezburger: link. Some first impressions:

A few sync/rpc methods are used quite frequently, like preferences, cookies, permissions. Thankfully these complete very quickly. Still this might be worth optimizing though?
Lots of http is going on, so lots of http messages. Maybe nothing we can do about that?

Emscripten Updates

2010-09-12T09:36:00.000-07:00

Minor note: Updates about Emscripten will be on my other blog - fits in better with the theme over there. Though I do believe the two should converge, when all is ready.

Tools For Debugging Wakeups

2010-09-09T10:30:00.000-07:00

Removing unnecessary CPU wakeups in Fennec is very important, in order to reduce the amount of power used. The main tool used for that is powertop, which basically tells you how many wakeups each process has. But how can you tell what specifically in your app is causing those wakeups?

There doesn't seem to be a simple solution for that. One idea is to use the debugger, manually. Perhaps that can be automated using debugger scripts, but I'm not sure, and in any case it would be very specific to the debugger and OS.

So I've tried a different route - automatic rewriting of the source code. I've got it sort of working so far. The idea is this:

First you apply this patch. It modifies some critical files, in particular in the condition variable handling code and in the timer thread code. It adds a parameter to both, so that when you call them, you must identify (by a string) who you are.
Then, you run python timeout_rewriter.py (that script is included in the patch). This goes through the entire codebase, rewriting all the calls to condition variables (PR_WaitCondVar etc.) and timer creation (->initWithCallback etc.) to add that parameter. It modifies both C++ and JS code. Over 100 files are modified. How it does the rewriting is sort of hackish: It doesn't use anything fancy, just some heuristics derived from trial and error ;)
Then you build Firefox normally, and run it. All timeouts will then be printed with explanations, for example:

-------------------------------------------------

WaitCondVar: ./js/src/xpconnect/src/xpcjsruntime.cpp:820 (0.818000 seconds since previous; timeout: 1000)
WaitCondVar: ./js/src/xpconnect/src/xpcjsruntime.cpp:820 (1.000000 seconds since previous; timeout: 1000)
TimerThread: ./dom/base/nsJSEnvironment.cpp:3733
WaitCondVar: ./xpcom/threads/TimerThread.cpp:347 (0.000000 seconds since previous; timeout: 997)
WaitCondVar: ./js/src/jstask.cpp:95 (0.058000 seconds since previous; timeout: -1)
WaitCondVar: ./js/src/jstask.cpp:95 (0.045000 seconds since previous; timeout: -1)
WaitCondVar: ./js/src/xpconnect/src/xpcjsruntime.cpp:820 (0.825000 seconds since previous; timeout: 1000)
TimerThread: ./content/events/src/nsEventStateManager.cpp:793
WaitCondVar: ./xpcom/threads/TimerThread.cpp:347 (0.000000 seconds since previous; timeout: 329)
TimerThread: ./mobile/components/SessionStore.js:345
WaitCondVar: ./xpcom/threads/TimerThread.cpp:347 (0.329000 seconds since previous; timeout: 532)
-------------------------------------------------

Explanations:

The 1st line shows that the JS watchdog thread woke up - you can just look at the source line mentioned to see what it is. Additional info is that the previous CPU wakeup was ~0.8 seconds before it, and that this particular call to WaitCondVar was with a timeout of 1 second (so, it would like to be woken up again in 1 second).
The 2nd line shows the same place waking up, after 1 second.
The 3rd line shows the DOM GC timer being fired. The next line is the condition variable that the timer thread uses.
The 5th line is the JS background thread.
etc.

I might write some tools to parse the output (like profilers do), but so far it already seems useful. Some conclusions: When the browser is idle, we have two 1/5 second timers, the IdleService (should be fixed soon), and nsEventStateManager (need to look into that).

The reasons for rewriting the source code automatically are that, as mentioned, it's over 100 files - so a lot to do by hand. But also, we don't want those changes to go into our actual tree (even with ifdefs, it would be horrible), and maintaining such a big patch as the actual tree changes would be very problematic. So automatic source code rewriting seemed the best approach. Just remember to throw away all those changes, and not commit them by mistake ;)

iX script

2010-09-08T13:48:00.000-07:00

I wrote a small script for investigating xpconnect objects at runtime, iX (investigate Xpconnect).

If you paste that code in a JavaScript file, and run iX(something), it will dump all the info it can find about the object: all regular JavaScript properties, as well as all valid interfaces that you can QueryInterface and GetInferface on that object, and also the node hierarchy. And it tries not to crash when doing so.

(There is probably already something to do this that I am not aware of.)

Mozilla Try Servers on Demand?

2010-08-03T12:46:00.000-07:00

Has having try servers run on demand, say in Amazon EC2, been discussed in the past? Seems like it might make sense. We'd be able to have as many try servers as we need (especially important when there is a lot of load, like before a code freeze - but also in general, to reduce wait times). And we'd be able to not run any try servers when we don't need them.

I guess it can't handle all cases - EC2 can't run all OSes, and the machines are servers, so no normal screen rendering, etc. Also their hardware and performance might not be standardized enough for performance results to be informative. But certainly for building and for a lot of the tests (xpcshell, etc.), it seems like it could speed development up. Maybe building could be sent to EC2 servers on demand, and the results sent back to normal servers for the tests?

Power Update

2010-08-02T16:32:00.000-07:00

An update on power-related stuff:

Patch to remove the 1/sec JavaScript timer should be pushed soon. This is one of the biggest sources of wakeups, and can drain your battery over time, so fixing this is very cool.
Patch to fix unnecessary image animation timers awaiting review. On certain websites, this can leave many, many useless timers, so on those websites this is even more important than the previous point.
Still working on patch for grouping timers - rewrote it with a new and better approach, works well locally but seeing some odd oranges on try server, not sure if random or not. Once this works, it will reduce the # of wakeups due to animations, probably by something like 10-20%
Patch with a nice scriptable API for freezing/throttling the content process waiting for feedback. Two main uses:

Freezing the content process when the OS tells us to. Still working out what the various OSes want us to do and when, see wiki page here.
Throttling websites that use too much power. As a first iteration of this idea, I was thinking about an addon that has a blacklist of websites that are known CPU hogs, and how much to throttle them.

Not strictly a power-related issue, but preloading the browser with SIG_STOP/CONT is also being worked on. What is really nice is this lets the browser be started almost instantly from the user's perspective, while being frozen until that time prevents it from using any power before that. Still need to work out the details of this though.

Experiments with 'Static' JavaScript: As Fast As Native Code?

2010-07-13T20:51:00.000-07:00

I don't really know much about computer language theory. I just mess around with it in my spare time (weekends mostly). Here is one thing I've been thinking about: I want the speed of native code on the web - because I want to run things like game engines there - but I don't want Java, or NaCl, or some plugin. I want to use standard, platform-agnostic web technologies.

So, how about if we could compile JavaScript into native code? Of course, in general we can't, at least not very well. But if a JavaScript program - or part of a program - happens to be implicitly statically typed, and in other ways 'performance-friendly', then we should be able to compile at least such code and run it very quickly.

In fact PyPy does basically that with RPython - a subset of Python that it can translate into C. So, building on that, I made a demo of the following process:

Begin with some benchmark in JavaScript
Parse the JavaScript using the SpiderMonkey parser (a small hack of the code was needed in order to get it to dump the syntax tree in a nice format)
Translate the parsed code into Python (using a Python script, see below)
If the generated code happens to be RPython, happily run that through PyPy's RPython translator to get native code, and execute that

The code I wrote for this is horribly ugly, but if you must look at it, here it is (if I decide this idea is worth doing any more work on, I will almost certainly rewrite it from scratch). Here are the results for the fannkuch benchmark (run on the value n=10, on a Core 2 Duo laptop):

The first row in the chart shows a C++ implementation of fannkuch, compiled with -O3. All the other rows start with a JavaScript implementation and proceed from there. First there is simply the result of running V8 and SpiderMonkey on the benchmark. Then, the results of automatically converting the JavaScript to Python and running CPython on it are shown - very slow. Finally, that Python is run through PyPy's RPython translator, which generates native code, which runs very fast.

The final result is that the JavaScript => Python => PyPy pipeline runs the code only 26% slower than a C++ implementation of the same benchmark compiled at -O3. For comparison, V8 run on that same JavaScript is 3.76 times slower. So, it seems there is potential here to get (some) JavaScript running very fast, pretty much as fast as possible.

Of course, there is a big startup cost here - the conversion process takes a lot of time (at least PyPy is fun to watch as it runs ;). And JavaScript on the web needs to start up quickly. But there should be technical solutions for this, for example, running the code normally while trying the conversion process in a separate thread (so, running on another CPU core). If the process finishes successfully, swap out the code for the faster version (or if you can't hot-swap it, at least remember and use the compiled version next time you visit that website - would still help for websites you visit a lot).

So, as I said this is just a little demo I worked on in my spare time, to see if it can even work, and to learn more about the topic. It seems to sort of work actually (I was not very optimistic before I started), but I'm still unsure if it makes sense to do. Feedback is welcome.

(I also have a few other experiments I'm working on, that are closely related, and involve LLVM. I'll leave those for a future post.)

Edit: Jernej asked in a comment about Rhino. So for comparison I also ran this code in Rhino and Jython. Rhino took 22.51 seconds - more than twice as slow as SpiderMonkey - and Jython took 74.53 seconds, closer to the slowness of CPython than the next slowest result (Rhino). Neither of these - Rhino or Jython - is particularly fast in general, but they do give other significant benefits (integration with the JVM, proper multithreading, security, etc.).

Edit 2: Following comments by Sayre here and dmandelin elsewhere, here are the results of JaegerMonkey on this benchmark: 4.62 seconds. So, much faster than TraceMonkey, and almost as fast as V8 (very cool!).

Power Measuring & Saving

2010-06-17T18:05:00.000-07:00

With help from some friendly people in the Maemo IRC (thanks SpeedEvil and DocScrutinizer51!), I was able to get realtime power usage measurements from Fennec on the N900. The goal was to see how much clumping together CPU wakeups would help save power, the idea being that even if the same amount of CPU calculation is done, doing it together in 'clumps' is better than spread out, as each wakeup means the CPU will be in a non-sleep mode for longer. Here are the figures:

(click for fullsize version)

The settings are:

Baseline: Fennec is not run at all. This is for comparison, to see how much power is drawn by the N900 when idle
Fennec 200 clumped wakeups: 200 wakeups/second are scheduled, and all of them are scheduled at the same time. In other words, 200 tiny operations are done, then the CPU can sleep for almost 1 second.
Fennec 200 separate wakeups: As before 200 wakeups/second are scheduled, but at equal intervals over the 1-second period. So the longest the CPU has to sleep is about 1/200th of a second.
All settings were tested both with the screen off (left chart) or screen on (right chart).

Why were 200 wakeups used here? No special reason. As an absolute # of wakeups it's a large number - few websites would have that many animated images/setIntervals()/etc. - but these wakeups were very very short (just incrementing a counter). So this could be similar to having fewer but longer wakeups. But in any case, the results should be treated as just a general indication. The goal is to see whether reducing timeouts is worth doing or not, and not to calculate exactly how much it will improve battery life (that would depend on the specific device, websites visited, usage style, how much clumping we do, etc. etc.). Note also that clumping has a price - it adds latency, as timers don't fire at their normal times. When it is worthwhile, and how much, is a hard question.

For now, though, the results show that clumping wakeups can be noticeable: 38% savings when the screen is off, and 8% when on. As expected much less is saved when the screen is on, because the screen itself drains a lot of power (so our savings become less significant). Which of the 38% or 8% figures is more relevant is a good question - it depends on how often the user takes a 'break' from browsing (so the screen goes off) but leaves the browser on. But anyhow, again, these figures can't be taken literally.

FWIW, to put this in battery life terms, if you save 38% of your power drain then your battery life would be 60% longer, and if you save 8% of your power drain then your battery life would be 8% longer. Again, this is just a general indication, but I think it does show that implementing timeout clumping (properly) in Fennec can be useful.

Other Recent Work

I've also submitted patches for some other bugs related to saving power:

567339 / 359608 - A longstanding issue where image animation timers continue to fire even after browsing to another page. The patch uses the refresh driver to figure out when to suspend timers (but it doesn't use the refresh driver for the actual animations, which might cause more wakeups as it uses a fixed 50Hz timer).
568730 - Stop the JavaScript engine's GC (& other stuff) timer when no JS is running, by suspending it if enough time has passed since the last JS call context was exited.

EDIT: Improved charts

Powersaving Bughunting

2010-05-28T09:49:00.000-07:00

I spent most of yesterday working on Bug 567339, which is about removing unnecessary CPU wakeups in order to save power (which is very important on mobile devices - the fewer time the CPU is woken up from sleep, the longer your battery will last, since each wakeup disturbs sleep mode for a short while after the actual activity).

So, long story short, it seem the bug is that timers for image animations are not canceled. In other words, if you view an image with animations, then a timer is created for each image, and it does the rendering at the proper rate, but the timer keeps running after the image is no longer needed. So you keep getting CPU wakeups at the appropriate rate for the animations, wasting power.

(This is actually an issue on normal Firefox as well, not just mobile - some CPU cycles are being spent on these timers. It is probably negligible, though, but still wasteful.)

The longer story of hunting the cause of the bug is as follows:

A helpful person reports the bug, and mentions a suspicion of https being the cause (since enabling/disabling https affects the symptoms). At issue here are 4 wakeups/second which should not be happening.
Trying to reproduce the bug, it turns out that it happens in nightly builds, but not when building from source in trunk. Hmm.
Trying to figure out the difference, one noticeable change between the nightly builds and building from source is that the start page shows some suggestions of addons in the nightly, but building from source does not. Turns out this is because /trunk reports itself as 2.0, and no addons exist for that yet. Changing the reported version to 1.1 leads to addons being suggested, and the bug in fact occurs.
Since fetching addon suggestions is done from AMO through https, and the original bug report strongly suspected https as the culprit, it would seem we are almost done here. Or are we?
Actually inspecting all the wakeups (this was where most of the time I spend on this bug was; I put up a wiki page with some details of the methodology I used) eventually reveals that the vast majority of wakeups are in fact of imgContainer::notify()... and not https or anything network related.
To confirm this, removing the animated image that was shown when loading addon suggestions removes the wakeups... and indeed, the image had 4 frames/second, exactly the number of wakeups we were hunting!

Still working on figuring out why this happens and how it can be resolved.

How to copy files to an N900

2010-05-24T16:46:00.000-07:00

Today was my first interaction with an N900, and it was not smooth at all. After several hours of messing around, it seems the 'proper' way to copy files from your desktop to the N900 is:

Connect N900 via USB, in 'mass storage' mode
Open the mounted drive in your desktop, and copy files
Unmount the drive
Disconnect the USB cable
If you have a terminal open in the N900, close it
Open a terminal in the N900 and go to ~/MyDocs. The files you transferred will be there.

Thanks go out to the people that helped me on IRC.

Electrolify()?

2010-05-19T13:34:00.000-07:00

I proposed an idea for e10s-enabling existing code - that is, making existing code work with the new electrolysis framework in Firefox, which splits things into separate processes. A bug to track the issue is here, and it includes a working patch I wrote (well, working just well enough to show the kind of JavaScript code this approach would allow).

Lesson #1: Improper XPCOM definitions can lead to silent failure

2010-05-07T16:50:00.000-07:00

I've been tasked with implementing electrolysis (e10s) support for FTP. That means letting child processes (content) send requests to the parent process (chrome), which actually does the networking - while from the perspective of the child process, the interface to an FTP channel is the same.

As this is just my second week at Mozilla, I am still learning the build process, fundamental technologies like XPCOM, and other stuff about the codebase. So I ran into some problems, which I'll mention here on the blog in hopes that they help other people that search for the same keywords.

So, lesson #1 is: Improper XPCOM definitions can lead to silent failure.

Specifically, the issue was that this:

NS_IMPL_ADDREF_INHERITED(FTPChannelChild, FTPBaseChannel)
NS_IMPL_RELEASE_INHERITED(FTPChannelChild, FTPBaseChannel)

NS_INTERFACE_MAP_BEGIN(FTPChannelChild)
    NS_INTERFACE_MAP_ENTRY(nsIRequest)
    NS_INTERFACE_MAP_ENTRY(nsIChannel)
    NS_INTERFACE_MAP_ENTRY(nsIFTPChannel)
NS_INTERFACE_MAP_END_INHERITING(FTPBaseChannel)

was needed, and I was missing the nsIRequest line. I missed it because nsIRequest is a parent interface, and I didn't realize it was also necessary. And, indeed, when sent through JavaScript to a function that expects to receive an object with that interface, it fails. What was tricky was it silently failed - I called a JavaScript function, which was known to work, and it simply returned a failure code, without even starting the function. Only by stepping through all the JS embedding code did it eventually become clear to me that a call to QueryInterface was failing, so the JS embedding code was failing to set up the parameters for the JS function, and not even calling it.