|
A little over a year ago I started the noVNC project, an HTML5 VNC client. noVNC does a lot
of processing of binary byte array data and so array performance is a
large predictor of overall noVNC performance. I had high hopes that one
of the new binary byte array data types accessible to Javascript (in
modern browsers) would give noVNC a large performance boost. In this
post I describe some of my results from testing these binary byte array
types.
After reading the title, you may have thought: "Wait ... Javascript doesn't have binary byte arrays."
Actually, not only does Javascript have access to binary byte arrays
but there are two unique variants available (technically neither are
part of ECMAScript yet).
Jump list:
Those who follow browser development and HTML standardization may
already be aware of one of these array types. ArrayBuffers (technically:
Typed Arrays) are a required part of the proposed WebGL and File API standards. To use an ArrayBuffer as a byte array you create a Uint8Array view of the ArrayBuffer.
The following Javascript creates an ArrayBuffer view that contains 1000 unsigned byte elements that are initialized to 0:
var arr = Uint8Array(new ArrayBuffer(1000));
But there is an older and more widely supported form of binary byte
arrays available to Javascript programs: ImageData. ImageData is a data
type that is defined as part of the 2D context of the Canvas element.
ImageData is created whenever the getImageData or createImageData method
is invoked on a Canvas 2D context. The "data" attribute of an ImageData
object is a byte array that is 4 times larger than the width * height
requested (4 bytes of R,G,B,A for each pixel).
The following Javascript creates a ImageData byte array with 1000 unsigned byte elements that are initialized to 0:
var ctx = getElementById('canvas').getContext('2d'),
arr = ctx.createImageData(25, 10).data;
There are two traditional ways of representing binary byte data in
Javascript. The first is with a normal Javascript array where every
element of array is a number in the range 0 through 255. The second
method is using a string in which the values 0 through 255 are stored as
Unicode characters in the string and read using the charCodeAt method.
For this post I'm going to ignore the string method since Javascript
strings are immutable and updating a single character in a Javascript
string implies reconstructing the whole string which is both unpleasant
and slow.
The following creates a normal Javascript array with 1000 numbers that are initialized to 0:
var arr = [], i;
for (i = 0; i < 1000; arr[i++] = 0) {}
We are left with three methods for representing binary byte data: normal
Javascript arrays, ImageData arrays, and ArrayBuffer arrays. One might
expect that since ImageData and ArrayBuffer arrays are fixed sized, have
elements with a fixed type, and are used for performance sensitive
operations (2D canvas and WebGL) that the performance of these native
byte arrays would be better than normal Javascript arrays for most
operations. Unfortunately, as of today, most operations are slower when
using these byte array types.
Originally I planned to show the performance numbers comparing browsers
on Linux and Windows. However, I discovered that there is very little
difference (for these array tests) between the same version of a browser
running on Windows vs Linux. Since all the Linux browsers also run on
Windows (but not vice versa) I have limited the performance results to
Windows.
For this post I have hacked together four quick tests to compare
normal Javascript arrays with ImageData and ArrayBuffer arrays. All the
tests use arrays that contain 10240 (10 * 1024) elements and repeat the
operation being tested many times in order to push the test times into a
more easily measured and comparable range. Each test is also run 10
times (iterations) and the mean and standard deviation across all 10
iterations is calculated.
You can run tests yourself by cloning the noVNC repository and loading the tests/arrays.html
page. These test results are based on revision bbee8098 of noVNC.
Running the test in a browser will output JSON data in the results
textarea. This JSON data can then be combined with JSON data from other
browser results and run through the utils/json2graph.py python script which uses the matplotlib module to generate the graphs.
The test machine has the following specifications:
- Acer Aspire 5253-BZ893
- AMD Dual-Core C50 at 1GHz
- 3GB DDR3 Memory
- AMD Radeon HD 6250
- Windows 7
Here are the main browsers that were tested:
In addition, older browser versions were also tested to see if the browsers are making progress:
- Chrome 9.0.597.98
- Chrome 10.0.648.204
- Chrome 11.0.673.0 (build 75038)
- IE 9.0 Platform Preview 7
- Firefox 3.6.13
- Firefox 3.6.16
- Firefox 4.0 beta 11
Please note that I am not a professional performance tester so I
probably haven't made use of optimal testing techniques and there is
certainly a possibility that I have made mistakes that invalidate some
or all of the numbers. I welcome constructive criticism and dialog so
that I can expand and improve on these results in the future.
- create - For each test iteration, an array is created and then initialized to zero and this is repeated 2000 times.
- randomRead - For each test iteration, 5 million reads are issued to pseudo-random locations in an array.
- sequentialRead - For each test iteration, 5 million
reads are issued sequentially to an array. The reads loop around to the
beginning of the array when they reach the end of the array.
- sequentialRead - For each test iteration, 5 million
updates are made sequentially to an array. The writes loop around to
the beginning of the array when they reach the end of the array.
First let's take a look at how the different array types perform at the different tests.
This is the only test where ImageData and ArrayBuffer arrays have
a significant performance advantage because they are automatically
initialized to 0 when created. IE 9 and Opera do not currently support
ArrayBuffer arrays.
Chrome and Opera have the best overall performance although Opera
does not support ArrayBuffer arrays yet. Firefox has particularly bad
random read performance across the board. The results show that there is
little advantage to using ImageData or ArrayBuffer arrays for random
reads and their performance in Chrome and Opera is significantly slower.
Sequential Read Test Results For the sequential read test the situation is quite different.
Firefox has consistent and leading performance across all array types.
Chrome has an order of magnitude worse performance for ImageData and
ArrayBuffer arrays. Opera 11 show a 3X fall in performance for ImageData
arrays compared to normal Javascript arrays. Normal arrays are still
the best choice overall.
Sequential Write Test Results The sequential write test relative results are very similar to
sequential reads with a slow down across the board. Firefox again shows
comparable performance across all three array types. Opera 11 continues
to show a 3X fall in performance with ImageData arrays. Chrome continues
to show an order of magnitude speed different between normal arrays and
the binary arrays.
Now let's slice the data differently to see how the different browsers compare across the different array types.
Normal Array Test Results Chrome is the best overall performer here with Opera pulling a close
second. However, the most notable result in this view is the terrible
performance of Firefox random reads. Given the huge amount of jitter in
the Firefox result compared to the others, my guess is this is a
degenerate case and that Mozilla has some low hanging fruit here.
Opera is now the overall performance winner with Chrome pulling a
close second. The Firefox problem with random reads continues to show up
with ImageData arrays (although this time without the jitter).
Excluding the random read result, Firefox would be the clear winner. IE 9
has a good showing here coming in a close third overall.
The pack thins out significantly since only Chrome and Firefox
support ArrayBuffers. Once again Firefox shows pessimal random read
performance. With that result excluded (or fixed), Firefox would be the
clear winner against Chrome.
- Chrome has the best overall performance for normal arrays.
- Opera has the best overall performance for ImageData arrays with Chrome a close second.
- Firefox has good performance except for random reads where performance drops off a cliff on all array types.
Now we will compare some older browser versions to see if the browser
vendors are making progress over time in improving the performance of
the binary byte array types.
Normal Array Test Results for Firefox Firefox mostly shows steady improvement for normal arrays, but once
again the terrible random read perform rears its head in the shift from
4.0 beta 11 to the 4.0 release version.
ImageData Test Results for Firefox Again, Firefox mostly shows steady improvement for ImageData arrays.
This time the terrible random read performance was introduced somewhere
between the Firefox 3 and Firefox 4 code base.
ArrayBuffer Test Results for Firefox Only Firefox 4 supports ArrayBuffer arrays. The awful random read performance still exists.
Normal Array Test Results for Chrome No strong trends appear in the Chrome data for normal arrays. The
array create speed shows a significant dropoff in Chrome 12. For
sequential writes there was a 2X regression for Chrome 10 and 11.
ImageData Test Results for Chrome There appears to be a significant regression in Chrome 12 related to
ImageData performance. The amount of the dropoff (3X to 6X) and the
significant jitter indicate to me that their is a obvious propblem that
should be fixed.
ArrayBuffer Test Results for Chrome There are no strong trends in Chrome ArrayBuffer array performance
although there seems to be a weak trend towards worse performance.
Normal Array Test Results for Internet Explorer ImageData Test Results for Internet Explorer The final release of IE9 shows a huge performance decrease
compared to the Platform Preview 7. If Microsoft is able to recover this
performance in a subsequent release then this would significantly
change the standing of IE 9 in relation to the other modern browsers.
ImageData and ArrayBuffer arrays have different performance
characteristics within the same browsers. I'm not sure why this should
be the case. In fact, I would recommend that the WHATWG/W3C and browser vendors standardize on ArrayBuffers for both purposes.
This could be done by adding an additional attribute to the ImageData
object perhaps named 'buffer'. The new 'buffer' attribute would be a
generic ArrayBuffer containing the the image data memory. The existing
'data' attribute would become a Uint8Array view of the ArrayBuffer (this
would maintain backwards compatibility). In addition to code
consolidation within the browsers (and one place to focus optimization
effort) this change would allow developers to create a Uint32Array view
of the buffer which would allow whole pixel updates (3 colors + alpha)
with one operation.
Using ImageData and ArrayBuffer (typed array view) arrays will
generally not give better performance for binary byte data than just
using normal Javascript arrays. This is unfortunate since these binary
array types exist specifically to serve performance sensitive
functionality (2D and 3D graphics). It is also surprising since they
have a fixed size and a fixed element type which in theory should allow
faster read and write access to the elements. I suspect (and hope) that
this performance problem is due to the fact that not enough optimization effort has been applied by any of the browser vendors to these binary arrays.
- Mozilla, Google (and Apple), Microsoft and Opera: please spend some effort to optimize your Javascript binary array types!
- Microsoft and Opera: it would be nice if you would implement WebGL. But if not, please at least implement typed array (ArrayBuffer) support
since it stands on its own and will likely be used in the near future
in other places where it make sense such as FileReader objects and in
the WebSocket API to support binary data.
The browser wars are back and new browsers versions are being released
every few weeks. My plan is to continue updating these tests to include
the most recent browser versions. I would also like to expand the tests
to include a random write test and also to test the random and
sequential read performance of binary data stored in Javascript strings.
Stay tuned.
|