Fly together Forever 比翼恒飞

Over the weekends, I spent some time on Dojo Objective Harness (DOH) to fix a reported bug: in some cases the global progress bar overshoots 100%. This is obvious in the dojo test suite, the progress bar goes beyond 900%.

After some digging, it turns out the algorithm to calculate the global progress bar has a flawed assumption: the number of tests in each group is known before the tests start. That’s not the case if nested test suites are used, such as the case in dojo._base.test.

The fix is quite simple as well: rather than trying to create a portion in the global progressbar for each test, we only create one for each group.

Another problem noted in the original bug report is overflow of test group progressbar. I never noticed it before, but keeping a close eye on each group progressbar when running dojo test suite does reveal it. By removing tests in the dojo test suite, I isolated the problem to dojo.date and dojo.date. The problem is the test group progress bar is not updated at all due to the fact that they don’t have a html page (instead both dojo.date and dojo.data test cases only use javascript files). The fix for this turns out to be quite straghtforward as well as soon as I know what causes it.

After the bug fixes, I feel like to add a new feature to DOH, clickable test cases which will hilight the relevant log line after the test finishes. Searching a failture/error in the log pane is a real pain especially if you have lots of tests generating huge number of messages. The log line will be hilighted according to whether the test failed or succeeded. (Note: the hilighting requires dojo, so if DOH is used without dojo, you won’t be able to see the hilighting animation.)

Similarly, when clicking on the global progressbar, the test group associated with that portion of the bar will be scrolled into view (for any failed test group, a tooltip is also attached to the global progressbar portion to indicate which test group it is). This is useful when locating a failed test group.

In addition, a tfoot is added to the test list table after all tests finish, which contains a summary of the test run. Check out the online demo in our nightly. This will be part of dojo 1.3 release.

Apparently, google maps has different features/capacities based on different user interface languages. My parents just complained to me that they can’t find “place of interests” in their google maps any more. After obtaining more information from them, it appears that they are actually using Chinese version of google maps, while when they were using the English version, “place of interests” was indeed visible as an option.

So the problem is how to change the UI language. The website URL for the Chinese version of google maps is the same as the English version, both of which is maps.google.com, so must be another way of doing it. Searching the UI, I can’t find a control to change language anywhere. However, I did notice on one page when clicking around, where I have this in my address bar.

http://maps.google.com/maps/mm?hl=en

by chaning it to:

http://maps.google.com/maps/mm?hl=zh-CN

As expected, I get Simplified Chinese version of google maps. So to switch UI language in gmap is to append “?hl=LANG_CODE” to the URL, such as:

http://maps.google.com/?hl=en

Will load the gmap front page in English. Some LANG_CODE I tried which are recognized by gmap is: de, en, fr, ko, ja, ru, zh-CN and zh-TW. (See language identifier for a complete list)

It is well known that Firefox (even the latest 3.x) does not support text-decoration=”underline” on text elements in SVG. However, it bites me hard when I found out imagemagick does not support that either when rastering svg to pixel images.

After digging around, (by looking at the ebuild file) I found out that imagemagick is actually making use of librsvg to do the heavy lifting when dealing with svg files.

librsvg is part of gnome-base packages (as can be seen in gentoo portage). Looking at the source code, I figured that it makes use of pango to render text.

From the days of working on SCIM project, I know for sure that pango is more than capable of rendering text with underlines. So there must be something missing in librsvg to render it. It turns out that, while the svg parser implemented in librsvg does indeed look for text-decoration, and recoganize underline (among strikethrough and overline), it is not actually making use of these parsed info to render the text.

After a bit of tracing the code, I found out how to patch it: the trick is just to add a pango underline attribute to the text layout, and everything else is taken care of by pango.

This bug is reported to upstream, and hopefully it will be merged there soon. The patch is available in that bug report, so you can grab it if you can’t wait. The patch also fixes a bug which is discovered after the underline problem is fixed: when a underlined text has stroke set, the stroke is rendered in wrong position.

In a desktop environmen, menus (be it context menu or menu in menubar) have a column which display the accelerator key (or shortcut key in windows). In the 0.4 dojo Menu widget, we used to have that support, but it was not ported to 1.x.

After seeing Bill’s commit to add a real MenuBar to dijit, I think it’s the time to bring back this feature. For now, see the test_Menu.html test file for a demo.

A new property is introduced for dijit.MenuItem, called accelKey, of type String. It can be set either in markup or in the arguments passed to the dijit.MenuItem constructor (the demo contains examples for both of them). In addition, dynamically setting it is also supported, such as:

//item is a dijit.MenuItem object
item.attr("accelKey","Ctrl+W");

As mentioned in the demo, this accelKey is only a string, you can basically pass in any string you want. However, when the combination is pressed, it won’t actually trigger the action associated with the menu item: that has to be done explicitly elsewhere (such as dojo.connect to body’s onkeydown event, and detect whether a recognized accelerator key is pressed, if so, invoke the corresponding event handler).

By default, dijit.form.DropDownButton requires a dropdown menu specified when initializating. However, with a little bit of patching via subclassing it, we can easily achieve lazy loading of the dropdown menu: only creating it the first time a user clicks on the button. This will speed up the initial loading time of the application, especially if a lot of these type of buttons are present on the page.

Here is the core of how to achieve lazy loading:

dojo.declare("MyDropDownButton", dijit.form.DropDownButton, {
	dropDown:1,
	_openDropDown: function(){
		if(this.dropDown===1){
			this.dropDown=new dijit.Menu();
			var items=[{label:'One'}, {label:'Two'}, {label:'Three'}];
			dojo.forEach(items,function(args){
				this.dropDown.addChild(new dijit.MenuItem(args));    
			},this);
		}
		this.inherited(arguments);
	}
});

The trick here is to set dropDown to something which evaluates to true so that when clicked the first time, DropDownButton will proceed to call _openDropDown, where we have a chance to create the real dijit.Menu we want to show. Without line 2 above, _openDropDown won’t be called because DropDownButton thinks there is no attached dropdown menu.

As you may have already figured, in order to use this, the Menu has to be created programmatically.

While trying to add setting clipboard content support to doh.robot, I encountered a very strange bug and spent several hours of trail and failure without going anywhere. I am using java 1.6 update 11 directly from Sun. Basically, I have two problems both of which are described here: copying HTML data to windows clipboard from java does not work, with the following symptoms

1. when puting data into the clipboard with a RepresentationClass other than java.lang.String does not work properly with any applications (it is the same issue as described in a java bug report, which is supposed to be fixed in java 1.6 update 11)
2. even if you do use java.lang.String as RepresentationClass, the pasted html only appear correctly in IE, not in safari (it contains extra garbage characters), nor in FF (nothing is pasted at all).

The problem as described in the mentioned webpage, is due to buggy support of windows HTML clipboard format in java. One workaround is suggested in the same article by modifying some internal stuffs in the supporting class in java with the source files available for download. However, the provided code fails to run against my java 1.6 (although it does compile fine), probably due to changed internal mechanisms.

(more…)

After introducing dojo.reload in my previous post, I talked with kriszyp in IRC, and he asked whether dojo.reload can actually modify already created objects from dojo.declare-d classes: in most web 2.0 apps, after the app is loaded, there are lots of instances of classes already created, and using dojo.reload to reload their definition won’t actually update any of these existing instances.

As I do not have an idea to actually implement that, I do come up with another “workaround” (if you like): rather than propagate changes to existing objects on the page, we can actually try to ask the app to reload itself without reloading the whole page. We don’t want to add in any special code in any frontend apps to handle this sort of “self reload” mechanism just to speed up development. Instead, a generic way should be available without any extra logic support in the application layer.

With the help of dojo and with one assumption, I did come up such an implementation to support that. The one assumption is that, all existing objects you care about in your applications are all dijits. If this assumption can be met, what we really need to do in order to reload an application is to:

1. call onunload handlers (in case any are registered with dojo)
2. destroy all dijits in the page
3. reset the document.body.innerHTML to what it looks like when the page is first loaded (before any js actually modifys it)
4. call all onload hanlders registered with dojo (such as dojo.parser etc.)

Item 3 can be achieved via a xhr call to retrieve current page from server and figure out the content within <body> tag. Item 4 requires a bit of extra work, because once all the onload handlers are called by dojo, they are removed. So in order to actually rerun them, we have to remember the original list of handlers.

Of course, as you may already figured, the method mentioned above won’t be able to run any modified onload handlers (even after they are dojo.reload-ed), however, it does cover most of other cases.

The one assumption mentioned above is not actually true in most web 2.0 applications using dojo, in such cases, any non-dijit objects have to be destroyed somehow so that they can be recreated, but that logic has to be application specific. On the other hand, in order to prevent memory leaks in browsers (espcially IE), it is always a good idea to have a destroy function for your objects to clean itself up. So if you are not worried about any thing other than dijits, then the method mentioned above should work reasonably well, while if you do want to re-create other non-dijits stuffs, you can call some application specific cleanup routines (which as I mentioned above would be good to have in any case).

You can find the code in dojoc.util.loader, the function is actually called dojo.reloadPage(). The normal use case for this is:

1. dojo.reload any modules you modified and want to test;
2. dojo.reloadPage to actually test them without waiting for reloading all js files/reloading your browser window.

Note: dojo.reloadPage only works if it is included in the initial page load, if you dojo.require dojoc.util.loader after the page is loaded, dojo.reloadPage won’t work (see comment at the beginning of the file to know why)

dojo.reload now also resides in dojoc.util.loader, and compared to the previous version, two additional features are add:

• Partial module name match: now you just need to type in a partial name of the module you want to reload, if it only matches one module already loaded, that module will be reloaded. If there are multiple matches, it will list all possible modules and error out.
• Automatically remove cache of dijit template: if a module references widget template files (using templatePath), these cached templates will be cleared automatically, so next time you create such a dijit, the new template will be loaded from your sever directly.

A new function to reload css files is also included in dojoc.util.loader, dojo.reloadCss, which also supports partial name match. If it is given no argument, it will reload all css files in the page (with a href attribute, of course), similar to ReCSS feature in the firebug lite shipped with dojo core.

Warning: use at your own risk.

For the past several months, I am developing frontend against a bare cherrypy server which also serves all static files, like javascripts/images etc. Without building frontend code (so there are lots of small js files to load), it takes more than 20 seconds to reload the unbuilt frontend application I am working on (which is the main driver behind my hacking on dojo.reload).

Use Apache as proxy server

Today I decided to shield an apache proxy server in front of the cherrypy server to off load all static files serving duties from the latter, in the hope of speeding up reloading speed of the app.

The apache proxy should be setup so that it directly serves any files under /debug and /release, all other requests are dynamic and should be handled by cherrypy server. In addition, our backend app sometimes use HTTP redirects to direct client to a new page. In Apache configuration file, all this can be achieved by:

ProxyPassMatch ^/(?:debug|release)/.* !
ProxyPass / http://127.0.0.1:8000/
 
ProxyPassReverse / http://127.0.0.1:8000/
 
Alias /debug /var/www/htdocs
Alias /release /var/www/htdocs/release

Note: the /debug directory is the unbuilt frontend code, while the /release points to the built frontend code (by default, dojo will put the built version under release directly as peer of dojo dir).

With the above settings, cherrypy is nicely sitting behind the Apache server without worrying about any static files, and the reloading time of the unbuilt frontend code now reduces to about 4 seconds, which is a dramatic improvement.

Try nginx instead

nginx is normally considered to be a faster reverse proxy server than apache, so I want to give it a try.

In nginx configuration file, proxy_pass is used to pass the request to a backend server, while it does not use proxy_pass_reverse, instead the equivalent in nginx is proxy_redirect directive. As long as your host domain name is properly set up, the following nginx directive is equavalent to the above apache directive:

location /debug/ {
    alias /var/www/htdocs;
}
location /release/ {
    alias /var/www/htdocs/release;
}
location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_redirect default;
}

More info on proxy_redirect can be found in official documentation.

Impression of Nginx compared to Apache

While I don’t want to do any thorough comparsion of the two reverse proxy servers, I just tried each of them several times and monitors the net panel output in firebug. It seems, nginx delievers more consistent performance: for the same page, apache sometimes deliver it in 1 second, sometimes in 4 seconds, while nginx always delivers it in 1 second. Thus I guess I will just keep using nginx at least for now.

When working with FAT client web 2.0 application, it is normally quite slow whenever you need to reload your application because you are developing against unbuilt code (the source files are changed frequently, so you don’t want to build your frontend code every time a modification happens). As a frontend developer, you are using Firefox + Firebug, with console/script tabs enabled (sometimes net tab in firebug as well), which makes the loading time of a page with lots of javascript files even more slower.

While we are used to ajax ways of updating part of a page without reloading the entire webpage, no one seems to apply the same methodology to reloading javascript source files.

Dojo has the powerful dojo.require mechanism which can pull in javascript source files on the fly, and it is smart enough to cache any source files which are already downloaded. However, it does not provide any way to force reloading of a js file.

After digging into dojo.require, I came up with a simple function to overcome this very issue: basically, it will unset any memory dojo has about that downloaded source file (or module, as called in dojo.require), and dojo.require that module again.

Here is the source code for this function, which is called dojo.reload for now:

dojo.reload=function(d){
	//remove the module namespace from its parent namespace
	var sym=d.split('.');
	if(sym.length>1){
		var pname=sym.slice(0,-1).join('.');
		var prop=sym[sym.length-1];
		var parent=dojo.getObject(pname,false);
		if(parent){
			console.log('delete property',prop,'on object',pname,':',delete parent[prop]);
		}
	}
 
	//remove _loadedModules entry so that dojo.require will re-load it
	delete dojo._loadedModules[d];
 
	//tidy up _loadedUrls so that dojo._getText will reload it
	sym=dojo._getModuleSymbols(d);
	var relpath = sym.join("/") + '.js';
	//copied from dojo._loadUri
	var uri = ((relpath.charAt(0) == '/' || relpath.match(/^\w+:/)) ? "" : dojo.baseUrl) + relpath;
 
	delete dojo._loadedUrls[uri];
	dojo._loadedUrls.splice(dojo.indexOf(dojo._loadedUrls,uri),1);
 
	//set cacheBust to make sure we load latest file
	dojo.config.cacheBust=+new Date;
 
	dojo._loadreloaded=d;
	if(!dojo._reloadhotkey){
		dojo._reloadhotkey=dojo.connect(document.documentElement,'onkeydown',function(e){
			if(e.keyCode===dojo.keys.F6){
				dojo.reload(dojo._loadreloaded);
				dojo.stopEvent(e);
			}
		});
	}
	console.log('reloading module',d);
	dojo.require(d);
};

Just copy and paste this function somewhere to your frontend code base, and when you want to reload a particular module, just type dojo.reload('mymodule.myfile') in Firebug console, and the latest version of your module will be downloaded and ready to be used/tested on immediately. In order to make sure the browser does not use one from its cache, cacheBust will be set to current timestamp to make sure it is loading from server directly. Another fine touch in the dojo.reload function is that, if you just want to reload the module you last reloaded, simply hit F6 which saves you some type.

WARNING: this is only meant for development, don’t put this in your deployed code base. This is only tested in firefox 3, but it should work in other browsers as well. I think this will only work with the default loader and it will fail if you want to use it in xdomain case. At last, the famous phrase: use at your own risk.

Edit: fixed mangled greater than symbol (thanks slackorama). changed all this->dojo so that no matter what this function is called, it can always correctly unset any “dojo memory”. – December 2, 2008 at 7:51 am