Asynchronous and Messaging

Our extensive unit test suite discovered, that

• if two WebKit instances are loaded simultaneously, which use the same JS file from the local disk, then initializing one of the JS engines sometimes fails.

The average WebKit programmer will never experience this. The average WebKit programmer uses WebKit to make a web browser. And web browsers do not create many instances simultaneously. You press CTRL-T to open a new tab. Then another. Not two at the same time. Especially not from the local file system. Maybe that's the reason why nobody discovered the "feature" before.

Anyway, this means, that creation of WebKits must be serialized. This in turn, introduces all kinds of asynchronous behavior, which must be handled properly in many places. Asynchronous programming is always more challenging than spaghetti coding. Fortunately the client is message based. The image shows events which wait for other things to complete, before they are forwarded. In other words: the display must be loaded before it can show something. As simple as that.

This kept me busy several evenings and made a huge commit. Now I can continue with the setup/install process.

Transparent Browser Bounty Awarded

We have the construction!

Brent's changes to the WebKit code have been added to the trunk (http://trac.webkit.org/changeset/84990).

WebKit can now be used as a transparent browser. Brent also made a sample for how to do Javascript calls between the browser engine and C++.

We now have an HTML renderer at our disposal to create a UI with web technologies. Expect UI development soon.

The Open Virtual World project awards the bounty to Brent for step 1 and step 2. We agreed to redefine steps 3 and 4 for more advanced features later. We thank Brent for his contribution to the project. This was really fun.

(We also thank Lupus Labs for sponsoring the OVW project, which made sponsoring of a WebKit feature possible.)

OVW Transparent Browser Bounty About to be Claimed

Brent Fulgham is currently working on the transparency feature for WebKit. Brent is the developer of the WebKit-Cairo port. It looks like the bounty will be claimed soon. Here are first impressions of the test.

What does this mean for the project? We will soon be able to start working on the GUI. The GUI is the last missing part of the new OVW avatar client. Thanks to Brent, the GUI will be purely HTML based. That's the future folks: Web technologies, CSS, HTML, SVG, Canvas, WebGL.

Later we might also incorporate the multi-process rendering and WebGL developments of WebKit. This means, that rendering will be even faster in the future and we can support real 3D avatars.

Also, thanks to Lupus Labs for sponsoring this open source bounty.

OVW Transparent Browser Bounty

The OVW project needs a transparent browser as GUI platform. Transparent browser means, that if I set CSS transparency of the HTML-body to 0.5, then I want to see the desktop partially. All graphics and dialogs of the OVW client will be displayed by this browser. There will be no other native windows or GUI components. There are so many other things to do in the OVW project. We would like to outsource this task to people who have browser library know how. We would like to sponsor your open source work, and work on other OVW parts in the meantime.

The transparent browser will be used by a OVW client plugin module. We will wrap the OVW specific module around the transparent browser library. What we need is a transparent browser DLL, which our GUI module DLL can use. To be able to integrate the transparent browser into our project, we need a MS VisualStudio project which compiles all required components, links them into one or more DLLs and has a driver program which teaches us how to use the transparent browser. We would like to add features later, hence the need for a compileable project, not just a release DLL. Yes, we know that we can build WebKit from SVN, but it's not simple and there is so much else to do.

Key properties of the project:

• transparent browser, no default window background
• in other words: window alpha channel controlled by CSS only
• only libraries which can be distributed with a BSD like license
• Cairo or Skia does not matter, but not CoreGraphics
• Synchronous calls into and out of the browser's Javascript
• normal performance, meaning no bad hacks, which make transparency, but harm performance
• Keyboard input and mouse move, click, scroll
• IFRAME-Support
• max. 20 MB for everything needed before compression
• better no app installation required, no registry prep, just plain DLLs
• preferably as MS Visual Studio 2005 Project, 2008 possible
• Release + Debug builds
• Unicode and CJK-script support, right-to-left script is nice to have
• BSD license

Chromium comes to mind as a base technology because this is where the buzz currently is. There are also Chromium derivatives like Awesomium and Berkelium which could be a starting point. Pure WebKit is also an option. We might migrate to speedy Javascript, out-of-process rendering, and WebGL later. Gecko is also possible.

The bounty is max. 4.000 € in 4 steps:

• 1 k € for a DLL which paints a web page on a transparent window incl. a sample driver program (which has a URL input). We can compile it.
• 1 k € for keyboard/mouse input, IFRAME support, and max 20 MB.
• 1 k € for synchronous calls with arguments between the driver program and the Javascript engine. Int and String type arguments should be enough, Dictionary arguments would be cool. BeforeNavigate, DocumentComplete notifications similar to the MSIE component.
• 1 k € for additional cool stuff to be defined later, e.g. MSVC 2005 because it lets us omit recent C runtime libs making the package smaller, helping us actively to use the thing, shrinking to 10 MB including all support DLLs (beware of the ICU data DLL), WebGL support, DOM API support, helping us to channel HTTP over the OVW client's HTTP component, etc.

More about the Open Virtual World project: http://www.OpenVirtualWorld.com

First Avatar

I started programming the avatar display. The image shows my test avatar on a test scene.

The test scene uses the scene graph module.

The avatar is already animated. An animation module decodes the old animation format and the animated GIF files and feeds images to the avatar display.

There is no web browser in the background, yet (the background is my Visual Studio). The avatar does not enter a real chat room. This is still a test. The overlay is created by a test function. The test function creates the avatar, assigns an animation, etc. The reason is, that I can develop the display quicker, more reliable, and without network connection. I do not have to open a browser and enter a chat room. The test just pretends that there is a browser and an avatar.

Later, when the avatar display works, then it will be connected to a browser and operated by a real chat room.

Scene Graph

I have been working on a 2D Scene Graph module. The scene graph will be used to show all graphics of avatar and virtual items on top of the web page.

The image to the right shows a test screen with all graphical elements.

What is the scene graph good for? There are 2 different ways to paint graphics and text: one is the old style of painting when the operating system asks the progam to re-paint a part of the screen. In the more modern style, the program defines how the graphics looks like and the scene graph paints it automatically.

We know this modern painting model from HTML, where we define what a web page contains and then the web browser paints it. If we change the HTML DOM, then the browser adapts the display to the new DOM automatically. The same model is used in 3D, where programs define a scene with lots of triangles and then let the graphics card display it.

Our scene graph has an additional feature. It can be scripted by other modules.

As you know, in the Open Virtual World client, everything is s module DLL. Third party developers can add modules easily without changing the core program. Developers can modify how the program works and what it does. Messages from and to the OVW scene graph can be intercepted by other modules. The modules can change messages, add messages and delete parts. They can manipulate the graphics of the avatar display They can add and change elements. They can paint new objects and change the style of text before it is painted on the screen.

OVW graphics is open to scripting by third party modules. This enables add-ons like WoW add-ons, which add all sorts of functionality. At the beginning, this is not a big thing for end users. But when module developers use the feature, then users can choose from a great variety of additional functions.

The scene graph has a minimal set of functions. But it has enough to display avatars on web pages. Previously, I implemented the avatar animation module. Now, everything is ready for the final piece:

I am about to start programming the avatar display.

Transparent GIF Animation Decoding

This was a long time. I had very much to do in my job and 3 weeks vacation. I also was kind of stuck with a problem while decoding animated GIFs with alpha transparency. The decoder did not want to return proper alpha values. So, I stepped deep inside the GIF decoder and prepared myself to add the missing function. But tonight I discovered the method"CxImage::AlphaFromTransparency", which did the job. Yes, the project takes much longer, than expected because of day time work. But it is still alive.

Thoughts on Testing

While programming we write extensive tests. Tests are extra program code, which checks many functions. Basically every piece of code must be tested and be testable all the time. This is the programmers way to make sure, that a complex program works, even if things change over time.

The new client has now more than 100 test functions, each checking a group of functions. The upcoming portal just reached 50. Both are growing quickly with the number of feature blocks in the code. Sometimes we spend complete days writing tests. This sounds much, but it is worth the time, because tests easily spare much time later that would be spent debugging. Writing so much test code is a part of agile development.

Read more about testing on my blog

Working on an Installer

We are currently working on an installer program. We were very happy with the weblin installer, which is a small EXE and downloads a separate ZIP file. We aim for a similar design. It might not do automatic updates in the first version, but we are currently laying the foundations for it.

Tell us if you have suggestions or experiences with the old installer you want to share. Now is the time to improve things.

Jumping right into the Code

For the start I created a simple HTML-based test user interface. It shows the avatar scene on a web page without any fancy stuff, just the data. Using the test interface, I can check what exactly happens between client core and display.

(This is not the final display. There are other showcases for the final display with all the animation, transparency and eye candy.)

The client has a 2 tier architecture. There is the back-end, which has all the data structures and does the protocol processing. And there is the front-end, which does all the display, animations and user interaction.

The image shows 3 columns. The left column has controls where I can enter/leave web pages by entering their URLs. It also has an input area to request data from the back-end. The center column is a very simple display of the avatar scene. It shows 3 avatars: The same avatars which are shown by weblin (in the lower part of the image). They are compatible! I can chat and move, and even put my avatar to sleep. Everything is still very raw. The right column shows the communication between back-end and front-end.

All this is written in HTML. I could write it in C++ or Java, but it would take much longer. The first real display will be in Flash. This shows the flexibility of our 2 tier architecture. Anyone can write a display module like I did in a few days. Maybe someone contributes a specialized displays for power-chatters once we publish the API.

A Module Needs a Name

We are building a modular system. Anything is a module. The configuration database, the protocols, the avatar display, the buddy list, i18n, anything.

But there is one very central module. Some of us call it the "facility manager". It manages URLs, chat rooms, participants in rooms, and it builds a model of the user's presence. Display modules query the "facility manager" and enumerate the data structures of the presence model. The "facility manager" knows who is in which room, it knows what happens in rooms, and it mediates between protocol modules and display modules. The "facility manager" uses protocol modules to enter chat rooms. It gets network events, builds the presence model, and presents it to display modules.

The "facility manager" is the model in MVC. But neither "model" nor "facility manager" are cool names. The thing needs a name. A real name, a cool name. So, that we can talk about the thing. Any ideas?