Transparent Browser Bounty Awarded

We have the construction!

Brent's changes to the WebKit code have been added to the trunk (http://trac.webkit.org/changeset/84990).

WebKit can now be used as a transparent browser. Brent also made a sample for how to do Javascript calls between the browser engine and C++.

We now have an HTML renderer at our disposal to create a UI with web technologies. Expect UI development soon.

The Open Virtual World project awards the bounty to Brent for step 1 and step 2. We agreed to redefine steps 3 and 4 for more advanced features later. We thank Brent for his contribution to the project. This was really fun.

(We also thank Lupus Labs for sponsoring the OVW project, which made sponsoring of a WebKit feature possible.)

OVW Transparent Browser Bounty About to be Claimed

Brent Fulgham is currently working on the transparency feature for WebKit. Brent is the developer of the WebKit-Cairo port. It looks like the bounty will be claimed soon. Here are first impressions of the test.

What does this mean for the project? We will soon be able to start working on the GUI. The GUI is the last missing part of the new OVW avatar client. Thanks to Brent, the GUI will be purely HTML based. That's the future folks: Web technologies, CSS, HTML, SVG, Canvas, WebGL.

Later we might also incorporate the multi-process rendering and WebGL developments of WebKit. This means, that rendering will be even faster in the future and we can support real 3D avatars.

Also, thanks to Lupus Labs for sponsoring this open source bounty.

OVW Transparent Browser Bounty

The OVW project needs a transparent browser as GUI platform. Transparent browser means, that if I set CSS transparency of the HTML-body to 0.5, then I want to see the desktop partially. All graphics and dialogs of the OVW client will be displayed by this browser. There will be no other native windows or GUI components. There are so many other things to do in the OVW project. We would like to outsource this task to people who have browser library know how. We would like to sponsor your open source work, and work on other OVW parts in the meantime.

The transparent browser will be used by a OVW client plugin module. We will wrap the OVW specific module around the transparent browser library. What we need is a transparent browser DLL, which our GUI module DLL can use. To be able to integrate the transparent browser into our project, we need a MS VisualStudio project which compiles all required components, links them into one or more DLLs and has a driver program which teaches us how to use the transparent browser. We would like to add features later, hence the need for a compileable project, not just a release DLL. Yes, we know that we can build WebKit from SVN, but it's not simple and there is so much else to do.

Key properties of the project:

• transparent browser, no default window background
• in other words: window alpha channel controlled by CSS only
• only libraries which can be distributed with a BSD like license
• Cairo or Skia does not matter, but not CoreGraphics
• Synchronous calls into and out of the browser's Javascript
• normal performance, meaning no bad hacks, which make transparency, but harm performance
• Keyboard input and mouse move, click, scroll
• IFRAME-Support
• max. 20 MB for everything needed before compression
• better no app installation required, no registry prep, just plain DLLs
• preferably as MS Visual Studio 2005 Project, 2008 possible
• Release + Debug builds
• Unicode and CJK-script support, right-to-left script is nice to have
• BSD license

Chromium comes to mind as a base technology because this is where the buzz currently is. There are also Chromium derivatives like Awesomium and Berkelium which could be a starting point. Pure WebKit is also an option. We might migrate to speedy Javascript, out-of-process rendering, and WebGL later. Gecko is also possible.

The bounty is max. 4.000 € in 4 steps:

• 1 k € for a DLL which paints a web page on a transparent window incl. a sample driver program (which has a URL input). We can compile it.
• 1 k € for keyboard/mouse input, IFRAME support, and max 20 MB.
• 1 k € for synchronous calls with arguments between the driver program and the Javascript engine. Int and String type arguments should be enough, Dictionary arguments would be cool. BeforeNavigate, DocumentComplete notifications similar to the MSIE component.
• 1 k € for additional cool stuff to be defined later, e.g. MSVC 2005 because it lets us omit recent C runtime libs making the package smaller, helping us actively to use the thing, shrinking to 10 MB including all support DLLs (beware of the ICU data DLL), WebGL support, DOM API support, helping us to channel HTTP over the OVW client's HTTP component, etc.

More about the Open Virtual World project: http://www.OpenVirtualWorld.com

First Avatar

I started programming the avatar display. The image shows my test avatar on a test scene.

The test scene uses the scene graph module.

The avatar is already animated. An animation module decodes the old animation format and the animated GIF files and feeds images to the avatar display.

There is no web browser in the background, yet (the background is my Visual Studio). The avatar does not enter a real chat room. This is still a test. The overlay is created by a test function. The test function creates the avatar, assigns an animation, etc. The reason is, that I can develop the display quicker, more reliable, and without network connection. I do not have to open a browser and enter a chat room. The test just pretends that there is a browser and an avatar.

Later, when the avatar display works, then it will be connected to a browser and operated by a real chat room.

Scene Graph

I have been working on a 2D Scene Graph module. The scene graph will be used to show all graphics of avatar and virtual items on top of the web page.

The image to the right shows a test screen with all graphical elements.

What is the scene graph good for? There are 2 different ways to paint graphics and text: one is the old style of painting when the operating system asks the progam to re-paint a part of the screen. In the more modern style, the program defines how the graphics looks like and the scene graph paints it automatically.

We know this modern painting model from HTML, where we define what a web page contains and then the web browser paints it. If we change the HTML DOM, then the browser adapts the display to the new DOM automatically. The same model is used in 3D, where programs define a scene with lots of triangles and then let the graphics card display it.

Our scene graph has an additional feature. It can be scripted by other modules.

As you know, in the Open Virtual World client, everything is s module DLL. Third party developers can add modules easily without changing the core program. Developers can modify how the program works and what it does. Messages from and to the OVW scene graph can be intercepted by other modules. The modules can change messages, add messages and delete parts. They can manipulate the graphics of the avatar display They can add and change elements. They can paint new objects and change the style of text before it is painted on the screen.

OVW graphics is open to scripting by third party modules. This enables add-ons like WoW add-ons, which add all sorts of functionality. At the beginning, this is not a big thing for end users. But when module developers use the feature, then users can choose from a great variety of additional functions.

The scene graph has a minimal set of functions. But it has enough to display avatars on web pages. Previously, I implemented the avatar animation module. Now, everything is ready for the final piece:

I am about to start programming the avatar display.

Transparent GIF Animation Decoding

This was a long time. I had very much to do in my job and 3 weeks vacation. I also was kind of stuck with a problem while decoding animated GIFs with alpha transparency. The decoder did not want to return proper alpha values. So, I stepped deep inside the GIF decoder and prepared myself to add the missing function. But tonight I discovered the method"CxImage::AlphaFromTransparency", which did the job. Yes, the project takes much longer, than expected because of day time work. But it is still alive.

Animating Avatars

We are still building the avatar display. The most important part of the avatar display are the avatars. There are also chat balloons, nicknames, menus and more GUI stuff, but avatars are the most important part. And we want ths avatars to be animated.

What do we need for animated avatars?

1. a renderer which puts pixels on the screen
2. an animated avatar in whatever format
3. a decoder, that decodes the avatar data

In the previous posting I discussed cairo as the renderer of choice. That's point 1. We now have to decide on point 2, the animation format and choose (or make) a decoder for that format. Of course, the decoder is strongly related to the animation format.

What is an animation format? An avatar can perform multiple actions. It can stand, walk, wave, etc. Therefore, we need not just one, but a set of animations for a single avatar. An animator component will then decode one animation sequence of the set and display frame after frame. If the avatar is supposed to wave, then the decoder will play the "wave" animation. Usually it will just play the "stand" animation. And when the avatar moves, then it should play a "walk" animation. So, the avatar format must provide multiple animations.

Lets review our options for an animation format

• 3D animation formats, e.g. Collada, Doom 3, Quake, blender (?)
  
• Custom formats like config.xml with animated GIFs or PNGs
  

This list is very incomplete. Proposals are welcome.

Real 3D models would be cool, but I am not really a 3D expert. So, we will start by investigating the "config.xml" option.

This means: we need an XML parser (already in the Apollo framework) and a GIF decoder, which is capable of animations. Typical graphics libraries are candidates for the job. We are currently checking which library is capable of loading animated GIFs from memory and from file and be integrated into the project. Getting to know a code library takes some time. So, this will take a while. We are checking:

1. OpenCV (no GIF?)
   
2. CImg (no GIF?)
3. GraphicsMagick (very complex, GIF loader not found)
   
4. DevIL (GIF only from file, but we need decoding also from memory)
   
5. GIL (GIL from Adobe does not support GIF)
   
6. CxImage (GIF works)
   
7. Imlib2 (could not find a Windows MSVC build)
   
8. ...others?
   

The high profile image libraries seem to ignore GIF. This sounds weird, but is logical. GIF has only indexed color with max. 256 colors. It is not suited for real image processing applications.

Only CxImage survived the test. This is not an optimal solution, because CxImage is Windows based. It must be replaced for a Mac/Linux port. Replacing CxImage should not be too difficult, because there are only few lines of code in the application code. Basically loading an image file and iterating through the frames.

We do not want to spend too much time on this. So, we will now continue with the latest CxImage version.