Support Infrastructure

Last month was the month of the item server. This month I am working on the support infrastructure.

The client is supposed to paint avatars and objects on Web pages. To do this, it needs a fair amount of support.

The client needs:

• a user account on the server. The user account will be created when the client starts for the first time.
• an XMPP account to connect to the XMPP server. The XMPP account is created automatically so, that users do not have to care,
• avatar images and a way to select avatars,
• an item inventory contains all the stuff you find, trade, or make,
• an identity service which provides avatars and items to the client.
• all this data wants to be stored persistently in several databases. 

These are the services I am currently building.

The goal is to make using the Open Virtual World client very easy. Just install and go.

Three Musketeers

Chrome support arrived.

We now have extensions for all 3 major browsers: Chrome, Firefox, Internet Explorer. See the image where I am on this blog page with my avatar and 3 different browsers at the same time.

If my avatar moves in one browser, then it moves in all browsers. If I chat in one browser, I chat in all at the same time. The client logs in to the chat room only once. But the avatar is shown on all browsers.

This is very different from weblin where you could only be on a page with on browser. Other browsers of the same page/domain had no avatars at all.

Initially I wanted to defer Chrome support until after the release. But it is much better to have Chrome support, because I Chrome is my main browser. The client needs lots of day to day testing. Now it is being tested all the time while I am doing my normal web browsing.

By the way: a weblin just private chatted to me, but there is no private chat yet. I just saw it in the logfile, but I could not reply. Do we need private chat early?

IE Plugin, Installer, Setup, VS10, Support Software

Just a short notice what's currently happening: I am working on

• some issues with the IE support. Stability is very good, but avatars were "behind" the browser window
• installer for the IE extension
• simple setup process which starts after downloading/installing, where you can select install browser extensions, select a nickname/avatar.
• I upgraded to MS Visual Studio 2010. There are some migration issues to be solved. But I feel, that it is better to migrate now than during the beta phase.
• There is now a separate code repository for the WebKit SDK. It has 1.5 GB headers, libraries and DLLs, which are needed to build and run the WebView module. Surprisingly the result is not larger than 10 MB.

Still next on the list:

• Finishing Setup and browser extension installers
• Simple Web site
• Installer packaging and final testing

Asynchronous and Messaging

Our extensive unit test suite discovered, that

• if two WebKit instances are loaded simultaneously, which use the same JS file from the local disk, then initializing one of the JS engines sometimes fails.

The average WebKit programmer will never experience this. The average WebKit programmer uses WebKit to make a web browser. And web browsers do not create many instances simultaneously. You press CTRL-T to open a new tab. Then another. Not two at the same time. Especially not from the local file system. Maybe that's the reason why nobody discovered the "feature" before.

Anyway, this means, that creation of WebKits must be serialized. This in turn, introduces all kinds of asynchronous behavior, which must be handled properly in many places. Asynchronous programming is always more challenging than spaghetti coding. Fortunately the client is message based. The image shows events which wait for other things to complete, before they are forwarded. In other words: the display must be loaded before it can show something. As simple as that.

This kept me busy several evenings and made a huge commit. Now I can continue with the setup/install process.

Internet Explorer Plugin, Autostart and System Tray

Three of the remaining six tasks are done:

• IE Support: I implemented an IE browser helper which connects to the client and tells about opening and closing of tabs, about URL navigation and tab switches.
• System Tray Icon: a small fun project. Since the client has no window, we need something else to control it. I created a small system tray icon which can start and stop the client.
• Autostart: this was the easiest task. Just a registry setting, configured by the system tray icon menu.

Now remaining:

• Settings dialog (do we really need this, we could start with config files and add a settings dialog later)
• Simple Web site
• Installer packaging and final testing

Next is the Web site. We need some place to download the installer and a simple way to configure the avatar, e.g. change animations and nickname.

Finally, we have to bundle all files and create an installer for Firefox and IE extensions. Not a simple task, especially the IE installer. And finally, finally there is much testing required because we want the installer to do it's job smoothly.

Translations and Release Packaging

In the last blog post I mentioned a list of remaining tasks:

• Translations
• Release packaging
• Installer testing
• Auto start
• Tray icon
• Settings dialog
• Browser compatibility
• Web portal

I just ticked off 2 tasks: Translations and Release packaging. This is the remaining list in the most likely work order:

• IE Support
• Auto start
• Tray icon
• Settings dialog
• Simple Web site
• Installer testing

Most of these tasks are of medium size. We might defer Google Chrome support until after the first beta release. Maybe someone from the community wants to take this task. We hope to complete these tasks until August 2011.

There are more tasks which should and will be done after the beta release:

• Google Chrome support
• Personal inventory of virtual items
• Virtual Items on Web pages
• Real Web portal with item shop and (avatar) trading
• Facebook, Twitter, OpenID authentication
• More social interactions (wuscheln and more)
• Porting to MacOS

Some of these tasks are big chunks of work. But most will be done until the end of this year.

Chat Window

This is the brand new chat window.

It is a normal chat window as you would expect. But there are few facts worth mentioning:

1.

It is a module. Actually, it is a display module. This means, that while you are in the chat, you will stay in the room, even if the browser is closed. This is a feature which many weblin power

users wanted.

2.

It is HTML based. See the bold user names: a single line CSS style. See the resize bars between chat-out panel, chat-in panel, and participants panel: just a few lines of HTML with the help of jQuery UI Layout.

Do you remember a blog post one year ago about "Starting Avatar Display Development"? This was after the client core and protocol was implemented, but before you could see avatars.

Now the chat is implemented and I can paint a new picture with a blue box around the finished chat module (there is also a grey cross over the IRC protocol. We do not need it now. IRC is just a future option).

All this means, that the client is basically complete.

There are only small things missing, like:

• Translations: there is a translation module, but not all visible texts are translated,
• Installer: there is an installer, but the client must be packaged and tested before release,
• Auto start with windows: currently you have to start as any other program by double clicking the icon,
• Tray icon: would be nice

... and some bigger parts:

• Settings and configuration dialog: there are config files and a config database, but no dialog yet,
• Browser compatibility: currently only Firefox works, but we want IE and Chrome as well,
• Web portal: a portal like weblin.com would be nice where you can change avatar images.

There is still much to do. It is not configured for normal users. It is for people who know what they are doing. It has still less features than weblin. It is an alpha version. But it's cool and it's on the way.

More technical detail: http://blog.wolfspelz.de/2011/06/html-based-chat-window-with-webkit.html

Tabs and Windows

I am working on support for multiple browser tabs. Good old weblin had only one room per browser window. If you changed the tab, then you changed the room. If you switched between tabs very often, then you switched between rooms all the time.

For the OVW client we will support multiple tabs in the same browser window. Each tab will join a chat room. If you change the tab, then you stay in the room. The client will always show the avatars of the visible tab. If the tab is changed, then the next set of avatars will be shown and avatars of the hidden tab will be made invisible.

With weblin there was only one avatar display for each room. If you opened the same page twice in two different browser windows, then you saw avatars only on one of them. The other browser had no avatars. This was sometimes confusing. Now, there will be avatars on all browser windows, even if they show the same page. You can chat in one browser and see the chat also in the other window. You move your avatar in one browser and it magically also moves in the other browser. This is made possible by a real MVC implementation. The avatars will be duplicated. They are just a view of the scene model.

We currently have a Firefox plugin. But of course, we also need IE and Chrome. These plugins are much more complex, than the simple windows message hook of weblin. In other words: they are more work. But they will provide much richer features in the future and ... tabs.

Here is more: http://blog.wolfspelz.de/2011/06/more-ovw-progress.html

Avatar Display

Do you see the difference?

On the left is the new OVW client. On the right is a Weblin display. The OVW client still lacks the points below the nickname. But it has a green glow around the nickname instead. Otherwise, they look almost identical. That's the point.

There is now an avatar display in the new OVW client. It is programmed in HTML and JavaScript with support from the C++ core.

There are still features missing. For example, avatars do not walk yet, they slide. But there has been significant progress. Because of the HTML/JavaScript environment, we can use jQuery for effects and event handlers. This makes programming very quick - and fun.

Just as an example, here is how the nickname is changed. It is a piece of JavaScript code:

  ...
 SetAvatarNickname: function (hParticipant, sNickname)
 {
   $('#' + hParticipant + ' .cNickname').html(EscapeHTML(sNickname));
 },
 ...

Pure JavaScript with jQuery. Very simple.

Windows and Dialogs

Based on Brent's WebKit transparency feature I created 2 modules:

1. a WebView module, which manages transparent browsers of any kind
2. a Dialog module, which shows a dialog frame and the real content inside

1. WebView Module

The WebView module is used for all user interface elements. For a settings dialogs, for a chat window, for avatars on Web pages, for toast windows. Everythng you can see on the screen.

The display of WebView windows is completely created by HTML. Dialogs are designed with HTML, images, CSS, Javascript. The main reason for using WebKit as a user interface engine is portability. Modules and dialogs created with HTML will run on all platforms.

WebKit makes the user interface platform independent. Besides, it is really fun to program the user interface in HTML/Javascript.

2. Dialog Module

The Dialog module configures window border and window controls. The outside is identical for all dialogs. The window content is managed by whatever HTML page is put inside, be it a local page from the file system or a remote Web page.

Examples: an inventory window and the current test dialog.

The window can be styled with CSS. We have themes. I present the beloved - and sometimes passionately hated - wood theme:

Do you prefer the Mac theme?

Remember: it's all HTML. You can tell, that it is HTML from the small (reload) button on the window border. This button is for testing only. It will disappear later.

Next is the avatar display.

Transparent Browser Bounty Awarded

We have the construction!

Brent's changes to the WebKit code have been added to the trunk (http://trac.webkit.org/changeset/84990).

WebKit can now be used as a transparent browser. Brent also made a sample for how to do Javascript calls between the browser engine and C++.

We now have an HTML renderer at our disposal to create a UI with web technologies. Expect UI development soon.

The Open Virtual World project awards the bounty to Brent for step 1 and step 2. We agreed to redefine steps 3 and 4 for more advanced features later. We thank Brent for his contribution to the project. This was really fun.

(We also thank Lupus Labs for sponsoring the OVW project, which made sponsoring of a WebKit feature possible.)

First Avatar

I started programming the avatar display. The image shows my test avatar on a test scene.

The test scene uses the scene graph module.

The avatar is already animated. An animation module decodes the old animation format and the animated GIF files and feeds images to the avatar display.

There is no web browser in the background, yet (the background is my Visual Studio). The avatar does not enter a real chat room. This is still a test. The overlay is created by a test function. The test function creates the avatar, assigns an animation, etc. The reason is, that I can develop the display quicker, more reliable, and without network connection. I do not have to open a browser and enter a chat room. The test just pretends that there is a browser and an avatar.

Later, when the avatar display works, then it will be connected to a browser and operated by a real chat room.

Scene Graph

I have been working on a 2D Scene Graph module. The scene graph will be used to show all graphics of avatar and virtual items on top of the web page.

The image to the right shows a test screen with all graphical elements.

What is the scene graph good for? There are 2 different ways to paint graphics and text: one is the old style of painting when the operating system asks the progam to re-paint a part of the screen. In the more modern style, the program defines how the graphics looks like and the scene graph paints it automatically.

We know this modern painting model from HTML, where we define what a web page contains and then the web browser paints it. If we change the HTML DOM, then the browser adapts the display to the new DOM automatically. The same model is used in 3D, where programs define a scene with lots of triangles and then let the graphics card display it.

Our scene graph has an additional feature. It can be scripted by other modules.

As you know, in the Open Virtual World client, everything is s module DLL. Third party developers can add modules easily without changing the core program. Developers can modify how the program works and what it does. Messages from and to the OVW scene graph can be intercepted by other modules. The modules can change messages, add messages and delete parts. They can manipulate the graphics of the avatar display They can add and change elements. They can paint new objects and change the style of text before it is painted on the screen.

OVW graphics is open to scripting by third party modules. This enables add-ons like WoW add-ons, which add all sorts of functionality. At the beginning, this is not a big thing for end users. But when module developers use the feature, then users can choose from a great variety of additional functions.

The scene graph has a minimal set of functions. But it has enough to display avatars on web pages. Previously, I implemented the avatar animation module. Now, everything is ready for the final piece:

I am about to start programming the avatar display.

Transparent GIF Animation Decoding

This was a long time. I had very much to do in my job and 3 weeks vacation. I also was kind of stuck with a problem while decoding animated GIFs with alpha transparency. The decoder did not want to return proper alpha values. So, I stepped deep inside the GIF decoder and prepared myself to add the missing function. But tonight I discovered the method"CxImage::AlphaFromTransparency", which did the job. Yes, the project takes much longer, than expected because of day time work. But it is still alive.

Animating Avatars

We are still building the avatar display. The most important part of the avatar display are the avatars. There are also chat balloons, nicknames, menus and more GUI stuff, but avatars are the most important part. And we want ths avatars to be animated.

What do we need for animated avatars?

1. a renderer which puts pixels on the screen
2. an animated avatar in whatever format
3. a decoder, that decodes the avatar data

In the previous posting I discussed cairo as the renderer of choice. That's point 1. We now have to decide on point 2, the animation format and choose (or make) a decoder for that format. Of course, the decoder is strongly related to the animation format.

What is an animation format? An avatar can perform multiple actions. It can stand, walk, wave, etc. Therefore, we need not just one, but a set of animations for a single avatar. An animator component will then decode one animation sequence of the set and display frame after frame. If the avatar is supposed to wave, then the decoder will play the "wave" animation. Usually it will just play the "stand" animation. And when the avatar moves, then it should play a "walk" animation. So, the avatar format must provide multiple animations.

Lets review our options for an animation format

• 3D animation formats, e.g. Collada, Doom 3, Quake, blender (?)
  
• Custom formats like config.xml with animated GIFs or PNGs
  

This list is very incomplete. Proposals are welcome.

Real 3D models would be cool, but I am not really a 3D expert. So, we will start by investigating the "config.xml" option.

This means: we need an XML parser (already in the Apollo framework) and a GIF decoder, which is capable of animations. Typical graphics libraries are candidates for the job. We are currently checking which library is capable of loading animated GIFs from memory and from file and be integrated into the project. Getting to know a code library takes some time. So, this will take a while. We are checking:

1. OpenCV (no GIF?)
   
2. CImg (no GIF?)
3. GraphicsMagick (very complex, GIF loader not found)
   
4. DevIL (GIF only from file, but we need decoding also from memory)
   
5. GIL (GIL from Adobe does not support GIF)
   
6. CxImage (GIF works)
   
7. Imlib2 (could not find a Windows MSVC build)
   
8. ...others?
   

The high profile image libraries seem to ignore GIF. This sounds weird, but is logical. GIF has only indexed color with max. 256 colors. It is not suited for real image processing applications.

Only CxImage survived the test. This is not an optimal solution, because CxImage is Windows based. It must be replaced for a Mac/Linux port. Replacing CxImage should not be too difficult, because there are only few lines of code in the application code. Basically loading an image file and iterating through the frames.

We do not want to spend too much time on this. So, we will now continue with the latest CxImage version.

Cairo Test

I am testing the cairo drawing library. I modified the transparency test program from my last blog post so, that it paints an avatar image with nickname instead of the pattern from the last blog.

I can move the avatar around and positioned it on top of a browser window. The result looks like there is an avatar standing on the browser - looks like everything is ready. But it's just a fake avatar. It is static, not animated, not from a chat room. It is just an image painted on a transparent window with the cairo drawing library. But as a cairo test, it is quite successful.

Painting graphics with cairo feels really backwards for us web programmers. We are used to markup: HTML, SVG, and XAML. Programming with "hard coded" functions like cairo_move_to, cairo_set_line_width and cairo_fill feels odd. No CSS, no div, no img-tag. Just good old C++ code with function calls.

I discovered another argument for cairo compared to WebKit. In a blog post (which I do not find anymore) a Google Chrome developer discusses the choice of skia. Skia is not hardware accelerated, which makes skia look like a bad choice. But, the Chrome developer argues, that most of the time the browser spends in HTML code, arranging DOM elements and applying hierarchies of CSS styles. Once the rendering code knows where to paint something, then only a small fraction of the time is spent for the actual drawing. This is a hint, that using WebKit is much worse performance-wise than using plain cairo. When we use cairo directly, then we get rid of all the time that WebKit spends dealing with HTML features. The trade off is, that there is no CSS, no JavaScript, and no iframe in the avatar display. CSS makes applying different styles easy. Styles and themes need much more effort in plain cairo drawing code. But it is should be much faster. And speed counts, because we do not want to slow down your PC.

OK, so, cairo works. The image above is only a fake. It is still a long way to go, but we are on the way.

Starting Avatar Display Development

The client has currently much backend code. It has all the protocol handling, user data, and chat functions we need. But there are no avatars yet. The "green" region is (almost) ready. The "red" region is still missing.

(Another part that is missing is a chat window. But when avatars are done, then the chat is a simple, trust me.)

(The IRC module will not be necessary for chat with avatars on web pages. Supporting the IRC chat protocol in addition to XMPP/Jabber is an option for later.)

So, now it is all about the "red" circle: animated avatars on a transparent layer over the web page.

The data is available, graphics is available, is just has to be painted. This sounds simple. But, it is not that simple. We are programming software that should be portable to different platforms, specifically to MacOS and Linux although the original development (and first release) is on Windows. Painting on different platforms is not as simple as it sounds, because it is not possible to use the painting functions of the operating system. These functions (called drawing API) are not available on other operating systems. And because drawing code is heavy and takes much time to write, it is not good to adapt it for each operating system.

What we need is a cross platform drawing API. Many of us use Firefox or Chrome on Windows, Linux, and Mac. These programs have the same problem. And they solved it by using cross platform drawing APIs. Firefox pains everything you see on the screen using a library called cairo. Chrome uses skia. These libraries are ways to abstract drawing, so that the display can be ported to other platforms with reasonable effort.

There are other ways to paint stuff in a platform independent way. One possibility is to use a HTML-renderer, such as WebKit. We could integrate WebKit and paint avatars and chat balloons with HTML and JavaScript. Actually, this is a good solution, because we intend to integrate WebKit anyway for program windows and dialogs. The only drawback is, that it is not much fun to write lots of drawing code in JavaScript. Also, performance might be a problem, because the drawing code would be interpreted by the JavaScript runtime and every visible element would be in HTML with CSS. This is how web browsers work and it is OK for them. But when you surf the Web and your animated avatar is painted all the time, then you don't want your avatar to take much CPU. A web browser can take much CPU while it paints a page, but after that it is silent. Avatars, however are animated and paint again and again. Very much like a Flash banner on newspaper page, that always draws CPU and slows your PC. We do not want to slow your PC.

On the other hand painting is much easier with WebKit than with graphics libraries like skia and cairo. Speed speaks for skia and cairo. The JavaScript programming issue speaks against WebKit. No easy decision. Do you have any ideas?

I the meantime, I made a simple program that just paints patterns transparently on the background. It is a demo to check how that works on Windows. All we have to do is to use skia, cairo, or WebKit or someting else to paint avatars on such a transparent layer. We will decide soon which library we will use.

If you have comments, let us know.

Hammer vs. Coffee Machine

I implemented a simple "competition" feature. Competition means, that some items can damage other items. This is another check of the item server design.

Competition is a bit more advanced and demanding, than the crafting. During implementation I learned, that we need floating point values in addition to integer values. I could map all crafting actions onto integers. But competition needs finer grained computations. So I had to change the database. I also added time values to implement a cool down period and had to change the database structure again. Effectively, the cool down implements a hit-rate limitation, which leads - in combination with damage values and modifications - to the usual DPS (damage per second) statistics.

Before I started coding, I modeled the behavior in Excel:



Here is what we can see at on the inventory web pages. This is the inventory with a hammer and a coffee machine. I used the coffee machine to make coffee, but the hammer can also damaged it:



This is the coffee machine. It is in perfect shape (Condition: 100%, Hitpoints: 1000). Obviously, the coffee machine is susceptible to shock damage and piercing damage. But shock damage is more dangerous. Piercing damage can be absorbed more easily.



The hammer. It can be used once per second (Attack cooldown). The hammer is a shock device. It smashes into something. It does no piercing damage. A knife would be different. The hammer makes 100 points shock damage. But the victim will absorb some of the damage.



Now I can hit the coffee machine one with my hammer with this command:

http://localhost:3277/Portal/Inventory/DroidAction/61/Hit/47

After the "hit", the coffee machine is in worse condition than before. It lost a bit of its "Condition" value (Condition: 93.0 %):



The hammer lost also a bit of its condition. Much less than the coffee machine. It is just a bit worn down (Condition: 99.9 %). The hammer will have to be repaired after some time in order to project the full damage.



Very simple, isn't it? But there are powerful mechanisms implemented which we will use later. The design of the item server has been verified and completed with workflows of different areas. The item server:

• caches items,
• loads items from a database
• returns item properties on request
• has user actions
• has interactions between items, and
• saves changes to the database
  

I will now leave the item server and return to work on the client. The client has lots of back-end and protocol code. But it does not really show avatars. We want to see animated avatars chatting on web pages. The client also needs a user interface. It needs windows, dialogs, a settings dialog, a buddy list, etc. But most important are avatars. That's the next step. Not a small one. I'll keep you updated.

Coffee and Cola Sample Implementations

I implemented a crafting process in the item server and the portal.

I am using two examples to check if the details make sense: coffee machine and cola dispenser with their respective ingredients and products.

The image shows all items which are currently defined. Most of them are for testing various functions.

This is a view of my visible inventory. One of them is a cola dispenser, a cola machine which makes a noname cola from water and syrup. It also needs energy. If I click the cola machine, then ...

... I see details of the cola machine. The machine is already equipped with water, syrup and batteries. It has a button to produce cola. When I press the button ...

... then I get an additional item, a small can of cola. Actually, a noname cola. Not very fancy, but still.

(For game designers: all item interactions are modeled with items. We took some days to replace a property driven model by an item driven model. It might be a bit less realistic, but manipulating items is more fun than manipulating item statistics.)

I can click on the cola can and see the datails ...

... including a button which lets me drink the can. Usually I won't drink it myself. I would offer it to visitors of my web site. If someone else drinks it, then we both get social points.

This is all just in aplain white design, actually not really a design. Only the bare minimum to check HTML/CSS structure and classes.

There will be a real design later. What about the green we were used to? or facebook blue? What do you think?

For developers: the item server has now 63 unit tests, the portal is still at 69. There are 74 different types of properties, which manage all item interactions.

Next steps: we will implement a little bit more item interaction, including damaging some stuff. After that we will turn back to the client and add a cool avatar display.

Virtual Goods Update

Neptun is growing. After the basic functions are complete, we started implementing an experimental version of virtual item interactions. Interactions enable simple work flows. Later we will extend these work flows, add more complex work flows and graphics for objects on web pages.

This sounds very theoretic. Lets make an example: you can have a coffee machine. You put the coffee machine from your inventory down to your web site. You can add stuff to the coffee machine, say coffee beans, water and a power source. Then you can tell the coffee machine to make coffee for your friends.
Your visitors can use a cup to take coffee from the machine, which results in ... (I will tell later). You will get the water somewhere on the web from a fountain. The water can also be used to grow plants on your web site. Plants need seeds. Some plants can be harvested for wood. This is just an example.

Virtual goods on the Web will work like in other games. But now they can be everywhere. Not just inside a facebook game. They can be on your homepage, on your community page, your forum, anywhere you go. We will take care, that plants will not overgrow web pages. Hopefully you will help us. And plants are not the end. All you can do INSIDE of games will be possible OUTSIDE on the Web. Not inside an artificial world, but on the real Internet.

And yes: we feel a little bit guilty for programming virtual goods now, before you have avatars. But I admit, that virtual goods are so much fun and we want to research this segment a bit further before we go back to the client and the avatar. Please be patient. In the meantime: the community version of weblin is still running. You can grow the community by offering the download on your Web site.

For programmers: This is how Neptun looks like in Firefox. It is just a list of controls for programmers. Neptun has now 40 test functions (the portal has 69). The large blue text shows links for these tests. Each link execute a single test. The "All Test" link executes all at once. At the top is a link ("Items") to the item list as before and a way to stop the server ("Quit").