First Avatar

I started programming the avatar display. The image shows my test avatar on a test scene.

The test scene uses the scene graph module.

The avatar is already animated. An animation module decodes the old animation format and the animated GIF files and feeds images to the avatar display.

There is no web browser in the background, yet (the background is my Visual Studio). The avatar does not enter a real chat room. This is still a test. The overlay is created by a test function. The test function creates the avatar, assigns an animation, etc. The reason is, that I can develop the display quicker, more reliable, and without network connection. I do not have to open a browser and enter a chat room. The test just pretends that there is a browser and an avatar.

Later, when the avatar display works, then it will be connected to a browser and operated by a real chat room.

Scene Graph

I have been working on a 2D Scene Graph module. The scene graph will be used to show all graphics of avatar and virtual items on top of the web page.

The image to the right shows a test screen with all graphical elements.

What is the scene graph good for? There are 2 different ways to paint graphics and text: one is the old style of painting when the operating system asks the progam to re-paint a part of the screen. In the more modern style, the program defines how the graphics looks like and the scene graph paints it automatically.

We know this modern painting model from HTML, where we define what a web page contains and then the web browser paints it. If we change the HTML DOM, then the browser adapts the display to the new DOM automatically. The same model is used in 3D, where programs define a scene with lots of triangles and then let the graphics card display it.

Our scene graph has an additional feature. It can be scripted by other modules.

As you know, in the Open Virtual World client, everything is s module DLL. Third party developers can add modules easily without changing the core program. Developers can modify how the program works and what it does. Messages from and to the OVW scene graph can be intercepted by other modules. The modules can change messages, add messages and delete parts. They can manipulate the graphics of the avatar display They can add and change elements. They can paint new objects and change the style of text before it is painted on the screen.

OVW graphics is open to scripting by third party modules. This enables add-ons like WoW add-ons, which add all sorts of functionality. At the beginning, this is not a big thing for end users. But when module developers use the feature, then users can choose from a great variety of additional functions.

The scene graph has a minimal set of functions. But it has enough to display avatars on web pages. Previously, I implemented the avatar animation module. Now, everything is ready for the final piece:

I am about to start programming the avatar display.

Transparent GIF Animation Decoding

This was a long time. I had very much to do in my job and 3 weeks vacation. I also was kind of stuck with a problem while decoding animated GIFs with alpha transparency. The decoder did not want to return proper alpha values. So, I stepped deep inside the GIF decoder and prepared myself to add the missing function. But tonight I discovered the method"CxImage::AlphaFromTransparency", which did the job. Yes, the project takes much longer, than expected because of day time work. But it is still alive.

Animating Avatars

We are still building the avatar display. The most important part of the avatar display are the avatars. There are also chat balloons, nicknames, menus and more GUI stuff, but avatars are the most important part. And we want ths avatars to be animated.

What do we need for animated avatars?
  1. a renderer which puts pixels on the screen
  2. an animated avatar in whatever format
  3. a decoder, that decodes the avatar data
In the previous posting I discussed cairo as the renderer of choice. That's point 1. We now have to decide on point 2, the animation format and choose (or make) a decoder for that format. Of course, the decoder is strongly related to the animation format.

What is an animation format? An avatar can perform multiple actions. It can stand, walk, wave, etc. Therefore, we need not just one, but a set of animations for a single avatar. An animator component will then decode one animation sequence of the set and display frame after frame. If the avatar is supposed to wave, then the decoder will play the "wave" animation. Usually it will just play the "stand" animation. And when the avatar moves, then it should play a "walk" animation. So, the avatar format must provide multiple animations.

Lets review our options for an animation format
This list is very incomplete. Proposals are welcome.

Real 3D models would be cool, but I am not really a 3D expert. So, we will start by investigating the "config.xml" option.

This means: we need an XML parser (already in the Apollo framework) and a GIF decoder, which is capable of animations. Typical graphics libraries are candidates for the job. We are currently checking which library is capable of loading animated GIFs from memory and from file and be integrated into the project. Getting to know a code library takes some time. So, this will take a while. We are checking:
  1. OpenCV (no GIF?)
  2. CImg (no GIF?)
  3. GraphicsMagick (very complex, GIF loader not found)
  4. DevIL (GIF only from file, but we need decoding also from memory)
  5. GIL (GIL from Adobe does not support GIF)
  6. CxImage (GIF works)
  7. Imlib2 (could not find a Windows MSVC build)
  8. ...others?
The high profile image libraries seem to ignore GIF. This sounds weird, but is logical. GIF has only indexed color with max. 256 colors. It is not suited for real image processing applications.

Only CxImage survived the test. This is not an optimal solution, because CxImage is Windows based. It must be replaced for a Mac/Linux port. Replacing CxImage should not be too difficult, because there are only few lines of code in the application code. Basically loading an image file and iterating through the frames.

We do not want to spend too much time on this. So, we will now continue with the latest CxImage version.

Cairo Test

I am testing the cairo drawing library. I modified the transparency test program from my last blog post so, that it paints an avatar image with nickname instead of the pattern from the last blog.

I can move the avatar around and positioned it on top of a browser window. The result looks like there is an avatar standing on the browser - looks like everything is ready. But it's just a fake avatar. It is static, not animated, not from a chat room. It is just an image painted on a transparent window with the cairo drawing library. But as a cairo test, it is quite successful.

Painting graphics with cairo feels really backwards for us web programmers. We are used to markup: HTML, SVG, and XAML. Programming with "hard coded" functions like cairo_move_to, cairo_set_line_width and cairo_fill feels odd. No CSS, no div, no img-tag. Just good old C++ code with function calls.

I discovered another argument for cairo compared to WebKit. In a blog post (which I do not find anymore) a Google Chrome developer discusses the choice of skia. Skia is not hardware accelerated, which makes skia look like a bad choice. But, the Chrome developer argues, that most of the time the browser spends in HTML code, arranging DOM elements and applying hierarchies of CSS styles. Once the rendering code knows where to paint something, then only a small fraction of the time is spent for the actual drawing. This is a hint, that using WebKit is much worse performance-wise than using plain cairo. When we use cairo directly, then we get rid of all the time that WebKit spends dealing with HTML features. The trade off is, that there is no CSS, no JavaScript, and no iframe in the avatar display. CSS makes applying different styles easy. Styles and themes need much more effort in plain cairo drawing code. But it is should be much faster. And speed counts, because we do not want to slow down your PC.

OK, so, cairo works. The image above is only a fake. It is still a long way to go, but we are on the way.

Starting Avatar Display Development

The client has currently much backend code. It has all the protocol handling, user data, and chat functions we need. But there are no avatars yet. The "green" region is (almost) ready. The "red" region is still missing.

(Another part that is missing is a chat window. But when avatars are done, then the chat is a simple, trust me.)

(The IRC module will not be necessary for chat with avatars on web pages. Supporting the IRC chat protocol in addition to XMPP/Jabber is an option for later.)

So, now it is all about the "red" circle: animated avatars on a transparent layer over the web page.

The data is available, graphics is available, is just has to be painted. This sounds simple. But, it is not that simple. We are programming software that should be portable to different platforms, specifically to MacOS and Linux although the original development (and first release) is on Windows. Painting on different platforms is not as simple as it sounds, because it is not possible to use the painting functions of the operating system. These functions (called drawing API) are not available on other operating systems. And because drawing code is heavy and takes much time to write, it is not good to adapt it for each operating system.

What we need is a cross platform drawing API. Many of us use Firefox or Chrome on Windows, Linux, and Mac. These programs have the same problem. And they solved it by using cross platform drawing APIs. Firefox pains everything you see on the screen using a library called cairo. Chrome uses skia. These libraries are ways to abstract drawing, so that the display can be ported to other platforms with reasonable effort.

There are other ways to paint stuff in a platform independent way. One possibility is to use a HTML-renderer, such as WebKit. We could integrate WebKit and paint avatars and chat balloons with HTML and JavaScript. Actually, this is a good solution, because we intend to integrate WebKit anyway for program windows and dialogs. The only drawback is, that it is not much fun to write lots of drawing code in JavaScript. Also, performance might be a problem, because the drawing code would be interpreted by the JavaScript runtime and every visible element would be in HTML with CSS. This is how web browsers work and it is OK for them. But when you surf the Web and your animated avatar is painted all the time, then you don't want your avatar to take much CPU. A web browser can take much CPU while it paints a page, but after that it is silent. Avatars, however are animated and paint again and again. Very much like a Flash banner on newspaper page, that always draws CPU and slows your PC. We do not want to slow your PC.

On the other hand painting is much easier with WebKit than with graphics libraries like skia and cairo. Speed speaks for skia and cairo. The JavaScript programming issue speaks against WebKit. No easy decision. Do you have any ideas?

I the meantime, I made a simple program that just paints patterns transparently on the background. It is a demo to check how that works on Windows. All we have to do is to use skia, cairo, or WebKit or someting else to paint avatars on such a transparent layer. We will decide soon which library we will use.

If you have comments, let us know.

Hammer vs. Coffee Machine

I implemented a simple "competition" feature. Competition means, that some items can damage other items. This is another check of the item server design.

Competition is a bit more advanced and demanding, than the crafting. During implementation I learned, that we need floating point values in addition to integer values. I could map all crafting actions onto integers. But competition needs finer grained computations. So I had to change the database. I also added time values to implement a cool down period and had to change the database structure again. Effectively, the cool down implements a hit-rate limitation, which leads - in combination with damage values and modifications - to the usual DPS (damage per second) statistics.

Before I started coding, I modeled the behavior in Excel:

Here is what we can see at on the inventory web pages. This is the inventory with a hammer and a coffee machine. I used the coffee machine to make coffee, but the hammer can also damaged it:

This is the coffee machine. It is in perfect shape (Condition: 100%, Hitpoints: 1000). Obviously, the coffee machine is susceptible to shock damage and piercing damage. But shock damage is more dangerous. Piercing damage can be absorbed more easily.

The hammer. It can be used once per second (Attack cooldown). The hammer is a shock device. It smashes into something. It does no piercing damage. A knife would be different. The hammer makes 100 points shock damage. But the victim will absorb some of the damage.

Now I can hit the coffee machine one with my hammer with this command:
After the "hit", the coffee machine is in worse condition than before. It lost a bit of its "Condition" value (Condition: 93.0 %):

The hammer lost also a bit of its condition. Much less than the coffee machine. It is just a bit worn down (Condition: 99.9 %). The hammer will have to be repaired after some time in order to project the full damage.

Very simple, isn't it? But there are powerful mechanisms implemented which we will use later. The design of the item server has been verified and completed with workflows of different areas. The item server:
  • caches items,
  • loads items from a database
  • returns item properties on request
  • has user actions
  • has interactions between items, and
  • saves changes to the database

I will now leave the item server and return to work on the client. The client has lots of back-end and protocol code. But it does not really show avatars. We want to see animated avatars chatting on web pages. The client also needs a user interface. It needs windows, dialogs, a settings dialog, a buddy list, etc. But most important are avatars. That's the next step. Not a small one. I'll keep you updated.

Coffee and Cola Sample Implementations

I implemented a crafting process in the item server and the portal.

I am using two examples to check if the details make sense: coffee machine and cola dispenser with their respective ingredients and products.

The image shows all items which are currently defined. Most of them are for testing various functions.
This is a view of my visible inventory. One of them is a cola dispenser, a cola machine which makes a noname cola from water and syrup. It also needs energy. If I click the cola machine, then ...
... I see details of the cola machine. The machine is already equipped with water, syrup and batteries. It has a button to produce cola. When I press the button ...
... then I get an additional item, a small can of cola. Actually, a noname cola. Not very fancy, but still.

(For game designers: all item interactions are modeled with items. We took some days to replace a property driven model by an item driven model. It might be a bit less realistic, but manipulating items is more fun than manipulating item statistics.)

I can click on the cola can and see the datails ...
... including a button which lets me drink the can. Usually I won't drink it myself. I would offer it to visitors of my web site. If someone else drinks it, then we both get social points.

This is all just in aplain white design, actually not really a design. Only the bare minimum to check HTML/CSS structure and classes.

There will be a real design later. What about the green we were used to? or facebook blue? What do you think?

For developers: the item server has now 63 unit tests, the portal is still at 69. There are 74 different types of properties, which manage all item interactions.

Next steps: we will implement a little bit more item interaction, including damaging some stuff. After that we will turn back to the client and add a cool avatar display.

Virtual Goods Update

Neptun is growing. After the basic functions are complete, we started implementing an experimental version of virtual item interactions. Interactions enable simple work flows. Later we will extend these work flows, add more complex work flows and graphics for objects on web pages.

This sounds very theoretic. Lets make an example: you can have a coffee machine. You put the coffee machine from your inventory down to your web site. You can add stuff to the coffee machine, say coffee beans, water and a power source. Then you can tell the coffee machine to make coffee for your friends.
Your visitors can use a cup to take coffee from the machine, which results in ... (I will tell later). You will get the water somewhere on the web from a fountain. The water can also be used to grow plants on your web site. Plants need seeds. Some plants can be harvested for wood. This is just an example.

Virtual goods on the Web will work like in other games. But now they can be everywhere. Not just inside a facebook game. They can be on your homepage, on your community page, your forum, anywhere you go. We will take care, that plants will not overgrow web pages. Hopefully you will help us. And plants are not the end. All you can do INSIDE of games will be possible OUTSIDE on the Web. Not inside an artificial world, but on the real Internet.

And yes: we feel a little bit guilty for programming virtual goods now, before you have avatars. But I admit, that virtual goods are so much fun and we want to research this segment a bit further before we go back to the client and the avatar. Please be patient. In the meantime: the community version of weblin is still running. You can grow the community by offering the download on your Web site.

For programmers: This is how Neptun looks like in Firefox. It is just a list of controls for programmers. Neptun has now 40 test functions (the portal has 69). The large blue text shows links for these tests. Each link execute a single test. The "All Test" link executes all at once. At the top is a link ("Items") to the item list as before and a way to stop the server ("Quit").

Neptun: Virtual Items Server

Avatars will have an inventory of virtual items. They will be able to buy, sell, build, trade and loot virtual items. A server manages all these items and everything you do with items. Over the last 3 weeks we implemented such a component. This is our first version of a virtual item server. It is called "Neptun".

In October and November we implemented a first version of the web portal and a simple form of an inventory. Back then, we simulated our items. Now, they are real.

The inventory is now populated by the item server. In other words: items are alive inside the item server. If you see them on the web portal, then the portal fetches all it needs to display them from the item server. As usual, items are also stored in a database for safety. But in the database they are only passive. In the item server they are active and you can do something with them.

What you can do comes later. For now these items have a place to be seen (the inventory of the portal) and a place to live (the item server). It is actually much better to see virtual items on a web page of the portal, than in the item server. On the portal they have translated text and images.

Developers can also see items in the item server, but this is much more bare bone. So, this is how Neptun looks like. It has a web user interface, but it shows only technical data. There is a list of item numbers and item properties. Not much for the user, but very informative for developers.

For the programmers: Neptun is implemented in C#. We program, debug and test in MS Visual Studio 9. Then, we run the program on Linux with mono.

State of the Avatar

It was quiet during the last 2 months. The main reason is, that we started working permanently in a day job. Since this project has no funding, we can not afford any longer to live without income.

I continue to work in this project in the evenings and on weekends. Clearly, work progresses more slowly. We can not hold any previously announced deadline of the roadmap.

There has also been a shuffle in responsibilities in the project. More about that later.

I just want to tell you, that we did not abandon the project. There will be a new virtual presence system with avatars and virtual goods on all web pages. We are continuing to work. Only the mode and the speed changed, because we have to work for a living.

On the other hand I will blog more in the future. I will tell more frequently what we are developing so, that you kan keep track of the project.

Browser Navigation (FF)

Browser Navigation has been implemented for Firefox. It is a substantial Firefox extension in Javascript and some C++ client code. But we are very happy, that we do not need the hook DLL like weblin.

Internet Explorer has been deferred. We are an open source project, so supporting open source browsers has priotity. IE support will come later.

Christmas and New Year

We planned to work on browser navigation for Firefox and IE over the holidays. Browser navigation means to detect browsers and tabs and feed navigated URLs to the avatar client. In addition the client needs coordinates of browser windows and always up to date visiblity information of tabs. Windows and tabs can do a lot of weird stuff, including being closed and switched at any time and being minimized. Many cases, much work.

Unfortunately, the holidays were full of visiting friends, chill out, party. So, the navigation work did not progress as planned. We will continue tomorrow. Promised.

Happy New Year!