Archive for the ‘Progress’ Category

CGI plugin moving to events

The CGI plugin was initially blocking, because that was the easiest way to write it. This week I’ve so far worked on moving it to the event-based model.

Initial results were terrible, while I could get it stable, its performance was only 50-60% of that of the blocking model, latency was doubled, and it came with 250 more lines of code.

Today, investigating showed that epoll was firing some 2k pointless write events when we had nothing to write. The couple additional syscalls to remove the wait for write events when we know we don’t need them were quite a win.


The latest CGI plugin is event-based, stable, and better than the blocking one, not counting lines of code though (they increased to 300 ;)).

Performance increased 18% for shell scripts, 88% for the C app, and 26% for PHP. Latency is less than half of the blocking model’s one, it’s sub-10ms now (vs 15-20ms).

Transactions/sec:

Bash C PHP
Blocking 2.2k 3.5k 500
Events 2.6k 6.6k 620

For comparison, static pages get 29k on the same setup.

June 19, 2012cand No Comments »
FILED UNDER :Benchmark , Progress

Weekly review of week 1

The first week of GSOC is up. This is the weekly review of what’s happened.

First I re-read Drepper’s “How to write shared libraries”, to refresh my memory on the correct techniques.

I then proceeded to test those against the Monkey plugins, which are shared libraries by themselves, although used via dlopen instead of dynamic linking.

The plugins had no visibility control whatsoever, exporting a hundred or so unneeded global variables and functions. This causes slower startup for Monkey and each plugin has 5-10kb of extra size.

The only needed exports are those of the plugin API; those starting with _mkp_ and the informational struct plugin_info.


First I attempted to use GCC pragmas and hiding; after discussion on the ML that approach was deemed unsatisfactory, given its requirement of GCC 4.0. (side note: I really think a minimum GCC version should be decided, so that advanced features may be used without ambiguity such as this.)

So the patchset was revised to use the usual approach of defaulting to hidden and specifically exporting each necessary symbol. This results in more writing (one addition per symbol, instead of two per file), but is supported in older compilers too.

As a result, each plugin’s exported symbols dropped from > 100 to about 30, with a corresponding drop in the binary size as well.

This work has not yet been merged.


Later in the week, in preparation for testing my own work later on, and in celebration of the release of LLVM 3.1, I ran monkey through Clang’s static analyzer.

Clang found about 20 issues, most of which were possible crashers. Some were dead writes.

Patches to fix most of these have been applied to master.


Lib-wise, I started on designing the API. It follows the general style of other embeddable www server libs. Right now the API contains the necessary functions for all desired functionality, as well as the necessary enums (assuming I didn’t forget anything ;)).

The public header was posted on the ML as a request for comments; only Ed (my mentor) commented. I assume it’s more of a lack of interest than a general silent acceptance.

Conclusion

  • Clang’s analyzer had never been ran on the codebase, based on the results
  • Plugin symbol visibility awaiting merging
  • Library design right on schedule

Assuming no incoming comments on the API, next week I’ll start on the tasks of week 3-4, moving one week ahead of schedule. The tasks include setting up the build system, function stubs, and then the library implementation.

May 27, 2012cand No Comments »
FILED UNDER :Progress , Weeklies

Initial design thoughts

The main goal is to let an application serve its own data.

Given this, the only plugins that make sense are liana, liana_ssl and perhaps auth. Liana* don’t need the plugin hooks, and auth would require further changes, so for the first edition I think it makes sense to disable the plugin hooks altogether. (note: this means stage_run only)


Of the five plugin hooks, only the first four would make sense for the library: one call to check the IP, one to check the URL, and one to handle the data. The last would be useful to free any possible dynamic memory.

As the first two are access control, they would be optional. Return values for both could be MK_TRUE/MK_FALSE. (pass/forbidden)

For the data handler, the app would receive the virtual host and URL. It should only concern itself with the actual data (+headers, status), TCP_CORK and various other parts are out of scope for the lib use case.
The data handler would also return either true or false, signalling whether data would be sent. Returning false, monkey should fall back to accessing normal files in the specified directory.

This would make it possible to serve dynamic pages, but have the css, js and images be in a static folder somewhere, saving the application from having to serve them itself in the data callback.

Config

So far it appears that most config parameters cannot be easily changed at runtime. This is OK for the lib, so it can only allow config changes when the server is not running.

As for what config parts should be supported, everything. Including virtual hosts.

Soname

I think it makes sense to bind the library version to the Monkey version. Monkey 1.0.0 -> libmonkey-1.0.0.so.

May 22, 2012cand No Comments »
FILED UNDER :Progress

Status update

With recent events, there’s something to post about (!).

First, a bug was found in the depth buffer handling. Patches for that haven’t yet been applied in Mesa master.

Second, the Evergreen loop bug is fixed.

So in current master, the MLAA color filter works on:
- softpipe
- llvmpipe
- r600g (r700 and Evergreen confirmed myself, r600 should work, no idea about *. Islands)
- Nouveau on Fermi (maybe more?)

In master + those patches, the depth filter works on the same set as above.

For r300g, neither filter produces correct results, so I suspect a bug in r300g. Given four other drivers working fine it shouldn’t be in common code.

January 27, 2012cand 2 Comments »
FILED UNDER :Progress

Merged in master!

http://cgit.freedesktop.org/mesa/mesa/commit/?id=6571c0774af1f5ebd0fab40bf4769702d3c9ded5

The post-processing queue is now in Mesa master.

August 25, 2011cand 3 Comments »
FILED UNDER :Meta , Progress , Tickbox

r600g success!

With the loop bug worked around (https://bugs.freedesktop.org/show_bug.cgi?id=40034), MLAA now runs on my netbook with comparable quality to llvmpipe.

Since this was the last point to do, next I will focus on integration.

August 16, 2011cand No Comments »
FILED UNDER :Progress , Tickbox

First HW success

MLAA is successfully running on nouveau (nvc0) / GTX470.

Note that ROUND support for nouveau doesn’t appear to be in master yet, and I hear there is another bug with nouveau master and pp right now. No progress with the r600g bugs on my netbook either.

August 14, 2011cand No Comments »
FILED UNDER :Progress , Tickbox

HW binding

The queue is now bound to hw as well. Should be working on all Gallium drivers.

As for MLAA, it needs ROUND support from the driver; currently that means softpipe and llvmpipe in master, and r600g with my branch. The ROUND support has been tested on the hw I have available (Evergreen), but should run on r600, r700, and Cayman.

However, MLAA is broken for me on the netbook. The userspace version is broken as well, so this looks like a bug in r600g; it may be beyond my ability to solve.

Due to the sample size of one, I don’t know whether this bug affects Zacate only, Evergreen only, or all of r600g.

August 7, 2011cand 2 Comments »
FILED UNDER :Progress

Optimizations

All of the TGSI shaders have now had some human touch. In total, +40% fps in glxgears on llvmpipe. There could still be smaller optimization possibilities in the shaders, removing some temps mainly, but they no longer do anything stupid (yes, that’s the glsl compiler I just insulted, now shoo, it’s hand-tuned assembly from now on).

In other news, none of the PP functions so far show up in profiling. Remains to be seen whether that is the case with more powerful gpus.

July 14, 2011cand 3 Comments »
FILED UNDER :Meta , Progress

Unify – done.

MLAA is running inside Mesa, with the rest of the queue. Currently doing some final cleanups, you can expect the code to be in my private repo in some hours.

edit: Up. Go here to test.

An irrlicht demo, running with nored + MLAA:

July 12, 2011cand 2 Comments »
FILED UNDER :Progress , Tickbox