cand @ gsoc

The first week of GSOC is up. This is the weekly review of what’s happened.

First I re-read Drepper’s “How to write shared libraries”, to refresh my memory on the correct techniques.

I then proceeded to test those against the Monkey plugins, which are shared libraries by themselves, although used via dlopen instead of dynamic linking.

The plugins had no visibility control whatsoever, exporting a hundred or so unneeded global variables and functions. This causes slower startup for Monkey and each plugin has 5-10kb of extra size.

The only needed exports are those of the plugin API; those starting with _mkp_ and the informational struct plugin_info.

First I attempted to use GCC pragmas and hiding; after discussion on the ML that approach was deemed unsatisfactory, given its requirement of GCC 4.0. (side note: I really think a minimum GCC version should be decided, so that advanced features may be used without ambiguity such as this.)

So the patchset was revised to use the usual approach of defaulting to hidden and specifically exporting each necessary symbol. This results in more writing (one addition per symbol, instead of two per file), but is supported in older compilers too.

As a result, each plugin’s exported symbols dropped from > 100 to about 30, with a corresponding drop in the binary size as well.

This work has not yet been merged.

Later in the week, in preparation for testing my own work later on, and in celebration of the release of LLVM 3.1, I ran monkey through Clang’s static analyzer.

Clang found about 20 issues, most of which were possible crashers. Some were dead writes.

Patches to fix most of these have been applied to master.

Lib-wise, I started on designing the API. It follows the general style of other embeddable www server libs. Right now the API contains the necessary functions for all desired functionality, as well as the necessary enums (assuming I didn’t forget anything ;)).

The public header was posted on the ML as a request for comments; only Ed (my mentor) commented. I assume it’s more of a lack of interest than a general silent acceptance.

Conclusion

• Clang’s analyzer had never been ran on the codebase, based on the results
• Plugin symbol visibility awaiting merging
• Library design right on schedule

Assuming no incoming comments on the API, next week I’ll start on the tasks of week 3-4, moving one week ahead of schedule. The tasks include setting up the build system, function stubs, and then the library implementation.

The main goal is to let an application serve its own data.

Given this, the only plugins that make sense are liana, liana_ssl and perhaps auth. Liana* don’t need the plugin hooks, and auth would require further changes, so for the first edition I think it makes sense to disable the plugin hooks altogether. (note: this means stage_run only)

As the first two are access control, they would be optional. Return values for both could be MK_TRUE/MK_FALSE. (pass/forbidden)

For the data handler, the app would receive the virtual host and URL. It should only concern itself with the actual data (+headers, status), TCP_CORK and various other parts are out of scope for the lib use case.
The data handler would also return either true or false, signalling whether data would be sent. Returning false, monkey should fall back to accessing normal files in the specified directory.

This would make it possible to serve dynamic pages, but have the css, js and images be in a static folder somewhere, saving the application from having to serve them itself in the data callback.

Config

So far it appears that most config parameters cannot be easily changed at runtime. This is OK for the lib, so it can only allow config changes when the server is not running.

As for what config parts should be supported, everything. Including virtual hosts.

Soname

I think it makes sense to bind the library version to the Monkey version. Monkey 1.0.0 -> libmonkey-1.0.0.so.

Before May 21: familiarize myself with the Monkey codebase

Weeks 1-2: draft the API design, request comments, iterate

• design the various API calls
• assemble them as an easy to read text file (header + comments)
• post this RFC to the mailing list for public feedback
• address all shortcomings and comments

Weeks 3-4: change the build system to build a shared library as well.
Implement the API.

• create stubs for all the functions, so that the build system can be tweaked
• add –enable-shared-library and –disable-shared-library options to the configure script
• edit the Makefiles according to those options: the library won’t include the binary main function, and the binary won’t need the library functions.
• with the build system in place, start implementing the functions in priority order (start, stop, restart, configure, callbacks, then the rest)

Weeks 5-6: Create various examples, testing the viability of the API

• first example: simple “hello world”
• second example: directory listing
• third example: something interactive, maybe a quiz

Weeks 7-8: Documentation, any tweaks left over

• create man pages for the library functions
• any cleanups that remain
• run the code through all static analyzers and checkers I have available; fix all warnings

Right now, I’ve gotten familiar with the Monkey codebase. I started with fixing compiler warnings and proceeded to benchmarks and profiling, and many cleanups are already integrated in master.

With the official program starting today, it’s API design time.

I asked on #nginx (freenode) why nginx scaled badly (5% improvement) when going from 1 to 5 workers.

The consensus was that nginx is designed so that every worker calls accept(). I think that this is a bad design decision, but given a modern linux host, it was said that this is no longer an issue.

Nginx has an option to disable the mutex controlling accept(), accept_mutex off in the event block of the main config file. Given how important this one option is, I’m extremely disappointed that

• It’s not the default on suitable systems
• It’s not even documented in the shipped config files

This one option is why nginx scales terribly in its default configuration. When compiled on a suitably new linux system, ideally it would turn it off by default. And in all cases, it should be documented in the shipped config files, even if commented out.

Tested with 5 workers and accept_mutex off, nginx got 28000 +- 0.2%.

This puts nginx up to the expected numbers.

Curious from the even results on the Raspberry Pi, I thought it was time to test nginx on the 6-core comp too (Phenom II X6 2.8GHz).

The website was the static one shipped with monkey, containing html and one picture. Siege was run with the usual settings, benchmark mode, 10 concurrent threads and 10 seconds of test time.

siege -c10 -t 10S -b localhost:1234

Versions:

• nginx 1.2.0

• Monkey ea83cb6860f9555011a4959294bd7d765b1fe533 plus some small fixes

Monkey was tested only in the default config (5 workers). Nginx was tested both in its default config (1 worker) and with 5 workers, to gain even ground with Monkey.

All numbers are averaged over three runs. Without further ado:

Woah.

Given the equal results under the Pi, I wasn’t expecting this. Further, it was surprising how little nginx gained from moving to 5 workers (+1 management thread, all on a 6-core cpu).

Here’s the raw data for one of the three runs:

      Date & Time,  Trans,  Elap Time,  Data Trans,  Resp Time,  Trans Rate,  Throughput,  Concurrent,    OKAY,   Failed

nginx default, 1 worker

2012-05-04 18:02:16, 122599,       9.75,         159,       0.00,    12574.26,       16.31,        9.34,  122599,       0

nginx 5 workers

2012-05-04 18:04:15, 136634,       9.99,         177,       0.00,    13677.08,       17.72,        9.34,  136635,       0

monkey default

2012-05-04 18:47:07, 254261,       9.01,         330,       0.00,    28219.87,       36.63,        9.33,  254261,       0