Code, samples and statistics

04/08/2008
I always was wondering how famous the online demo on this site really was. Well, it's not exactly what people call a pastebin, but you can use it as such if you store your source when initially highlighting.

But what I always was wondering about when looking at all those pastebins out there: What was the percentage of each highlighted language? Well, this depends on the visitors of such a pastebin, but also on how regularly they come visit this pastebin or if they update their sources regularly and thus produce dupes.

So how comes I'm doing this post? I stumbled upon the internal cache tables for the demo application that holds all the information to provide you with those little stats of average highlighting time, snippet size and the like. I played around with it a bit and though: There hasn't been such information out yet, so maybe people will like to hear about it. So here it comes:

For results to be as current as possible I did a quick lookup of most values just before starting thist post. At that time I had 123680 entries in the DB covering all languages GeSHi 1.0.8rc3 (the version on the server) supports. These entries cover dates from 12th June 2004 at 02:02 am to today, 04th August 2008 08:28 am, i.e. a total of 1514 days, and 6:30 hours. Given an even distribution of requests this would make one request every 15 minutes (1058 seconds between requests).

The traffic to the demo site is somewhat interesting too. The total size of all samples together is 336.41 MiB (352,754,019 Bytes) with an average of 2852 Bytes per request - roughly one page per request. It took about 52338.23 seconds of processing time for the server which equals 14 hours, 32 minutes and 18 seconds of continious work for the server.

Let's move on to the language ranking: Currently there are 108 supported languages in 1.0.8rc3. You see: 108 languages in 1.0.8rc3 ;-) All of them have been used although the distribution concentrates on about 20 languages. With 22857 hits and 70.75 MiB of input data at first place is (without surprise) PHP, followed (to my surprise) by ActionScript (29.58 MiB input)in 12313 samples. After this follows HTML (html4strict, 6449 hits, 29.39 MiB), C++ (cpp, 6211 hits, 15.39 MiB) and ABAP (abap, 5124 hits, 7.45 MiB).

Although this ranking stays quite stable when looking at input size and total hit count it dramatically changes once you look at the average sample size: If you do this you get ActionScript (french) ranked first with about 8.4 KiB per highlighted sample, followed by Plaintexts with 6.5 KiB and Tcl with 6.3 KiB per sample.

Let's draw some conclusions out of this:

• People highlight different stuff than you expect
• Although languages like C++, CSS, JavaScript, Perl and Python are quite famous around the globe they do not seem to draw too much attention to this demo
• People seem to test primarily small samples and rarely highlight large ones.

And now to another statistical fact: The next release is overdue ;-) The work on the next release has already reached the attribution stable and only minor issues are left, but there is still some work open that will have to be fixed before the next version can be released to the^W^W^W^W go wild. This will - based on the current schedule for the release - be on 08th August 2008. The next version will not only have a new sub-minor release number, but also loads of new features, yet another bunch of optimizations and new languages, but also many other features I'll introduce to you soon. So stay tuned for the next updates!

BenBE