GeSHi 1.0.8 Released

08/08/2008
Right on the scheduled date of release we are proud to announce the new version 1.0.8 of GeSHi, released 8:08 on 8th August 2008. There's plenty of bug fixes and thirteen new language files in this release making a total of 109. Also this release had a strong focus on performance and quite a load of other features added, improved or otherwise revised. One of the biggest changes you will notice is the boost on performance this new release brings. We sometimes got up to 75 KBs on our computers. So even larger sources should now be no longer a problem. As announced in the previous release this release contains the second set of changed color schemes. I hope you like it; it's at least by far more readable. If not, this release brings a new external style settings file feature where you can override the language defaults without touching the language files: Simply create a file langname.style.php and copy in the STYLES-array as $styledata.

Another big change in this version - if not the biggest at all - is the rewritten highlighting engine that now allows for some more flexibility and fixes some bugs that seemed unsolveable in previous releases.

I bet I still missed some patches that you guys have sent in :). Please send again if you think I missed any!

Lately there has been an issue with especially old PHP versions like 4.3.X - if you are using such an old version please consider upgrading PHP or if this is not possible check the SF.net bugtracker on updates on open issues and I'm sure you'll get information on patches affecting you in time. Please also note that some features added in this release will be auto-disabled in old PHP versions as I won't go through the hassle of writing work-around for legacy PHP systems.

Once again we want to remind you of our code repository: There we collect source snippets to test, verify AND improve GeSHi. So if you have some small snippet of a language not yet supported there, feel free to donate it!

Download from the usual place, bug reports to the sf.net tracker please etc. etc...

BenBE.

Call for Participation and Release Preview

07/08/2008
Users of GeSHi, it's bright days for you with tons of new features and a much improved and streamlined code base: You'll get prettier code faster.

But we (BenBE and milian) are only two core developers hacking on GeSHi in our spare time. It's simply not possible that every single language of the 100+ supported ones is as shining and feature-rich as those we two use regularly. Often we don't even know the slightest of some of the languages we have support for. This is why we need your help:

• Send us example code!
  During the 1.0.8 developing process we decided to start a code repository with snippets of various languages to test GeSHi on - yeah, that one I already mentioned here earlier. It served us well and we could spot various errors in some of the languages we would not have tested otherwise. So if you have a nice example of some language, send it to us. To check which languages are already covered, take a look at https://geshi.svn.sourceforge.net/viewvc/geshi/branches/MISC_STUFF/coderepo/
• Make your favourite language shine!
  Starting with 1.0.8 we added quite some features which your language might not yet use (but should!). Take the new number highlighting for example, it supports various formats of floats, hex, binary and octal numbers. But we cannot know the required setup for all of the 100+ languages! So it's your turn to send us some patches. Or write us an email saying: "Language Foo supports Bar, Asdf, and Zark style numbers."
  Maybe add a codesnippet as well (see above). This will help us for future releases.
• If you are not able to fix your language by yourself, but spot an issue with GeSHi - please help by submitting a bugreport with code example and expected behaviour description. That way we can work something out. And don't forget to include some contact details so we can contact you in case of questions.

Other NEW features you might like:

• Comment Regexps and Strict Block Regexp delimiters
  You already noticed the limitations of some tstuff simply not showing up correctly as it's too complex for simple find and replace? This is your chance!
• Support for multiple styles for strings
  Your language supports different ways of enclosing literal data? You want them to show up differently depending of the type of delimiters? No problem with GeSHi 1.0.8! You just specify styles for different string delimiters and get what you want.
• Multiple styles for numbers
  What goes for strings is possible for numbers too! Many languages have standard ways to define integers, floats or even octal or binary and hexadecimal values. Coming with the next release you can select what types of numbers your language supports and how to style them.
• PARSE_CONTROL madness
  We give much more control to language files, which makes it possible to set per-keyword-group DISALLOWED_BEFORE/AFTER and more.
  
  You can now also explicitly disable given features of GeSHi if you don't need it, take a look at the text.php language file for an example.

And this are only some of the various improvements! Overall there's speed improvement all over the place! Fell free to test it out yourself!

Also there are - as always - some security fixes, including one somewhat critical that might be used to produce heavy load on your server. More details on this as soon as the final release is out.

As always we like to hear your oppinion. So feel free to use the usual contact mechanisms to send in your feedback.

Regards,
BenBE and milian

Code, samples and statistics

04/08/2008
I always was wondering how famous the online demo on this site really was. Well, it's not exactly what people call a pastebin, but you can use it as such if you store your source when initially highlighting.

But what I always was wondering about when looking at all those pastebins out there: What was the percentage of each highlighted language? Well, this depends on the visitors of such a pastebin, but also on how regularly they come visit this pastebin or if they update their sources regularly and thus produce dupes.

So how comes I'm doing this post? I stumbled upon the internal cache tables for the demo application that holds all the information to provide you with those little stats of average highlighting time, snippet size and the like. I played around with it a bit and though: There hasn't been such information out yet, so maybe people will like to hear about it. So here it comes:

For results to be as current as possible I did a quick lookup of most values just before starting thist post. At that time I had 123680 entries in the DB covering all languages GeSHi 1.0.8rc3 (the version on the server) supports. These entries cover dates from 12th June 2004 at 02:02 am to today, 04th August 2008 08:28 am, i.e. a total of 1514 days, and 6:30 hours. Given an even distribution of requests this would make one request every 15 minutes (1058 seconds between requests).

The traffic to the demo site is somewhat interesting too. The total size of all samples together is 336.41 MiB (352,754,019 Bytes) with an average of 2852 Bytes per request - roughly one page per request. It took about 52338.23 seconds of processing time for the server which equals 14 hours, 32 minutes and 18 seconds of continious work for the server.

Let's move on to the language ranking: Currently there are 108 supported languages in 1.0.8rc3. You see: 108 languages in 1.0.8rc3 ;-) All of them have been used although the distribution concentrates on about 20 languages. With 22857 hits and 70.75 MiB of input data at first place is (without surprise) PHP, followed (to my surprise) by ActionScript (29.58 MiB input)in 12313 samples. After this follows HTML (html4strict, 6449 hits, 29.39 MiB), C++ (cpp, 6211 hits, 15.39 MiB) and ABAP (abap, 5124 hits, 7.45 MiB).

Although this ranking stays quite stable when looking at input size and total hit count it dramatically changes once you look at the average sample size: If you do this you get ActionScript (french) ranked first with about 8.4 KiB per highlighted sample, followed by Plaintexts with 6.5 KiB and Tcl with 6.3 KiB per sample.

Let's draw some conclusions out of this:

• People highlight different stuff than you expect
• Although languages like C++, CSS, JavaScript, Perl and Python are quite famous around the globe they do not seem to draw too much attention to this demo
• People seem to test primarily small samples and rarely highlight large ones.

And now to another statistical fact: The next release is overdue ;-) The work on the next release has already reached the attribution stable and only minor issues are left, but there is still some work open that will have to be fixed before the next version can be released to the^W^W^W^W go wild. This will - based on the current schedule for the release - be on 08th August 2008. The next version will not only have a new sub-minor release number, but also loads of new features, yet another bunch of optimizations and new languages, but also many other features I'll introduce to you soon. So stay tuned for the next updates!

BenBE

An old bug dies ...

25/07/2008
This might be surprising for you guys, but it isn't ;-) So, what happened?

Many of you might have noticed at some point, that GeSHi sometimes had problems with code like



as described in a bugreport opened on 19th Oct 2005. The reason for this issue and also the reason it took so long to fix it was the way GeSHi determined where PHP-code (and other stuff for other languages like HTML, CFM) was. This has been changed today by a patch and thus an extension to the parser now supporting an somewhat more elaborate way of finding PHP-Blocks. The old way was simply to look for the start end end of such blocks by simple means like strpos. This works for simple cases like when the ender may not appear inside such an block itself, but breaks as soon as this prerequisite is not given - which it isnn't for CFM and PHP.

So the solution was simple: Allow the language file to describe a way to identify how such a block has to look like. This is done with regular expressions in the case of PHP and if you look at the most recent trunk version of the PHP language file you might notice some weird looking string in the place where according to the old documentation an array should have been. Well, that it is: The Strict Block RegExp! - Our new feature that already fixed an problem with the PHP language file, but will be reused to fix some more simular issues we already know about with other language files.

Therefore we might remind you of the Code Repo I posted about previously, since we only can fix issues we know about and can reproduce ourselves.

But Strict Block Regexps isn't the only new feature you'll note in the upcoming release. If you're curoius feel free to test out the Trunk SVersion of GeSHi or have a look at the Changelog we constantly update inside the SVN.

Farewell SF#1330968, that grew 1009 days, 14 hours and 27 minutes old! We'll miss you!

BenBE.

Call for Code

21/07/2008
Maybe some people may already have noticed that we added a new directory to the SVN repostiory that we use for testing the highlighting of GeSHi.

This has allowed us to do many improvements to GeSHi recently while ensuring at the same time we didn't break any old stuff. But at the moment we are missing examples for various languages in our Code Repo so we can't check highlighting for all languages yet.

So what we need is:

• Small pieces of code of approximately 10 - 50 lines of code
• Typical code, i.e. they should illustrate what Code of this languages usually looks like, real-world examples preferred
• Preferrably for yet uncovered languages
• Not more comments as needed, basically as much comments as usual code of this language would have.
• Avoid code like the Sun JRE where there's one page of documentation for 3 lines of code. If you have some code like this, try reducing the comments
• The code snippet in itself should be free of syntax errors and be valid code for this language.
• The code should use as many features of the language as possible. If this becomes too elaborate, split it up into multiple files.
• The code must be available to be published in our repo (and stay there)

There are already some example snippets we gathered during debugging GeSHi, so feel free to contribute to them.

Please send your Code Snippets you'd like to contribute to this Code Repo to nigel @@ geshi .dot. org.

And if there is enough code available we even might release some additional package we are working on. But due to the lack of code, we couldn't test it on real-world examples yet.

Thanks in advance for any contributions!

BenBE and milian

100th language file for GeSHi included

20/06/2008
Today in the afternoon (local time ;-)) while I was asking for support for some older issue "blocking" up the bugtracker at SourceForge I stumbled upon the people of the Boo interpreter. One of them already had a complete Boo language file laying around - that was number 99 of the languages added to GeSHi.

So now comes the interesting part: After negotiating how to transmit the file (as the SF.net tracker denied him attaching the file) he suddenly asked, if I'd mind getting yet another language file: CIL or Common Intermediate Language which is the .NET assembler language common to all .NET compilers. So I asked him to file me a bug attaching the language file and there we got: Language File #100!

There had been just another two files added in meantime so language file count is already up to 102, but I hope you folks will keep on sending in language files to add to improve language support even further.

GeSHi 1.0.7.22 Released

08/06/2008
After just 14 days after the proposed release date comes this new version 1.0.7.22 of GeSHi. There's plenty of bug fixes and nine new language files in this release. Also this release had a strong focus on performance and quite a load of other features added, improved or otherwise revised. One of the biggest changes you will notice is the changed color scheme for this release. I hope you like it; it's at least by far more readable - and other language files that use the old defaults are soon to follow.

I bet I still missed some patches that you guys have sent in :). Please send again if you think I missed any!

This version should fix some problem with the Symbol highlighting added in the last release - thank you guys for the patience and all the help with figuring the exact cause and possible workarounds out.

Lately there has been an issue with especially old PHP versions like 4.3.X - if you are using such an old version please consider upgrading PHP or if this is not possible check the SF.net bugtracker on updates on open issues and I'm sure you'll get information on patches affecting you in time.

Download from the usual place, bug reports to the sf.net tracker please etc. etc...

BenBE.

GeSHi moved to another Server

08/06/2008
After about one week of waiting for international authorities the GeSHi website finally has moved to another server! As this server seems to have some less power than the old one I'd like to ask you to be patient - all your requests will be served (I hope).

As I somehow have to manage keeping the software on the server up-to-date there will be regular downtimes of a few minutes - primarily after midnight local time (CET).

The server now allows for 64 MB RAM being used for highlighting with maximum input of 2 MB. Execution time is up to 5 minutes, so everyone should get its code - even if it's somewhat bigger. Oh, and I nearly forgot to mention: qbnz.com (as well as geshi.org) are using PHP5 instead of PHP4 which the old server used. This change has made (and probably will continue to make a bit of trouble. Also this means that as PHP differs a bit between versions 4 and 5 your result might look a bit different than on the server - there have a least two such reports lately, that both will be addressed soon.

If you experience problems, feel free to tell me about them. If the server is too slow, just feel free to use the Donate Button at the left and I'm sure, I can do something about it ;-)

So long,
BenBE.

GeSHi 1.0.7.21 Released

23/03/2008
Very loooong overdue, but 1.0.7.21 is out. There's plenty of bug fixes and one new language file in this release, and I bet I still missed some patches that you guys have sent in :). Please send again if you think I missed any!

Also this version adds some new feature - Symbol Highlighting - which superseeds and somewhat replaces Bracket Highlighting, I already know of one issue I couldn't fix before the release. Please check the SF.net bugtracker on updates on open issues and I'm sure you'll get the patch for that one in time.

Download from the usual place, bug reports to the sf.net tracker please etc. etc...

GeSHi got a new Owner

15/03/2008
After a long time of inactivity Nigel handed over GeSHi to Benny Baumann in the hope that the developement of GeSHi 1.0.X and GeSHi 1.1.X (which will someday hopefully become 1.2.X) being continued.

And work will continue! The next steps will be stabilizing some features added in GeSHi 1.0.X lately and adding more languages. After this developement will focus on the developement branch, reimplementing the old-style API, although with a somewhat changed behaviour as the branch and developement versions differ in some points which have to be accounted for.

As always your help is really appreciated and I'm looking forward to all the patches you send in.