Engine of 1.2.X Largely Finished

05/01/2005
A happy new year to all of you who use GeSHi :). Though while I might have been slacking around, I have instead been working on GeSHi 1.2.X, and I now have a progress report for you all :).

The GeSHi 1.2.X engine is largely completed now, with an approximate tripling of speed, sometimes better, for simple sources. But the best gains have been in the following areas:

1) Much better highlighting: You thought it was good before? Think again! The new engine handles any source code far better than ever before. And due to the way everything is set up, it is now easy to have multiple languages embedded in one source highlighting - for example PHP within HTML - and each language is highlightedd correctly. Furthermore, it handles such things as ?> enders in // comments for PHP much better. Here's a demo of what I mean.

Firstly, this is a file that I use for testing highlighted with GeSHi 1.0.4, using standard (non-strict) mode: here.

That's sorta bad, eh? For one thing, the <?php never seems to be highlighted, and it doesn't catch that the last single-line comment before the main block of HTML has a close-PHP (?>) in it that should stop highlighting of the single comment. Now here's the equivilant using the new parser. Strict mode off, and no attempt to highlight the HTML for starters: here.

Now that looks sorta bad as well, but that's because the source that we're giving it to highlight is not exactly pure PHP. At least it caught the ?> trick, and were you noticing the time? Twice as fast!

Here's another go at the same page. This time, we will use strict mode, and things should be much better. Here it is using GeSHi 1.0.4: here.

Well quite simply, that's poor. It seemed to bail out around the multiline comment with ?> in it, and hardly did any highlighting at all, not to mention move the doctype declaration down a line without us asking. How about the new parser? here...

Well how about that? Twice as fast, *and* highlighted everything correctly!

Now let's try the page, but now we'll use the new parser and full PHP-with-HTML-embedded parsing: here...

Are you drooling yet? ;). The CSS is highlighted, javascript inside HTML strings is highlighted - and all at the same speed as the best the old parser could do (which, if you recall, highlighted less than half the source correctly!)

Here's another set of examples, using the source from a well known open-source project:

Old parser, non-strict
New parser, non-strict
Old parser, strict
New parser, strict
New parser, full

Note the improved times and improved highlighting!

GeSHi 1.2.X update

15/12/2004
I've been working on GeSHi pretty hard recently, and have managed to get together a pretty stable context "psuedo parser" engine. It's really quite neat - defining languages now is less about characteristics of the the language, and more about the characteristics of contexts within that language. For example, PHP has the single string, which has a default style, type (for backward compatibility, more on that later), starter and ender, and knows how to highlight itself (by use of a child class of the generic GeSHiContext class). And HTML has the "entity" context - starts with &, finishes with ;, no children, default style (...)... it's going to be an interesting exercise for everyone, designing the trees for various languages.

Anyway, the advantages of this method? Well, they are manyfold:

1) No Regexps! This makes the engine up to 10 times faster to begin with - even reasonable sized sources get parsed in well under a second. I haven't even begun to optimise yet, there is still much code that is just "copy and paste"'d from one part to another, and it is parsing code much more quickly and without as much server load as the 1.0.X series.

2) "Perfect" highlighting - in GeSHi 1.0.X a lot of the work of highlighting was done by regular expressions, with really one regular expression in particular looking after the highlighting of keywords. This lead to the possibility that some languages who allowed certain symbols next to keywords not getting their keywords highlighted - even PHP suffered from this, &new is perfectly valid but unparsable by GeSHi, because semi-colons next to keywords were banned for other languages. However, in GeSHi 1.2.X, things will be different - because keywords will already be in context, a str_replace will be all that is needed. Furthermore, for contexts it is possible to define a child class to decide how context should highlight itself - so for example, I have the PHP Double string context highlighting interpolated variables (including class field variables, which are even highlighted a different colour!), and I can specify exactly the escape characters that are to be used - in GeSHi 1.0.X it simply decided anything after an escape character should be highlighted, which is silly for q for example.

3) More control over the highlighting - In GeSHi 1.0.X you only have the ability to do, for example, set_strings_style(), and that would set the style of every string in the source. With GeSHi 1.2.X you will be able to specify an identifier - for example, html/entity, or php/single_string, and the styles will apply only to that part of the source. This point leads on ...

4) Languages within languages! Yes, this is now possible! There's no more need for strict mode, and no more sad looking HTML embedded amongst your PHP - the HTML can be highlighted also. And, of course, the HTML context also has the CSS and Javascript child contexts for example (in fact, in my test HTML context file I've also embedded javascript highlighting inside html/double_strings that start with javascript:!)

Furthermore, the highlighting will be a lot "safer" - anyone who has seen the current parse_code method in GeSHi 1.0.X can see that it is quite unsafe - keywords are substituted for placeholders, which only at the end after methods, numbers and the like are parsed get replaced by the actual HTML. Instead in GeSHi 1.2.X, a completely new result string is built based on the code string, making it much less likely that some obscure string in the source code will make the parser bail out.

In short, things are looking up! But there is still quite some work to do before I release an alpha for everyone to play with...

1) I haven't written the second most important part yet - the in-context keyword/symbol/number/method parser. This is the part that will take the place of the parse_non_string_part() method in the current GeSHi. Although in GeSHi 1.2.X certain contexts will be able to overrule this code (for example, it makes sense to have strings overrule this, because they have no keywords in them usually and also have to deal with escape characters), this code will be a valid, centralised base for parsing the actual "meat" of the source code - the code that isn't in a string, comment etc.

2) There's only experimental support for PHP at the moment - in fact, the PHP language has a root context of HTML (if this seem strange, remember that PHP code isn't parsed unless it is within <?php ... ?> blocks, and the rest is straight HTML), which has children PHP, CSS, Javascript, HTMLTag, Entity and the like. And as I change and develop things, these contexts will grow, have more added to them, or may even disappear. Basically, the code is to volatile to release.

3) Only a couple of methods from the original GeSHi API are implemented - parse_code and enable_classes. Both are pretty much done (parse_code is neat now - just ask the root context to parse itself ;)), but of course there isn't any of the other methods that people would use so often - get_stylesheet for example.

So, lots of work for me to do still! But keep checking back for updates - I'll do another update as soon as I write the in-context parser :)

GeSHi version 1.0.4 Released

04/12/2004
Version 1.0.4 of GeSHi has been released. This version includes several minor bugfixes from version 1.0.3, most of which resulted because I was in too much of a hurry to release it. But in particular, if you use the method get_stylesheet, you should get version 1.0.4.

Concept work for 1.2.X is coming along smoothly, although not a line of code has been written (as with 1.0.0, I spent quite a long time drafting before even writing a line beyond what I already had as a phpBB extension). What I can say for sure is this - although the API will remain largely the same, language file structure is likely to become completely different, to account for things such as highlighting with PHP/HTML/CSS/Javascript all at once for example. I promise 1.2.X is gonna be huge :)

GeSHi version 1.0.3 Released

27/11/2004
New to version 1.0.3 of GeSHi is the ability to have multiple object splitters, support for different character sets as seen in Asian/Cryllic languages, support for targets for links and a few bugfixes. You can grab this new release from sourceforge.

In addition, I now have a full time job over the summer, so development will be slower than normal for a while (don't forget I am a New Zealander, with opposite summertime to what you may be used to!). However, after February next year development will resume normal pace and if all goes well, soon after that I will release a tester of the 1.1 series

The GeSHi project surpasses 1000 downloads

04/11/2004
The GeSHi project has officially surpassed 1,000 downloads at sourceforge.net. Although GeSHi is available elsewhere, sourceforge.net is the official counter. Thanks to all downloaders, and especially to plugin-developers around the world :)

GeSHi version 1.0.2 Released

27/10/2004
GeSHi version 1.0.2 has been released. The new release incorporates all of the changes that were available in the BETA version, and complete documentation, as well as numerous bug fixes.

Make sure you read the documentation fully! There are several new features available, and also a major change in the way you should specify styles for the outputted code.

Enjoy this new release!

On a side note, I'm still looking for language files, so keep sending them in to be included in 1.0.3 or 1.2.0.

Version 1.0.2 progressing

20/09/2004
The next version of GeSHi (1.0.2) has largely been completed now, with only testing and documentation to be completed before it is released. If you'd like to play with the new version before it is released, check out the CVS Repositry for it (in the 1.0.2 directory, I'm no great shakes with CVS).

Also, Firewiki and php-fusion seem to be using GeSHi as well: Good stuff! :)

GeSHi rated second at phpclasses

03/09/2004
GeSHi has been awarded second place at the phpclasses Innovation Award for July! This is a major achievement for a project so young, and has scored me the PHP Anthology series of books, to give me more ideas for how to improve GeSHi :).

GeSHi 1.0.2 is nearly ready. New features will include:

• Line numbering using ordered lists, for perfection (apologies to Amit Gupta - I said it was impossible without even trying it!)
• Many new languages, including asp, delphi, javascript, perl, python and more!
• Function URLs (eg. wherever "echo" is specified in PHP source will be made into a link to the appropriate documentation)
• Some lines can be specified to be highlighted "extra"

There'll be more features as well as better documentation, and a "roadmap" for future development of GeSHi. Look out for it soon!

New forum set up

25/08/2004
This site now has its own forum for you to post comments/complaints/queries about GeSHi. You could discuss how to make a language file (or post your language file for others to use), how to gain a certain effect with the output of GeSHi or just chat with other users of GeSHi. Check it out!

DocuWiki uses GeSHi

25/08/2004
DocuWiki, a simple to use but powerful WIKI system, is now using GeSHi to highlight code snippets posted in it. Check out another cool use for GeSHi!

In addition, this site now has an FAQ section for you to find answers to common questions in, and an examples page for you to see various demoes of GeSHi in action.