I wrote about syntax hightlighting in Ruby earlier. The Ruby Syntax library supports Ruby, YAML and XML out of the box. But it's also pretty easy to extend to handle other languages.

Since I've been writing my compiler in Ruby series and including a lot of x86 assembler, I figured I'd see how much (or little) work adding a syntax highlighter for assembler would take.

It's by no means perfect - I've only spent half an hour or so throwing this together, but it's reasonable, and easy enough to keep adjusting.

Here it is:

require 'rubygems'
require 'syntax'
class AsmTokenizer < Syntax::Tokenizer
  def setup
    @state = :newline
  end
  def step
    @state = :newline if bol?
    if @state == :newline
      # Handle labels and operands
      if label = scan(/[a-zA-Z.][a-zA-z0-9_]*:/) then start_group :label, label
      elsif words = scan(/\.[a-zA-Z0-9_]*/)
        start_group :directive, words
        @state = :operands
      elsif words = scan(/[a-zA-Z]+/)
        start_group :operator, words
        @state = :operands
      else start_group(:normal, getch)
      end
    else
      # Handle operators and assorted punctuation
      if words = scan(/,/)                then start_group :comma, words
      elsif words = scan(/[\-0-9][0-9]*/) then start_group :number, words
      elsif words = scan(/%[a-zA-Z]+/)    then start_group :register, words
      elsif words = scan(/[\.a-zA-Z][a-zA-Z0-9]*/) then start_group :label, words
      elsif words = scan(/\$/)     then start_group :value, words
      elsif words = scan(/[\(\)]/) then start_group :paren, words
      elsif words = scan(/\".*\"/) then start_group :quoted, words
      else start_group(:normal, getch)
      end
    end
  end
end
# Register the custom highlighter
Syntax::SYNTAX['asm'] = AsmTokenizer

I don't have time do write a lot of explanation - the Syntax manual does a reasonable job of describing it. The one pitfall to be aware of, is that you must make sure that your #step method advances at least one character no matter what, or you'll get stuck in an infinite loop.

Here's an example of how to use the new highlighter:

require 'syntax/convertors/html'
convertor = Syntax::Convertors::HTML.for_syntax "asm"
puts convertor.convert( File.read("/tmp/step5.s"))

And here's some CSS to color the output:

body { background: black; }                                                                                                          
.directive { color: purple; }                                                                                                        
.comma     { color: white; }                                                                                                         
.paren     { color: white; }                                                                                                         
.value     { color: white; }                                                                                                         
.number    { color: yellow; }                                                                                                        
.label     { color: blue; }                                                                                                          
.register  { color: brown; }                                                                                                         
.operator  { color: lightgrey; }                                                                                                     
.quoted    { color: green; }                                                                                                         

And some example output from the compiler series:

        .text
.globl main
        .type   main, @function
main:
        leal    4(%esp), %ecx
        andl    $-16, %esp
        pushl   -4(%ecx)
        pushl   %ebp
        movl    %esp, %ebp
        pushl   %ecx
        subl    $4,%esp
        movl    $.LC0,(%esp)
        call    puts
        addl    $4, %esp
        popl    %ecx
        popl    %ebp
        leal    -4(%ecx), %esp
        ret
        .size   main, .-main
        .section        .rodata
.LC0:
        .string "Hello World"

I love syntax highlighting - I have gotten so dependent on it I find it painful to use editors without it. So it's extremely annoying to post code snippets without it online too.

Thankfully it's trivially simple to do in Ruby. For this blog I really wanted a solution that lets me keep the text clean in the database, but marks up the posts when rendering, so after some searches I ended up doing this:

gem install hpricot
gem install syntax

Then I added this:

require 'hpricot'               
require 'syntax/convertors/html'
def filter_content(content)
    h = Hpricot(content)
    c = Syntax::Convertors::HTML.for_syntax "ruby"
    h.search('//pre[@class="ruby"]') do |e|
        e.inner_html = c.convert(e.inner_text,false)
    end
    h.to_s
end

Then all I need to do is call filter_content() with the post content, and put any code in pre tags with class="ruby".

(Hpricot is a great tool btw - I use it all over the place for mangling HTML.)

The remaining piece is just some CSS. The CSS was adapted from CSS found here, but I ended up changing most of the colors to match my editor settings better:

pre { background: #111122; padding: 10px; color: #228822; }
.ruby .normal { color: #fff; }
.ruby .comment { color: #005; font-style: italic; }
.ruby .keyword { color: #A44; font-weight: bold; }
.ruby .method { color: #44f; }
.ruby .class { color: #0c4; }
.ruby .module { color: #050; }
.ruby .punct { color: #668; font-weight: bold; }
.ruby .symbol { color: #ff0; }
.ruby .string { color: #4f4; }
.ruby .char { color: #F07; }
.ruby .ident { color: #fff; }
.ruby .constant { color: #0c4; }
.ruby .regex { color: #B66; background: #FEF; }
.ruby .number { color: #F99; }
.ruby .attribute { color: #fc4; }
.ruby .global { color: #7FB; }
.ruby .expr { color: #227; }
.ruby .escape { color: #277; }