Over a year ago I posted an XSL file for prettying up Graphviz diagrams. 

I got lots of feedback, and recently I've been fixing various bugs, and I finally got around to adding support for more of the Graphviz node shapes etc.

Here's an example of various node types and colors with the new version. Note that while rounder corners are "sort of" working, I haven't found a good way of making the gradient fill work nicely with it. The current version has been tested with Graphviz 2.20.2.

(Scaled down a bit to fit my layout; if you can't see it you need a browser that can embed SVG's, such as any recent Firefox or Safari)


The XSL file can be found with various other Graphviz related tools at my diagram tools repository on GitHub

This is a fairly substantial set of changes, so please do let me know if you find any bugs (if so, I appreciate an example dot-file as well as the Graphviz XSL output and the "cleaned" XSL output, and the version number of the Graphviz version you're using - please try to upgrade to at least 2.20.2, though).

Thanks to Jonas Tingeborn, Earl Cummings and Kevin Keraudren for various bug fixes and help debugging. Thanks to Michael Kennedy for posting a variation with lots of gradients (I've pilfered a smallish subset for my updated version - but you can copy the rest from his version if you prefer)

One thing that comes up time and time again when I mess around with a system, is quickly looking at frequencies of various things - for example disk usage by sub-directory, or referrer entries in my Apache access log.  Like this:http://librsvg.sourceforge.net/

# cat /var/log/httpd/access_log | cut -d' ' -f11 | grep -v '"-"' | grep -v hokstad.com | sort | uniq -c | sort -rn | head -n 10
 74 "http://www.google.com/reader/view/"
 42 "http://www.rubyflow.com/items/1606"
 35 "http://www.dzone.com/links/creating_graphviz_graphs_from_ruby_arrays.html"
 27 "http://www.dzone.com/links/rss/creating_graphviz_graphs_from_ruby_arrays.html"
 20 "http://www.reddit.com/r/ruby/"
 12 "http://www.reddit.com/r/ruby/comments/7tw1a/creating_graphviz_graphs_from_ruby_arrays/"
 9 "http://www.graphviz.org/Resources.php"
 8 "http://www.netvibes.com/"
 8 "http://www.google.com/reader/view/#overview-page"
 8 "http://www.google.com/notebook/fullpage"

Especially the "sort | uniq -c | sort -rn | head -n something" is a very frequently recurring pattern, in order to get a list of something in descending order of frequency.

But I'm difficult. I want something visual; a chart. Something like this:


Incidentally, there's a nice Ruby package called SVG::Graph that allows you to generate SVG's from this. The above is a PNG for best compatibility, but here is the SVG version - at least Firefox renders it better than rsvg which I used to generate the PNG.
Assuming you install SVG::Graph from the above page, it's pretty simple to generate the SVG's - just pipe the output from the above command straight into this script:

require 'SVG/Graph/BarHorizontal'
data = []
fields = []
ARGF.each do |line|
 line = line.chomp.split
 data << line[0].to_i
 fields << line[1]
end
graph = SVG::Graph::BarHorizontal.new(:height => 20 * data.size, :width => 800, :fields => fields.reverse)
graph.add_data(:data => data.reverse)
graph.rotate_y_labels = false
graph.scale_integers = true
graph.key = false
print graph.burn

It tries to be reasonably intelligent about adjusting the height of the graph, but you might want to adjust the width and other parameters. My goal was to get something simple that'll "just work" when I pipe the output of "uniq -c" into it, but any space separated data with a number first and the label afterwards should work as long as the data set is small enough that the graph it produces doesn't become ridiculously large. I'll probably extend it with some command line options, and I want to change the styles used to set colors etc., but the above is enough for 90% of what I want. Note that SVG::Graph also support pie charts, line charts etc.

It's rare I see javascripts that makes me go wow!.

I've been experimenting with the canvas element myself, and there are some cool demos of it, but I loved this both because it looks good and because the resulting code is so simple..

At Edgeio we had a fairly complicated network setup, and at one point I quickly hacked together a Ruby script to merge the paths generated by multiple traceroute runs together into directed graphs, showing the routing from a few selected host in our environment to all our other hosts. I generated dot-files suitable for Graphviz from it.

It was a helpful way of looking for weird inconsistencies in routing, in particular between our two locations.

Unfortunately, when Edgeio closed down I think the script was lost, and in any case if it isn't I wouldn't be able to get permission to release it without more hassle than it'd take me to recreate it from scratch. So recreate it from scratch is exactly what I did.

Here's an example of a traceroute from www.hokstad.com to www.gmail.com, docs.google.com and www.google.com (scaled down):

(Gradients and shadows courtesy of my XSL transform to make graphviz output prettier)

There's a couple of caveats: I just strip out failed probes, and I don't try to reconcile the names of the endpoints (which I preferred to include for readability) with the IP addresses of the trace, so the first/last grey nodes before the named/blue nodes may be redundant.

This script by default runs traceroute 3 times for each target, and that's the reason why there are more possible paths than endpoints, and it illustrates failover and/or load balancing mostly, but can also be affected by fluctuations in dynamic routing. It's usually fairly stable, and in fact at Edgeio I found several network problems by re-running the script when something was up and looking at how the routing had changed. 3 runs seemed sufficient for my use, but for large networks adding more may give a better picture of the routing.

The Ruby script

You can find the full script here, but here are the guts:

First I defined a convenience method to run traceroute and capture the output. This is intended for a POSIX OS (Linux / Unix / BSD's), but mainly requires a working traceroute where the output is a number of lines starting with a hop count and then the ip address. TRACEROUTE must be set to a valid traceroute command.

TRACEROUTE=`which traceroute`.chomp
def traceroute host
  `#{TRACEROUTE} -n #{host}`
end

The TraceViz class does the gruntwork. @edges contains the edges of the graph, in other words which pairs of ip addresses represent a hop further in the network. @nodes contains a set of the ip addresses found. @targets contains the hostnames of the start and end-points - it's used only to style them differently:

class TraceViz
  def initialize(times,timeout)
    @times,@timeout = times,timeout
    @edges = Set.new
    @nodes = Set.new
    @targets = Set.new
    @this_host = Socket.gethostname
    @targets << @this_host
  end

The #trace method executes the traceroutes, and enforces a timeout:

  def trace host
    @times.times do |i|
      STDERR.puts "Trace ##{i+1} for #{host}"
      Timeout::timeout(@timeout) do
        process_trace(host,traceroute(host))
      end rescue nil
    end
  end

#process_trace handles the parsing of the trace, by splitting the output into lines, extracting the IP addresses, and then adding each of them to @nodes, and adding each pair to @edges. I don't care if we've seen them before, since I use Set's so the previous (identical) nodes/edges will just overwrite the same values:

  def process_trace host,trace
    @targets << host
    trace = [@this_host] + trace.collect do |line|
      line=line.split
      line[0].to_i > 0 && line[1] != "*" ? line[1] : nil
    end.compact
    trace << host
    trace.each {|h| @nodes << h }
    trace.each_cons(2) {|h1,h2| @edges << [h1,h2] }
  end

Finally #to_dot generates a graphviz compatible directed graph:

 def to_dot
    res = "digraph G {"
    @edges.each { |h1,h2| res << "   \"#{h1}\"->\"#{h2}\"\n" }
    @nodes.each do |n|
      color = @targets.member?(n) ? "lightblue" : "lightgrey"
      res << "  \"#{n}\" [style=filled fillcolor=#{color}]\n"
    end
    res << "}"
  end
end

Running it and generating the image

First run the script to generate the dot-files, and then generate an SVG file from it:

ruby traceviz.rb www.google.com www.gmail.com docs.google.com >trace.dot
dot -Tsvg trace.dot >trace.svg

Optionally, process the script with my XSL transform to make it prettier (adding the gradients from above etc) - I'm using xsltproc from libxslt:

xsltproc notugly.xsl trace.svg >trace-notugly.svg

Then I used "rsvg" from librsvg2 to turn it into a PNG:

rsvg trace-notugly.svg traceviz.png

Of course these steps are easily enough wrapped into a script.

I keep trying to learn more about Cairo, and cheesy logo effects seems to be as good a way as any. This evening I was playing around with clip paths, and decided to use it to create some reflections. Here are the results:

You might want to take a look at my previous examples. This one, as my last one, uses functions from the first example. It also requires that Cairo and the Cairo Ruby binding are installed.

• Simple drawing in Ruby with Cairo
• Draw a logo with gradients

And here's what I did.

First I took the script I used to draw the logo with gradient and replaced the background with black instead of white, and made the border blue. You can find the logo I used here

I named the file "in.png".

Then it was time to write the script - You need to require 'cairo' and add the cairo_image_surface helper function from my first example. As usual I then defined a few variables, starting width/height. The script assumes that the input logo ends at h/2, so adjust accordingly if you use a different image. I then create the surface and open the block

h = 170
w = 350
black  = [0.0,0.0,0.0, 1]
cairo_image_surface(w,h, black) do |cr|

The next step is new: I create an image surface from the PNG file. I then set the image as the source. The second and third argument to set_source() are the coordinates where the upper left point of the surface should map to the context (cr) we're drawing with.

I then draw a rectangle with the surface, and fill it. This has the effect of copying the logo onto the image surface we're drawing with.

  image = Cairo::ImageSurface.from_png("in.png")
  cr.set_source(image,40,0)
  cr.rectangle(0,0,w,h/2)
  cr.fill

Then it's time for the reflection effect. The left one first:

 (0..h/2).each do |y|
    cr.rectangle(0,h/2 + y,w,1)
    cr.clip
    cr.set_source(image,40-0.3*(y+1),y*2)
    cr.paint
    cr.set_source_rgba(*black)
    cr.paint(0.1+y/60.0)
    cr.reset_clip
  end

This code is fairly straightforward:

For the lower half of the image, we first create a rectangle of 1.0 unit high and the full width. We use "cr.clip" to define this rectangle as the current clip mask. The clip mask defines what part of what you draw subsequently that show. In effect we're creating a "slit" one pixel high we can draw on.

I then use set_source to reposition the source image (the original logo). By manipulating the y position we turn the image upside down. By manipulating the x position we slant the reflection to the left.

I then use cr.paint to pain the source image onto the "slit" I established by clipping.

Then I use set_source_rgba to switch to black "paint" and paint with alpha blending to gradually erase more and more the further down we get by painting over the slit with black. I chose this approach to making the reflection fade because the logo I used didn't have a transparent background. I could make the background transparent, but didn't think about it at the time. So the alternative approach is to drop painting over with the background color and instead set the alpha mask for the first "cr.paint" call and gradually reduce the alpha value instead of increasing it as we do for the second paint call.

All you need to do to get the wave effect instead is replace the set_source call we used to move the logo each line:

    cr.set_source(image,40-Math.sin(y*0.2)*y/7.0,y*2)

Experiment with different factors to increase the wave amplitude or frequency.

Last but not least you need to write the surface out, and close the block:

  cr.target.write_to_png("test.png")
end