Tag: visualization

2009-02-04 02:46 UTC Simple charts in Ruby using SVG::Graph

One thing that comes up time and time again when I mess around with a system, is quickly looking at frequencies of various things - for example disk usage by sub-directory, or referrer entries in my Apache access log.  Like this:http://librsvg.sourceforge.net/
# cat /var/log/httpd/access_log | cut -d' ' -f11 | grep -v '"-"' | grep -v hokstad.com | sort | uniq -c | sort -rn | head -n 10
 74 "http://www.google.com/reader/view/"
 42 "http://www.rubyflow.com/items/1606"
 35 "http://www.dzone.com/links/creating_graphviz_graphs_from_ruby_arrays.html"
 27 "http://www.dzone.com/links/rss/creating_graphviz_graphs_from_ruby_arrays.html"
 20 "http://www.reddit.com/r/ruby/"
 12 "http://www.reddit.com/r/ruby/comments/7tw1a/creating_graphviz_graphs_from_ruby_arrays/"
 9 "http://www.graphviz.org/Resources.php"
 8 "http://www.netvibes.com/"
 8 "http://www.google.com/reader/view/#overview-page"
 8 "http://www.google.com/notebook/fullpage"

Especially the "sort | uniq -c | sort -rn | head -n something" is a very frequently recurring pattern, in order to get a list of something in descending order of frequency.

But I'm difficult. I want something visual; a chart. Something like this:


Incidentally, there's a nice Ruby package called SVG::Graph that allows you to generate SVG's from this. The above is a PNG for best compatibility, but here is the SVG version - at least Firefox renders it better than rsvg which I used to generate the PNG.
Assuming you install SVG::Graph from the above page, it's pretty simple to generate the SVG's - just pipe the output from the above command straight into this script:

require 'SVG/Graph/BarHorizontal'

data = [] fields = [] ARGF.each do |line| line = line.chomp.split data << line[0].to_i fields << line[1] end

graph = SVG::Graph::BarHorizontal.new(:height => 20 * data.size, :width => 800, :fields => fields.reverse)

graph.add_data(:data => data.reverse) graph.rotate_y_labels = false graph.scale_integers = true graph.key = false print graph.burn


It tries to be reasonably intelligent about adjusting the height of the graph, but you might want to adjust the width and other parameters. My goal was to get something simple that'll "just work" when I pipe the output of "uniq -c" into it, but any space separated data with a number first and the label afterwards should work as long as the data set is small enough that the graph it produces doesn't become ridiculously large. I'll probably extend it with some command line options, and I want to change the styles used to set colors etc., but the above is enough for 90% of what I want. Note that SVG::Graph also support pie charts, line charts etc.


2009-02-01 02:07 UTC Creating Graphviz graphs from Ruby arrays

As part of my compiler project I wanted a way to visualize the programs, and since the syntax tree (so far at least) is represented with plain Ruby arrays I decided to throw together a script to use Graphviz to generate some graphs. I've written about using Graphviz previously here
The code does not make any assumptions tied to my compiler, but it's NOT an attempt at visualizing arbitrary object structures. You need to pass it an Array object, which can contain other arrays or objects that responds to #to_s.
NOTE: The code makes NO attempt to deal with structures that have loops - you'll run out of stack space soon enough if you try that. Feel free to post your fixes to do that in the comments (easy enough - just need to keep track of visited objects).

An example example. Given this:

 [:defun, :parse_quoted, [:c],
 [:while, [:and, [:ne, [:assign, :c, [:getchar]], -1], [:ne, :c, 34]], [:do,
  [:putchar, :c]
  ]
 ]
 ]


I generate this image (the gradients and shadows are thanks to my previously described XSL transform to pretty up the Graphviz SVG output):



The code is quite straightforward, though I'm not quite happy with the amount of monkey-patching. There were two easy choices: monkey-patching or lots of #is_a? calls, which made it horribly messy. I wouldn't advocate including this into a larger app without cleaning it up first, but as a quick hack it works well.

module ToDot
 def self.escape str
  str.gsub(/([<>{} |\])/) { "\"+$1 }
 end
end

class String def to_dot_label; '\"'+ToDot::escape(self)+'\"'; end end

class Array def to_dot_label; "..."; end

def to_dot_edge src, shorten " #{src}" + (shorten ? "" : ":#{object_id}") + " -> #{object_id};\n" end

def to_dot_subgraph return "" if nil ary = self[0].is_a?(Array) shorten = !ary && self[1..-1].detect{|o| !o.is_a?(Array)} == nil s = " #{object_id} [label=\"" if shorten s += self[0].to_dot_label + "\", shape=rect];\n" else s += collect { |o| "<#{o.object_id}> " + o.to_dot_label }.join("|") s += "\"];\n" end s += collect {|o| o.to_dot_edge(object_id,shorten) }.join s += collect {|o| o.to_dot_subgraph }.join s end end

class Object def to_dot_subgraph; end def to_dot_edge src, shorten; end def to_dot_label; ToDot::escape(to_s); end

def to_dot s = "digraph G {\n" s += " node [shape=record style=filled fillcolor=lightblue " s += "fontname=Verdana height=0.05 fontsize=10.0 ];\n" s += to_a.to_dot_subgraph s += "}\n" end end


As for how to use it:

require 'arytodot'

puts someArray.to_dot

Then pipe the output to "dot -Tsvg >output file" and use your favorite XSL processor to render an image from it. I used "rsvg file.svg file.png". If you want to use my XSL transform to pretty it up, follow the instructions in the article linked to above.

Here's the full "parser" example from my compiler series (click for the full size version):




Older Entries

About me

E-mail: vidar@hokstad.com Skype: vhokstad
Twitter: vhokstad
View my LinkedIn profile.

I was born April 21st, 1975, in Oslo, Norway. Since 2000 I've been living in London, UK. I'm married and we just had our first child, Tristan Ikemefuna Hokstad.

I'm working for Aardvark Media as Director of Technology. I'm also currently on the board of SpatialQ, a startup in the GIS space, and an advisor to Skoach, a startup doing a time management app for people with ADD.

Twitter Updates

    follow me on Twitter