A Curious Use of For Loops In Ruby

I was thinking about the use of for loops in Ruby the other day, triggered by one of the Ruby problems over at rubeque.

Now, a for-loop in Ruby usually looks something like this:

for i in collection
  # i takes each value that's in the collection

Interesting things happen when the loop variable (i in the case above) is assigned to a value already before the loop. In particular, interesting things can happen if it’s not even a variable.

Let’s see what one can do with a for-loop in Ruby (or rather to a for-loop). Enter pry (or your favourite Ruby-REPL [Read-Evaluate-Print-Loop]):

pry(main)> for Object.new in [1,2,3]; end
NoMethodError: undefined method `new=’ for Object:Class
from (pry):3:in `block in __pry__’

Oh, that’s interesting: The interpreter does not complain about Object.new not being a variable, but rather that there’s not method ‘new=’ for class Object.

OK. let’s play with this… Let’s assume a class Foo:

class Foo
  def initialize foo
    @foo = foo

  def foo= other
    @foo.unshift other

Note that there’s no method named foo for class Foo, just ‘foo=’. Initialising a variable now with, say, an empty Array, there’s a curious way to reverse an Array:

pry(main)> f = Foo.new(res)
=> #<foo:0x007fd392c02368 @foo=[]>
pry(main)> for f.foo in [:first, 2, 'third']
pry(main)* end
=> [:first, 2, "third"]
pry(main)> pp res
["third", 2, :first]
=> ["third", 2, :first]

There is a way to use an empty loop to reverse an Array. (Note. this does not solve the problem given at rubeque, since a class definition is not allowed inside a method definition). In fact there are more ways than this!

A few new things I learned about for loops in Ruby:

  1. They do not require a variable name as the ‘loop variable’
  2. If one uses what looks like a method call (the ‘f.foo’ in the example above) as the  ‘loop variable’ then…
    1. …the object f refers to, doesn’t even need to respond to that method, but
    2. …instead it needs to respond to ‘method_name=’ (‘foo=’ in the example above).

I don’t know what to think about this, but you probably shouldn’t rely on this behaviour or use it in production code.


More Fun With Rakefiles

After Fun With Rakefiles, here’s another ‘behaviour’ I found in a Rakefile.
Some example code is available on GitHub: https://github.com/s2k/rake-loop.

The Setting

The project team worked on a larger number of modules, each in its own sub directory and a global rake task spec:all to run, well, all the specs for all the modules. Since they could run independently, we used the parallel gem to run RSpec in several processes, so our continuous integration system (or local machine) could use the available CPUs (as opposed to running in Ruby threads, which doesn’t utilise all processors [well, at the time of this writing]).

Also, while most of the RSpec code used the (now) old-style should-notation, some new specs were written using the new expect notation. For more on that topic see e.g. RSpec’s New Expectation Syntax by Myron Marston and the GitHub repo for RSpec expectations.

Trying To Fix A Hack

When working with the Rakefile and RSpec files I noticed something odd: The RSpec files using expect weren’t executed. The reason was some code in the Rakefile which (essentially) did this:

 run(rspec_filename) if File.read(rspec_filename).contains? 'should'

That explains why the newer RSpec code was not executed: It didn’t contain any shoulds. But what was going on?

Right after removing that code and re-running bundle exec rake spec:all, I found out that the command a) didn’t finish in any reasonable amount of time and b) over time there were more and more Ruby processes showing up in the system monitor. What was happening?

Why excluding RSpec files not containing ‘should’ helped

As far as the symptom was concerned, excluding RSpec files that didn’t contain the word ‘should’ helped avoiding the behaviour I observed. In any case, I wanted to also execute the other RSpec files, but I would not add to the hack by executing file which contained either ‘should’ or ‘expect’…

A Rake Feature

When rake is called from the command line in a directory that does not contain a Rakefile, then rake looks for a Rakefile in the parent directory. That’s a feature, since it allows you to have one description of a task in some directory, and use that task in all sub directories. (In our case however, something was going horribly wrong.)

Where It went wrong

I then figured out that the modules where the RSpec files didn’t contain the word ‘should’ also didn’t contain a Rakefile. So, when invoking rake in that case, it would move upwards the folder structure finally get to the project global Rakefile and call the task defined there.

Alas, the ‘global’ Rakefile not only contained a task definition for spec:all, but also a task called spec—which only invoked spec:all in a new shell:

namespace :spec do
  desc 'runs all rspec suites'
  task :all => :clean do
    # do stuff, in particular:
    sh "cd #{spec_dir}/../; bundle exec rake spec > #{report_folder}/#{module_name}.tmp 2>&1"

 desc 'delete old report files'
 task :clean do
   FileUtils.rm_rf report_folder

task :spec do
  sh 'bundle exec rake spec:all'

Notice that the global Rakefile expects a Rakefile containing a definition for the task ‘spec’ to be defined somewhere (see line 5 in the code snippet above), probably in the sub directory where rake is called.

Alas, if there is no Rakefile, rake will step up the directory structure, eventually find the global Rakefile call the task ‘spec’… and restart running ‘spec:all’ again, only to run into the exact same situation again and therefore start a loop.

Lessons Learned

  • Using the parallel gem made a bad situation worse, but it also helped finding the real problem by maximising the effect.
  • It’s probably a good idea to call rake tasks from within a rake task using the methods Rake itself provides (invoke and execute). There’s a discussion on stack overflow named How to run Rake tasks from within Rake tasks?
  • If something goes wrong, fix the underlying issue, rather than working around the problem. It’ll save later generations a lot of work.
  •  Generally speaking, it’s a good idea not to let specialised rake tasks (like running RSpec in some sub directory) end up invoking a more global rake task. That’s especially true if that more global task is going to invoke the more specialised one in turn.

Ruby Details: clone and dup

Another post that’s mostly a reminder to self. 😉

Over at Know Ruby: clone and dup Aaron Lasseigne explained the differences between dup and clone.

However there’s another difference, and that’s the way the methods reproduce the methods an object can respond to. As is so often the case, a short session in a Ruby REPL (like irb or pry) helps to see things in action:

[1] pry(main)> o = Object.new
=> #<Object:0x00000102d298a0>
[2] pry(main)> def o.meth
[2] pry(main)* puts "meth called on object #{self.inspect}"
[2] pry(main)* end
=> :meth
[3] pry(main)> o.meth
meth called on object #<Object:0x00000102d298a0>
=> nil
[4] pry(main)> o_dupped = o.dup
=> #<Object:0x00000102101eb0>
[5] pry(main)> o_dupped.meth
NoMethodError: undefined method `meth' for #
from (pry):7:in `__pry__'
[6] pry(main)> o_cloned = o.clone
=> #<Object:0x00000102138168>
[7] pry(main)> o_cloned.meth
meth called on object #<Object:0x00000102138168>
=> nil

Note that both, clone and dup, return a new object, but only the one returned by clone also has the method defined for object o.
In other words: dup does not replicate an objects singleton methods.

Fun With Rakefiles

I recently ran into an ‘interesting’ issue in a set of  Rakefiles.

Here’s a part of the global Rakefile, which imports the *.rake files from a tasks folder:

$LOAD_PATH.unshift File.join(File.dirname(__FILE__), 'lib')

Dir.glob('tasks/*.rake').each { |r| import r }

Some of these Rakefiles require other Ruby files (which is stored in the lib folder). Since executing one task involves loading all that code, there’s a chance that an error in one supporting Ruby file may cause an error in an entirely unrelated task.

Let’s look at an example, the code is available over at GitHub: https://github.com/s2k/fun_with_rakefiles

For simplicity there are just two rake files within tasks and a ruby file in lib. What we like to run is this:

rake make_it_so

However what we get is an error message pointing to some rather unrelated part of the code:

$rake make_it_so
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - partlist.txt
…dev/fun_with_rakefiles/lib/thing.rb:7:in `readlines'
…dev/fun_with_rakefiles/lib/thing.rb:7:in `<class:Thing>'
…dev/fun_with_rakefiles/lib/thing.rb:1:in `<top (required)>'
…dev/fun_with_rakefiles/tasks/unrelated_thing_tasks.rake:1:in `require'
…dev/fun_with_rakefiles/tasks/unrelated_thing_tasks.rake:1:in `<top (required)>'
…/.rvm/gems/ruby-2.1.1/bin/ruby_executable_hooks:15:in `eval'
…/.rvm/gems/ruby-2.1.1/bin/ruby_executable_hooks:15:in `'
(See full trace by running task with --trace)

Hmmm… When I first saw the error message I really wondered why suddenly there’s a file partlist.txt missing and then I asked myself where that Thing class was used in the code.

In this example it’s relatively easy to see what’s going on: The ‘central’ Rakefile gathers all the other .rake files that may be necessary (which, in turn, require all the files they need). However, if there are a few (sub) rake files around things can get a bit more confusing.

In my case, it was simple enough to repair the issue, since the missing file could be added to source control and everything worked again. But still there are a few things to think about:

  1. Should the be code in a Ruby class definition that’s executed while defining the class? Especially: Should it be there, in case you can avoid it?
    In my opinion it’s a good idea to only have code like that if it’s really necessary. In the example given, one could load the file when an instance of the class is created. Doing so will reduce the consequences of the missing file drastically: Only the rake tasks actually using that class would fail, not every rake task.
  2. Could one avoid the failure and still be able to run the desired task?
    It turns out that you can:

    $rake -f tasks/what_we_want_to_do.rake  make_it_so
    Processing something…

    In some cases you may also need to supply this command line parameter, in order to allow rake to find other Ruby files:

    -I, --libdir LIBDIR      Include LIBDIR in the search path for required modules.
  3. Can we check that nothing’s broken?
    In fact, we can:

    rake -T

    If that actually lists the available rake tasks and doesn’t exit with an error code, it looks OK, at least it didn’t break everything. However, if a file is missing form the version control system, it’s a works-on-my-machine situation.

In a project with a large number of rake files and supporting library code, it may be worth while to have rake -T as part of a smoke test on the continuous integration system.



osx-gcc-installer or XCode

  1. How to remove Xcode completely from your system:
    sudo /Developer/Library/uninstall-devtools --mode=all

    Use at your own risk!

  2. Where to get Xcode: The App store: https://developer.apple.com/xcode/
  3. OSX-gcc-intstaller: https://github.com/kennethreitz/osx-gcc-installer
Addtional info: Another way is to get the “Command Line Tools for Xcode” at https://developer.apple.com/downloads/ (which requires an Apple developer account).
Happy hacking!

NoMethodError in Rails Tests — Fun With Fixtures

In case there’s this weird error message when running unit tests for a Rails app, chances are that your fixtures need some attention. Especially if the schema changed…

NoMethodError: undefined method ‘name’ for #

method method_missing in test_process.rb at line 511
method method_missing in test_case.rb at line 158
method rescue in run in setup_and_teardown.rb at line 26
method run in setup_and_teardown.rb at line 33
method block (2 levels) in run_test_suites in unit.rb at line 641
method each in unit.rb at line 635
method block in run_test_suites in unit.rb at line 635
method each in unit.rb at line 634
method run_test_suites in unit.rb at line 634
method run in unit.rb at line 594
method block in autorun in unit.rb at line 492

There’s no test method given, because, well the tests don’t even get that far: It’s likely that there’s a key (column name) given in the fixture, which is not in the DB schema (anymore).

TextMate 1.5.10 … and Ruby 1.9.2

There’s a new version of Textmate available. Cool, thanks! However after installing … I couldn’t run Rake tasks anymore (the keyboard short cut to remember: Shift-Ctrl-R).

In ‘rake_mate.rb’ (line 49, /Applications/TextMate.app/Contents/SharedSupport/Bundles/Ruby.tmbundle/Support/RakeMate/rake_mate.rb). I inserted “.lines” to make it look like this:

tasks = [DEFAULT_TASK] + tasks.lines.grep(/^rake\s+(\S+)/) { |t| t.split[1] }

Additionally I had to ‘re-copy’ the plist.bundle as described on the rvm site.
Now everything works fine again.

Addendum: As mentioned in the rvm guide to using TextMate, I had to (re) move TextMate’s own Builder.rb out of the way:

cd /Applications/TextMate.app/Contents/SharedSupport/Support/lib/ ; mv Builder.rb Builder.rb.backup

About 40 Years Ago And Metrics

The term “Software Engineering” is coming to age: It was coined slightly more than 40 years ago in Garmisch-Partenkirchen, Germany. Tom DeMarco wrote a very interesting article about his opinion about the topic, which is available online at the IEEE Software magazine: “Software Engineering: An Idea Whose Time Has Come and Gone?” Let me cite just this part:

This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value.

I strongly recommend to read the whole article.

Remove files or directories from a git repository

Ever needed to remove a file or directory from a git repository? I had to: After pushing a change that included a rather large file to a remote repository, the repository couldn’t be cloned anymore (due to a memory limitation). The support is working on the issue, but being able to clone the repository seems more important to me than having the slides for a presentation (of the past) in this particular repository.

So here’s how: David Underhill has already done – and described – it: “Git: Forever Remove Files Or Folders From History”

Worked for me as well.

Sinatra and reloading

Since its 0.9.2 release on 18th of May 2009, Sinatra doesn’t automatically reload files anymore – not even in development mode.

To achieve this use ‘shotgun’

sudo gem install shotgun

and then start the server using:

shotgun <your_applicationname>.rb

The effect is essentially the same (apart from avoiding the issues the ‘traditional’ way of reloading apparently had – which is why it was removed). 

Thanks for pointing this out in this thread @ rubyforen.de.

Euruko 2009 – back from Barcelona

These two days with Ruby enthusiasts form all over the world not just Europe was great. Nicely located at citilab, with very good public transportation to the city centre of Barcelona, there was enough space inside as well as out side the building to talk to the other attendees, just relax in the sun or work with the notebook. Two wi-fi networks inside and outside the building could deal well with all the notebooks and mobile phones connected to it. This worked very good indeed.

The presentations I liked most where the ones introducing really exciting and new stuff. First and foremost there’s Rhodes, which enables the creation of native mobile applications with Ruby and HTML. And of course Matz’ keynote was very entertaining as well. More really new stuff was about Adhearsion a way to build voice enabled application with Ruby (and Rails). Then some of the presentations were a really good show. Most notably Javier’s talk about gosu, a framework for games and Pablo’s presentation about Archaeopteryx which was probably the loudest presentation I’ve ever heard. 🙂

The only thing that was a tad bit disturbing for me was that I didn’t have one of the mobile microphones available and was kind of stuck behind the table as well as the phones themselves, as they seemed to go silent now and then. Aslak dealt with this best I think, repeatedly saying “Hello?” directly into the microphone whenever it went silent.

I think this year was the first time Twitter was heavily used during Euruko, reusing a bot made for Scotland on Rails. and this helped enhancing the communication and connecting people even more than in previous years. A special “Thank you” to all for the other attendees who gave me direct feedback about the presentation as well as the corresponding discussion about it, including but not limited to: Andrew Miller, Tim Becker, Mike Just, Dave Frey. Thanks a lot, indeed.

More summaries are available from Javier and Tomasz

Next year Euruko will take place in Krakow, Poland and I’m already looking forward to going there.

Pasting Code Is OK

While reading about the DRY principle (for “Don’t repeat yourself”) and the evil of copy-and-paste coding (again), I started thinking what to do instead. Actually, what to do is more or less obvious: Put the code into a place where its accessible to be reused — a method, may be in a new or existing module or class.

So, whenever I feel the ‘need’ to copy code, I now think about cutting it, creating a new method and calling that. Apart form avoiding duplication, the code is now testable immediately by calling the method (instead of getting the surrounding code exercised). And the methods using the code gets shorter.

In the end, it boils down to pasting code being perfectly OK, it’s the copying that causes the trouble.

Things We Do, But Shouldn’t

I regularly fire up the same applications: TextMate, a browser or two, mail programs, iTunes, a shell, irb and others. Now, while Quicksilver is excellent for firing up applications (and a lot of other things), I’d still be busy typing and/or mouse-pointing and clicking. And doing that is boring, cumbersome and not what I like to do anyway. I shouldn’t (have to) do it. And, in fact, I don’t. A little bit of Ruby code will do it:

%w( <full_application_paths_go_here> ).each{ | app | system "open #{app} &" }

Replace <full_application_paths_go_here> with a list of space-delimited full application paths, save it into a file (in ~/bin presumably), make it executable and there you go. Actually, the language doesn’t matter at all here. The only thing that does matter is to fire up the applications.

Starting all most used applications is now just a few key strokes away. There’s certainly a very similar way to do this on other *nix OSes and Windows.

Berlin – Day 3 / RailsConf Europe part 1 – Tutorials

The morning started with me finding out that breakfast at my hotel is served from 8 to 11 — while I had to leave before 8. Anyway there’s so many places in Berlin to get a great breakfast. So I had it on the way to the conference.

The 1st tutorial I atended was Trotter Cashion’s “Refactoring”. that was a good introduction as well as a few great tips for tools to help refactoring:

  • heckle: ‘Bebugs’ your code automatically and will tell you what you didn’t cover in you test cases in quite a few cases. It dynamically changes you code by turning ifs to unlesses, ‘==’ to ‘!=’ etc. You get the idea. It then checks whether you unit/tests are failing, i.e. wether they would allow you to find this particular fault. Which they likely won’t in so many cases.
  • grep: Well, it’s grep.
  • rcov: Another tool to tell you about test coverage. This one generates HMTL reports which show which code is and isn’t executed when running your test cases. Very useful a lot faster than heckle, at the price of being less rigousus.
  • glark: A new way for grepping.

After than lunch time revealed that this time lunch is a lot better than at last year’s RailsConf in London. In fact is was excellent. Fresh fish, salads, lettuce, desserts. Options – more than one indeed – for vegetarians.
A minor issue is, just like in London last year, the wireless network: With about 300 attendees around, the network is really really slow.

Now, I’m attending the GIS tutorial (while typing this post). Anyway the accompanying script seems to be a good starting point.

And my RejectConf talk still isn’t done…