error emails digesting: ar_mailer fork

26Oct09

There’s an ExceptionNotifier plugin wich makes errors caused in controller action sent to you by email. The problem is that the site sometimes can face an issue that makes it error on nearly each request. The emails make it quickly noticed and fixed appropriately, but if the site have some load there will be significant number of these error emails which is annoying (or often harmful) considering that you have a limit on SMTP server.

So in my company we wanted to make error email notifications sent by ExceptionNotifier and the likes not sent immediately but combined into digest email each minute. We used adzap-ar_mailer for error messages buffering and then made it combine them to digests based on To field. Also we already had BackgrounDRB so we didn’t need separate ar_sendmail process.

Adzap’s fork is great because you can use ar_mailer for just one class with it – just what we needed. So we decided to opensource our tweaks in hope that it will be helpful for someone, and put our changes to a fork on GitHub: artemv/ar_mailer. The code is quite limited but it’s fine for our needs, and if you need more you have smth to start with. E.g. it could be improved to support text/html messages or to make certain classes bypass digesting when sending emails with ARMailer.

So, what does this fork do out of the box:

• ARMailer messages don’t get send until they are combined into digests
• all messages to same addressee are combined into one digest email
• digesting only supports text/plain messages
• if the digest mail is too big it gets truncated, and full digest is dumped to a file on server; truncated digest will tell about it
• the sending itself can be triggered by method call, so now you have options to use ar_sendmail process or smth else, e.g. BackgrounDRB job
• you can specify :smtp_settings to be used when sending digests: we needed it to be other than default ActionMailer::Base.smtp_settings

Usage example:

ActionMailer::ARSendmail.digest_error_emails(
    :dump_path => '/var/log/myapp',
    :subj_prefix => ExceptionNotifier.email_prefix,
    :smtp_settings =>   {:address =>     "localhost",
        :port =>        25,
        :domain =>      'my_domain.com',
        :tls =>          false}
)

The fork is contributed by Your Net Works, Inc.

Links:

• Exception notifier plugin: http://agilewebdevelopment.com/plugins/exception_notifier
• Original ar_mailer gem: http://seattlerb.rubyforge.org/ar_mailer
• Your Net Works’ fork of ar_mailer: http://github.com/artemv/ar_mailer

Filed under: ActionMailer, emailing, rails, ruby   |  5 Comments

Rails template caching in 2.2.2

31Jul09

Turns out view templates are cached for test environment in Rails 2.2.2, and you can’t turn it off. This is unfortunate for me because I’m used to rspeccing with DRB server  – so now I need to restart it every time I change the view I’m working on. Bummer.

Good news is that in Rails 2.3 you can turn it off with

config.action_view.cache_template_loading = false

Filed under: ActionView, rails   |  Leave a Comment

another attachment_fu hack: partitioning strategy customization

07Oct08

In attachment_fu you can have attachment file pathes partitioned: uploaded file.jpg goes to files/0000/0167 directory, where 00000167 is internal ID of the file. With this fork you have some control over it:

Now you can specify partitioning strategy as lambda in has_attachment
statement, like this:

    class Attachment < ActiveRecord::Base
      has_attachment :storage => :file_system, :path_prefix => 'files/issues',
        :partition => lambda {|a| (a.thumbnail ? a.parent : a).document_id}

      belongs_to :document, :polymorphic => true
      ...

Attachment model object is passed to lambda. In this example we have Issue
model having many Attachments via polymorphic association, and we want
attachment files pathes to be like ‘files/issues/67/my_file.doc’, where 67 is
attachment’s issue id. The lambda can also return array of strings, then
['a', 'b'] will become path ‘files/issues/a/b’ in this case. This lambda will fit
if you use thumbnails; if you don’t it will be just lambda {|a| a.document_id}.

Note that partitioning is relevant only to :file_system storage.

Filed under: attachment_fu, rails   |  5 Comments

attachment_fu fork with validation messages customization

03Oct08

I’ve just made a fork of attachment_fu plugin: http://github.com/artemv/attachment_fu/tree/master.

This fork provides validation errors customization means.
Now you can pass custom error formatter method names to validates_as_attachment:


validates_as_attachment :error_formatters => {:content_type => :add_content_type_error, :size => :add_size_error}


Here’s an example of error formatter. It uses Globalize to translate the message template:

    def add_content_type_error(options)
      return if !filename #content type doesn't matter if there's no file
      msg = sprintf(("Your file's format (%s) is denied. Allowed are only " +
          "image formats.").translate, options[:value])

      #don't try to translate string expanded with parameters
      msg.translated = true

      #for it to not mix with attribute name (and break .translated state)
      errors.add_to_base(msg)
    end

Also it applies fixes against ‘Size is not included in the list error’ issue
(which is often reproduced on Windows) suggested at
http://railsforum.com/viewtopic.php?pid=26427#p26427.

Short attachment_fu tutorial: http://clarkware.com/cgi/blosxom/2007/02/24#FileUploadFu

Filed under: attachment_fu, rails, ruby   |  Leave a Comment

Formatting diff files as HTML

24May08

I wanted to add diff viewing function to my codenotifier tool and all I found in Ruby was this CGI/command line script. I adopted it to show line numbers, handle multifile diffs and make output similar to one of Trac, and integrated it to codenotifier. Plus, in case anybody else needs it, I created diff_to_html.rb opensource project at GitHub (liked it a lot btw!), which makes it available as a gem, like this:

gem install artemv-diff_to_html --source=http://gems.github.com

Enjoy!

The code is far from being perfect, actually I wonder why it works in some cases, plus it could output better (e.g. I like changeset viewer at GitHub – it puts the changes and line numbers in separate DIVs – this way you could simply select changes without line numbers.. at price of (sometimes) horizontal scrollbar I guess. The funny thing is that at GitHub result is still mixed with comment counters for each line – it was so close to being perfect.. as always :)

So, patches/contributions are very welcome.

Related links:

• blog entry on Codenotifier’s diffs
• sample diff at Codenotifier, from Shoes project
• original code by Adam Doppelt (thanks man!) appears in his blog entry (blogs became an important source of code, huh?)
• diff_to_html.rb project at GitHub

Filed under: codenotifier, diff_to_html.rb, my projects, opensource, ruby   |  2 Comments

reusing templates in ActionMailer

22Apr08

Well I’m writing quite a mail-intensive small application right now, and wanted to use the same template for 2 mail actions at some point. Searching for how to use template different from mailer action name didn’t give results, but looking into the ActionMailer code did (hail the open source!), so I’m placing it here to be a bit more visible )

Well the solution is quite laconic: just set self.template to what you need, just like you would render :action => :smth_different in ActionView. In example below enabled_notification action does it:

class GroupsMailer < ActionMailer::Base
 #...
 def disabled_notification(group, actor, disabled = true)
    verb = disabled ? 'disabled' : 'enabled'
    subject 'Usergroup \'%s\' was %s' % [group.name, verb]

    body :group => group, :actor => actor,
        :group_url => group_url(group.dn), :verb => verb

    common_headers(actor)
  end

  def enabled_notification(group, actor)
    disabled_notification(group, actor, false)
    self.template = :disabled_notification
  end
end

This was with actionmailer-2.0.1.

Filed under: ActionMailer, rails, tips   |  3 Comments

routes.rb don’t require restart

01Feb08

Just noticed that changes to routes.rb doesn’t require Rails application restart anymore (I’m on Rails 2.0.1). Cool stuff!

So now we can introduce another named route like this:

map.cloud 'projects/cloud', :controller => 'projects', :action => 'cloud'

and then immediately use cloud_url in templates.

Why I wanted to use named route for such a simple case? Actually not because it gives nice ‘cloud_url’ shortcut. I currently use it in a single place so it dosn’t matter to me. The reason is that my ‘projects’ controller is REST-friendly, i.e. I have ‘map.resources :projects’ in routes.rb. So, I have to add some line to routes.rb to use /projects/cloud – otherwise Rails thinks ‘cloud’ is an Id of project I’m trying to open. So I have 2 options, named route and map.connect, and now the choise is obvious.

Filed under: rails, tips   |  2 Comments

acts_as_ferret pagination

17Jan08

Ok, Ferret is great, but how to paginate?

I saw several solutions on the net and liked this one the most.

It appears that it’s quite easy to integrate acts_as_ferret with will_paginate, the officially recommended pagination plugin for Rails.

First, a patch is needed. I placed it to a new file in my special place for patches, vendor/plugins/patches/lib; it could also go in some of existing application files, e.g. bottom of environment.rb:

module ActsAsFerret
  module ClassMethods
    alias :find_all_by_contents :find_by_contents
  end
end

Then, in controller:

@users = User.paginate_by_contents(@search.query,
  :total_entries => User.total_hits(@search.query), :page => params[:page],
  :per_page => 10)

Change User to your model class here. You can also use sorting here, see Tutorial below for guidelines.

The trick is that will_paginate is friendly to model class’s methods like find_all_by_xxx; acts_as_ferret’s find_by_contents’s name is a bit wrong for it, so we make an alias, and later this method becomes our model’s method, when we put acts_as_ferret statement in the model class definition.

References:

• Acts_As_Ferret Tutorial
• This solution was taken from will_paginate’s author blog, thanks ocher!

Filed under: acts_as_ferret, rails, tips, will_paginate   |  5 Comments

Adventures with Ferret and unicode strings

16Jan08

Ferret is a Lucene port to Ruby. Looks like a very good port. What I like:

• it handles wildcards from both sides of a string (Lucene began to allow prefixing with * not long ago, may be with v2.0),
• it has good highlight facility out of the box and that it was very easy to plug into Rails application together with (also great) acts_as_ferret plugin: index dir configuration, population and updating – all appeared to be automatic, much simpler than using Lucene in Java
• acts_as_ferret makes it simple to use index in cluster environment with ferret_server, just start it and it works. Well, almost: I had to patch vendor/plugins/acts_as_ferret/lib/server_manager.rb:

@@ -35,8 +35,8 @@
begin
ENV['FERRET_USE_LOCAL_INDEX'] = 'true'
ENV['RAILS_ENV'] = $ferret_server_options['environment']
-  #require(File.join(File.dirname(__FILE__), '../../../../config/environment'))
-  require(File.join(File.dirname(ENV['_']), '../config/environment'))
+  require(File.join(File.dirname(__FILE__), '../../../../config/environment'))
+  #require(File.join(File.dirname(ENV['_']), '../config/environment'))
require 'acts_as_ferret'
ActsAsFerret::Remote::Server.new.send($ferret_server_action)
rescue Exception => e

What I didn’t like is that after I thought search was working it appeared that it’s broken for words with non-latinic symbols. Both on Windows platform where I develop and on Linux where I deploy to. For example, if text contains ‘Antônio’ and I search by ‘Antônio’ I don’t find anything; also if I search for ‘ant’ I get something like ‘Ant??nio’ in results (if highlight is used).More strict example:

require 'rubygems'
require 'ferret'

text = "Antônio"
include Ferret::Analysis
tokenizer = StandardAnalyzer.new.token_stream(:field, text)
while token = tokenizer.next
  puts token
end

The output was:

token["antÃ":0:4:1]
token["nio":5:8:1]

I have some experience with Lucene and know that it handles unicode without problems; Rails is also ok with unicode; the whole situation looked stupid (Ruby being not very unicode-friendly also looks rather stupid but I guess Japanese had some reasons for this – and I hope 1.9 will be much better) so I decided to dig this problem. Surprisingly I didn’t find a lot of information on this, it seems that people don’t really have it. Ok for those unlucky like me here’s the solution I found.Basically Ferret handles Unicode well out of the box if operating system is not Windows and default locale is Unicode-friendly. Both my environments don’t satisfy these requirements so I got into not-very-documented troubles. To resolve them we have to say Ferret some magic words:

require 'rubygems'
require 'ferret'

if PLATFORM =~ /win32/
  #strange hack for Windows from Ferret's author to make it unicode-friendly.
  #http://ferret.davebalmain.com/trac/ticket/326#comment:3
  Ferret.locale = ''
else
  #tell Ferret to be unicode-friendly. This unfortunately doesn't work on Windows.
  Ferret.locale = "en_US.UTF-8"
end
puts "failed to set locale" if Ferret.locale.nil?

text = "Antônio"
include Ferret::Analysis
tokenizer = StandardAnalyzer.new.token_stream(:field, text)
while token = tokenizer.next
  puts token
end

Output on Linux:

token["antônio":0:8:1]

Output on Windows:

token["antгґnio":0:8:1]

Well on Windows that wasn’t that good but at least it’s one word so search by ‘Antônio’ works.. and in Rails highlight doesn’t corrupt symbols now.That’s it!

Versions I used for this article:

• Ferret 0.11.5
• acts_as_ferret 0.4.3 (wow, both start with zeroes)
• rails 2.0.1
• ruby 1.8.6

Update:

Well this solution doesn’t exactly work on Windows. The test is broken (gives 2 tokens instead of 1) when I set Windows locale (‘Standarts and formats’) to English; but when it’s Russian it works fine. Ok, at least that was a stable solution for Linux.

Update:

ferret_server doesn’t work for Windows, so I cannot use it in my development environment :(

Ruby world pushes me more and more out of Windows :)

Filed under: ferret, i18n, rails, ruby   |  5 Comments

Markaby you’re too beautiful

26Dec07

..to resist. The more I write in Rails the more I find myself converting rhtmls to Markaby, another invention of _why’s genius.

table do
  for project in @projects
    tr.project :id => "#{project.name}" do
      render :partial => 'project', :locals => {:project => project }
    end
  end
end
br
link_to 'New project', new_project_url

The fact that template become a ruby script with all the syntax support from IDE is very powerful also. With conventional IDE I see the scopes of all those container elements like divs. This is also true for rthml, but not so for HAML which stops me from trying it – though it seems to be much more popular rendering engine these days.

The thing that stopped me for a while from using Markaby in all my templates is that forms were usually broken when rendered completely with it. What helped is to write ‘form_for’ in some wrapper rhtml and the form body in partial.mab.Now I’ve learned how to write the form wrapper in Markaby too: this ticket has all the answers I need. It’s a pity the ticket is not resolved: it says it’s a year old!

Filed under: markaby, rails, templating   |  2 Comments