On 3.2
January 30, 2012 § 2 Comments
Last Rails 3.2 post. I swear.
I mentioned ActiveRecord::Relation#pluck in a previous post because it was one I was particularly excited about. After reading through the rest of the release notes I wanted to highlight a few of the other smaller changes I thought were interesting.
ActiveRecord
ActiveRecord::Relation#uniq
Client.select('DISTINCT name')
is now written
Client.select(:name).uniq
You can also revert the uniqueness with
Client.select(:name).uniq.uniq(false)
Don’t user LOWER for case insensitive searches on MySQL
When querying against our databases there are a bunch of times we don’t care about case. In the case of searching against an email address, a case insensitive search does exactly what we want it to. When you add the :case_sensitive => true to a validates_uniqueness_of it use to wrap that part of the query in a LOWER(). This sucked because it prevents MySQL from using an index. When you put this restriction on your email column and have to do this query every time you save a record. Since MySQL does case insensitive searches, ActiveRecord doesn’t worry about adding the LOWER() around the columns. Time saved all around!
rake db:drop drops both dev and test
rake db:create creates both the development and test databases. rake db:drop now drops the both development and test databases instead of just the development database.
ActiveView
namespace form_for
When you have multiple forms on one page for the same type of objects, you can get conflicting ids form elements. Problem solved with form namespaces.
<%= form_for(@offer, :namespace => 'namespace') do |f| %> <%= f.label :version, 'Version' %>: <%= f.text_field :version %> <% end %>
The ids of the form elements have the namespace prepended with and underscore to the front of the original id.
ActiveSupport
Time Ranges
One of the things that I love most about Rails is ActiveSupport. And one of the things I love most about ActiveSupport is how it makes dealing with Time awesome and easy. They have added a few new convenience functions to help with date ranges:
Time#all_dayTime#all_weekTime#all_monthTime#all_quarterTime#all_year
You can use these with ActiveRecord queries like so:
Event.where(:created_at => Time.now.all_week) Event.where(:created_at => Time.now.all_day)
Obviously there are a lot of other additions in 3.2, these are just the few that I know that I am going to have a use for. Enjoy.
On Pluck
January 23, 2012 § Leave a comment
Rails 3.2 has been released and there are a few things in it that I am excited about. I mentioned in a previous post that a few of the things coming in 3.2 were things that I had thought about doing or had already done myself. Turns out I missed one.
ActiveRecord::Relation#pluck
This is the exact same functionality provided by my select-column gem. It allows you to get an array of column values from the database without having to instantiate an ActiveRecord object for each one. So instead of having to do this:
Post.where(:published => true).collect(&:id)
you can simply do:
Post.where(:published => true).pluck(:id)
The place I see this having a lot of nice applications is in caching. I know that I wrote my gem so that I could use it to speed up some methods that would find appropriate records and then cache the ids for quicker look ups later. I am really excited to do a find and replace of collect for pluck in some of my projects.
On Observer Registration
January 9, 2012 § Leave a comment
If you are using the observer pattern with your ActiveRecord models, you have to register the observers. I don’t see why this can’t be done automatically. I place all of my observers in app/observers so I can easily register all of them with a few line in my application.rb file.
# Register all the observers in the observers folder. observers = [] pattern = File.join(Rails.application.config.root, 'app', 'observers', '**', '*.rb') Dir.glob(pattern).each do |filename| observers << File.basename(filename, '.rb').to_sym end config.active_record.observers = observers
More on ActiveRecord Observers and an example application.rb.
On My Projects Becoming Obsolete
December 27, 2011 § 1 Comment
Rails 3.2 RC1 was released last week and it announces some pretty cool new things. One of those things is something that I have played with building out myself (tagged logging) and another is something I actually did (ActiveRecord Store). When I wrote my data-attributes gem, I was inspired to do so because I had a users table that had an ever-growing number of email permission checks. I never queried against these, and they made doing a SELCET * FROM users; really painful to try and read. I decided to just throw them all in a serialized column and be done with it.
When I first read the release notes for 3.2 I had two thoughts go through my head.
1) Well, crap. data-attributes is now dead and useless. (yes, I use <code> tags in my head)
2) Awesome, the solution I came up with for a problem I was having is the same one that the Rails team came up with. Maybe I might know a thing or two about what I am doing.
But as I started to compare the new ActiveRecord::Store with data-attributes, I began to realize that my little project isn’t dead quite yet. The accessors generated by the store call with AR::S aren’t full attribute accessors. By this I mean they don’t go through read_attribute and write_attribute before committing the new data to the serialized hash. This prevents you from intercepting the accessor call and doing some pre/post processing. You also don’t get default values as with other attribute accessors. Definitely not a huge deal in the least, but something to be aware of. AR::S does mark things as dirty though. data-attributes does not do this as of yet.
As for the future for of data-attributes, I think that it is actually dead in the long run. If I were to work on this problem more in the future, I’d probably do so by adding the AR::S instead of continuing working on data-attributes.
On New Milestones
November 7, 2011 § Leave a comment
Earlier this year I got the itch to start doing some open source stuff. I hadn’t done anything of the sort before, so I didn’t really have any idea where to begin. There have definitely been some bumps along the way, but I think that I have made some progress. Looking back on my progress I identified a few milestones I hit along the way. They weren’t obvious at the time, and I certainly can’t say that I meant to achieve each one, but I can clearly see them now. Each one helped me a little more on my way to being an open source contributor. Each one also elicited different thoughts and feelings. Now in no particular order and mostly for me to remember what I was thinking…
First Repository Uploaded to GitHub
I signed up for GitHub back at the end of 2008. I forked a few repos here and there just because there was a button to do it. I didn’t really know anything about git and didn’t do anything with repos I had forked. I didn’t actually upload a repo of my own until the end of March 2011. I had been working with subdomains in Rails 3 and experimenting with the new cookie jar chaining. I found a use case for adding another cookie jar that assigned the domain of the cookie. I abstracted it out, gave a poor project name and put it up on GitHub. [tld-cookies]
It was a bit nerve-racking putting something you have made out there for whole world to see. I am sure artists feel the same way when they show their paintings or writing to the general public. I had no expectations that people would look at it or care about it, but still it was out there for people to scoff at if they saw fit to. Nervous as I was, I felt something that I wasn’t expecting… liberation. Even if no one ever used my code, I was now a contributing member of the open source community. It felt great.
Two more repos went up almost immediately.
First Gem Published
When I pushed my first repos, the code was more or less in it’s finalized state, so the gem(s) followed quickly behind. I remember having my RubyGems.org dashboard up all day watching the those first downloads trickle in. I quickly realized that the first dozen or so were mirrors downloading all the new and updated gems. Not going to lie, was a little sad when I realized that. That first gem still sits at 50 downloads, probably because of it’s bad name, but some of the other ones have a few hundred downloads.
It’s awesome. Some of my peers, not my coworkers, are using my code. Building things that people use is awesome, whether it’s a website or a library. As a software developer, I get a thrill when websites I work on get traction and usage. Building successful businesses is my end goal, but with that being said, I get a very different and more personal thrill when I see some of my code being used by other developers. It is a type of validation of your technical skills and it feels great.
First Gist Uploaded
We are a rails shop at work. One of the things that I was getting annoyed with when dealing with debug statements was where to print these statements out to. Do I print them to the log file or to standard out? The log file is the obvious choice, but if you are working in the console, then using puts so you don’t have to switch terminal windows might be preferable. I spent some time and came up with a useful little function. I realized that this might be useful to others so I put it up on GitHub as a gist, and called it a day. Looking back, this was really the first time I just casually threw something out there that I thought might be interesting, but wasn’t a fully functioning library.
First Issue Filed Against Me
Then came the bugs… I had my first GitHub issue opened against me about two months after I published my first gems. Now here is the funny thing, I got excited. Like watching the downloads trickle in on RubyGems.org, someone opening a bug against you means they care enough about what you are doing to want/need it to work right. Obviously I want it to work as well, but I know that I am bound to let bugs slip through. When someone took the time to report a bug and identify the potential source of the problem, I knew that I was doing something right. Well, something was wrong but the situation was right… right? [bug]
First Issue Filed Against Me Fixed
Fixing the issue and pushing out a new version was satisfying. I had the satisfaction of knowing that I cared enough about a little side project and the people using it to fix the issues and rerelease the gem.
Dealing With Future Compatibility
When I first pushed out my encrypted-cookie-store gem it was Rails 3.0 only. I figured I’d update it when 3.1 was released because I wasn’t expecting people to really care that much about it. Turns out I was mistaken. Not long after it was pushed out I got a request to add 3.1 compatibility. Well, one of my customers was asking for it and I was going to have to do it sooner or later, so I fired up rvm and got to work on figuring out what changed between 3.0 and 3.1. It wasn’t that big of a deal, but having different methods defined based on the which gems are installed seems like a less than ideal way to go about things. Thus concluded my first attempt at programing against an unreleased version of a project.
First Pull Request for One of My Projects
Now having someone open an issue against you and pointing you in the direction of the problem is nice and all, but having someone open a pull request against your project because they liked/needed it enough to fix the issue themselves, that is an awesome feeling. My little sparse matrix library was getting some love from across the Atlantic. Couldn’t have been happier.
First Issue Opened Against Another Project
Almost as scary as putting your own code out there is opening an issue against a well established project like Rails. You keep asking yourself, “Am I doing something wrong?” or “Do I just not understand what is going on?” I mean, what if your issue is just a result of you being stupid and not knowing what you are doing? These guys are busy and don’t really need to deal with bugs that probably aren’t actually bugs. So you run every test case you can think of, and then some that have nothing to do with your issue, you know, just in case. Then you hope that everyone is nice to the newbie. PS – They were.
First Pull Request to Another Project
It made me even more nervous to offer up a fix. Opening that Pull Request was nerve racking. I was patching one of the most used methods in ActiveRecord, so I obviously didn’t want to mess up. My pull request went through a couple of iterations, mostly on my test cases, but was finally accepted and merged into master 11 days after I opened the initial issue. Once again everyone was great and very helpful.
Started Blog
And then the blog started. I won’t lie to you, the last bit of motivation I needed to finally get this blog up and running was bit of selfishness. I realized that I needed some way to let people know about the stuff I had put out there that might be useful to them. This obvious thought happened when I came across a blog of someone working on a similar problem with encrypted cookies saying that they were thinking about packing it up and submitting it to rails. Since I liked the way I was doing it better, I figured I needed something other than the GitHub page as a way to promote it. Since then I have started writing on more topics than just the gems I have written. A blog, when properly utilized, is much more than just an advertising platform for your own code.
Being Asked to Contribute to a Project
The most recent development in my open source career is being asked to contribute to SciRuby. They liked the work I was doing on a sparse matrix library I was working on, and asked if they could include it in their offerings. Obviously I was ecstatic. Then I started thinking about the state of the code and how much work was left to do on it. Then I started worrying. It is very much still in an alpha/incomplete state. But now that I know that I have some people that are interested in it, I should be able to get working on it some more. Get that math side of my education some more use.
Conclusions
I have come a long way with my contributions to the open source community. From using to creating to contributing, each of these milestones has been a new challenge and a new experience. This isn’t a how to get a start in OSS, but it a list of things that you can look at for next steps when you are stuck and want to get more involved. It’s easy and most people will be excited that you want to help or are offering something up they can use. Even if you don’t want to write something, use OSS. That is your true first step.
On the Size of a String
July 21, 2011 § Leave a comment
In computer programs, number constants can be interesting and bewildering things. Trying to figure out why one was chosen over another can be really confusing. For a good while I was confused as to why ActiveRecord would set a string attribute to be a VARCHAR(255) in the database. It limits the size of the string attributes to 255 characters long. 256 is a bit more natural when choosing constants in computer science. 255 is commonly used to denoted the last index in a 0-based array of 256 elements. So why 255? The short answer is “because of InnoDB and UTF-8 character sets.”
InnoDB has a limitation on the size of a key for a single column index of 767 bytes. When the table is encoded with a UTF-8 character set, each characters has the possibility of using 3-bytes to represent its intended character. That means in order to be able to fully index a UTF-8 encoded varchar column, the string must be able to be represented in 767 bytes. 767 / 3 = 255 2/3. This means that the largest UTF-8 encoded varchar column can be 255 characters long, hence the ActiveRecord default string attribute size.
Problems on the Way
As bigger and bigger pushes are made for complete internationalization, we’ll see more things encoded with UTF-16 and UTF-32. Characters using these encodings might require up to 4 bytes to represent their value. When this happens, ActiveRecord will need to reduce the size of indexable string attributes to 191 characters.
For Fun
Here is a truly awesome magic number that seems to come out of no where, 0x5f3759d5.
On When Create Is Not Like Create
July 14, 2011 § Leave a comment
I recently ran into an interesting issue in ActiveRecord while trying to set default values on an object using the after_initialize callback. One would think the following blocks of code would be equivalent:
# new, save Product.new(:name => "Awesome Product").save # new w/ block, save product = Product.new do |p| p.title = "Awesome Product" end product.save # create Product.create(:title => "Awesome Product") # create w/ block Product.create do |p| p.title = "Awesome Product" end
In 99.9% of cases this is going to be true. The last one, create with a block, however, can potentially cause you some problems if you are using after_initialize. The problem arises when you use after_initialize to set default values for attributes are are dependent on other attributes. Let us consider our Awesome Product has two more attributes, msrp and wholesale_price, that are tied to each other. If we have one of them we can always determine what the other should be. In this case, there wouldn’t really be a reason set both of them when creating a new object. Just set one and let the other one get set automatically.
For our example we’ll say, msrp = 2 * wholesale_price. You might use an after_initialize that looks something like this:
def after_initialize
# set wholesale_price based on msrp
if !msrp.nil? && wholesale_price.nil?
self.wholesale_price = msrp / 2
# set msrp based on wholesale_price
elsif msrp.nil? && !wholesale_price.nil?
self.msrp = wholesale_price * 2
end
end
We can instantiate an object like this:
product = Product.new(:name => "Awesome Product", :msrp => 20) => <Product ...> product.save => true product.msrp => 20 product.wholesale => 10
Everything is working as it should. Now let’s use create instead of new and save.
product = Product.create(:name => "Awesome Product", :msrp => 20) => true product.msrp => 20 product.wholesale => 10
Still works just fine. Now create with a block:
product = Product.create do |p| p.name = "Awesome Product" p.msrp => 20 end => true product.msrp => 20 product.wholesale => nil
Uh, oh… Why didn’t wholesale_price didn’t get set? Take a look at the implementation of create in ActiveRecord::Base.
def create(attributes = nil, &block)
if attributes.is_a?(Array)
attributes.collect { |attr| create(attr, &block) }
else
object = new(attributes)
yield(object) if block_given?
object.save
object
end
end
Notice in the else block that a new object is created and then the block is yielded. This means that the after_initialize callback is run on the instantiated object BEFORE the block code is run. msrp is not set yet when after_initialize is run, so wholesale_price can’t be set. create without a block work fine because it is literally the same as using new and save.
TL;DR – after_initialize runs before the block code when using create and a block. Be careful when using after_initialize to set default values for attributes that depend on other attributes.
On Selecting A Single Column
July 9, 2011 § 1 Comment
Many times when we are selecting a rows out of the database we just want a single column and have no need for the entire object. There are a number of ways to accomplish this with ActiveRecord. One can get all the records from the database and then collect the attribute needed:
Posts.where(:status => 'published').collect(&:id) => [ 1, 5, 8, 10 ]
This has the benefit of being able to us any overwritten accessors, but has a lot of overhead associated with generating the objects. Another way to do it is to go directly to the database:
ActiveRecord::Base.connection.select_values("SELECT id FROM posts WHERE status = 'published'")
=> [ 1, 5, 8, 10 ]
This is much faster, but requires one to use the direct connection to the database and have the SQL literal prepared. Not particularly user friendly even if you can get the SQL literal using the to_sql:
ActiveRecord::Base.connection.select_values(Posts.where(:status => 'published').select(:id).to_sql) => [ 1, 5, 8, 10 ]
Wouldn’t it be nicer if you could just do the following:
Post.where(:status => 'published').select_column(:id) => [ 1, 5, 8, 10 ]
The select-column gem provides the above functionality above. You can you it in your Rails 3 app or checkout the source code over on github.
Usage
select_column accepts a single optional argument. This is the column that you want to have returned in an array. The returned column can also be specified using the select query method.
If neither a select nor an argument is given, :id is assumed to be the column to be returned. If multiple select query methods are present, the first one defined will be the column returned.
Some examples:
# selects an array of ids Post.select_column # selects an array of titles Post.select_column(:title) # selects an array of ids Post.where(:status => 'published').select_column # selects an array of titles Post.where(:status => 'published').select_column(:title) # selects an array of titles Post.select(:title).where(:status => 'published').select_column
Update (Jan 21, 2012): It’s like they keep looking at my gems and integrating them into Rails. As of Rails 3.2 this gem’s functionality has been replicated by ActiveRecord::Relation#pluck. Check it out in the release notes.
On Serialized Accessors
June 16, 2011 § 1 Comment
One of the nice things about brand new Rails apps is that all of your database tables are nice, small and manageable. You have relatively few columns in any given table and you are probably querying on most of the columns at some point in the app. As your app grows in size and complexity, so do your tables. A table that once upon a time had 5 or 10 columns now has 20 or 30. Not unreasonable number, but you are starting accumulate a number of columns that aren’t queried against. What should we do with these columns?
One option is to just let them build up. It doesn’t really hurt anything does it? I am sure that some database experts out there can weigh in on how number of columns affects performance. Option two is where I’d to show off something a little more interesting. Check out the data-attributes ruby gem over on github.
The premise is pretty simple. ActiveRecord allows for easy serialization of objects to a text field in the database. data-attributes makes use of this and adds attributes that read from and written to a serialized hash that is stored in a text field in the database. From the developers perspective, once a data attribute is defined, it can be used just like any other attribute. Validations work just like they do on column based attributes. To use this gem just add the following to your Gemfile in your Rails 3 project.
gem 'data-attributes'
Let’s take a look at it in action. We start with a user model with a serialized attribute called data.
class User < ActiveRecord::Base serialize :data end
Now, let’s say we have some piece of information that we want to include in the user record, but it isn’t something that we are going to have every query on. We define a data attribute like so:
data_attribute :favorite_food
Now how do we use this? We can see the results of adding the above to our Userobject.
u = User.new
u.favorite_food = "watermelon"
puts u.favorite_food
=> watermelon
puts u.data.inspect
=> { "favorite_food" => "watermelon" }
Pretty easy. The default field that the gem tries to save everything to is data, but that is just based on personal convention. If you want to change this, it’s as easy as adding the following line to your model:
data_attribute_column :food_preferences
Or if you have multiple serialized attributes and you want to send different data attributes to different serialized attributes you can do the following:
data_attribute :favorite_food, { :serialized_column => :food_preferences }
You can also set the default value to be returned for the data attribute:
data_attribute :favorite_food, { :default => "peanut butter" }
One of the things that makes all of this possible is that fact that ActiveRecord has multiple layers of accessor that happen when you call something like user.name. What happens is that the method name gets called which in turns calls read_attribute(:name). Similarly name=(val) calls write_attribute(:name, val).
data-attributes contains similar under-the-hood method read_data_attribute and write_data_attribute. This way you can have a little more control over the values that are read and written to your object.
Some things to be noted. If your ActiveRecord object has a serialized attribute then that attribute will be saved to database every time you call save. This is because it doesn’t know if the serialized object has been edited in place or not, so it just writes it to the database every time for good measure.
Update 2011-12-26: ActiveRecord 3.2 now has some of this basic functionality built in by way of ActiveRecord::Store.
The On Blog 