Requiring
There are essentially two ways to require activerecord-import:
- Using Bundler (default Rails 3 app uses this)
- Not using Bundler
These instructions only work for activerecord-import 0.2.0 or higher. If you have a version before 0.2.0 then upgrade.
Using Bundler
There are two ways you can use Bundler. First, the Rails 3 way.
The Rails 3 Way – autoloading gem dependencies
Rails 3 by default autoloads your gem dependencies (thanks to config/application.rb). When this happens you only need to make sure activerecord-import is specified as a gem dependency in your Gemfile.
gem "activerecord-import", ">= 0.2.0"
Not autoloading
If your gem dependencies aren’t autoloaded then simply require activerecord-import after activerecord has been loaded, ie:
require 'active_record' require 'activerecord-import'
That’s it.
https://github.com/zdennis/activerecord-import
activerecord-import
activerecord-import is a library for bulk inserting data using ActiveRecord.
Why activerecord-import?
Because plain-vanilla, out-of-the-box ActiveRecord doesn’t provide support for inserting large amounts of data efficiently. With vanilla ActiveRecord you would have to perform individual save operations on each model:
10.times do |i|
Book.create! :name => "book #{i}"
end
This may work fine if all you have is 10 records, but if you have hundreds, thousands, or millions of records it can turn into a nightmare. This is where activerecord-import comes into play.
An Introductory Example
To run examples on this page you’ll need to first require activerecord-import, for that please take a brief look at Requiring.
Here’s an example with equivalent behaviour using the #import method:
books = []
10.times do |i|
books << Book.new(:name => "book #{i}")
end
Book.import books
This call to import does whatever is most efficient for the underlying database adapter. Pretty slick, eh?
Features
Here’s a list of some of the high-level features that activerecord-import provides:
- activerecord-import can work with raw columns and arrays of values (fastest)
- activerecord-import works with model objects (faster)
- activerecord-import can perform validations (fast)
- activerecord-import can perform on duplicate key updates (requires mysql)
Upgrading from ar-extensions
activerecord-import replaces the ar-extensions library and is compatible with Rails 3. It provides the exact same API for importing data, but it does not include any additional ar-extensions functionality.]
desc "导入mail"
task :batch_import_email => :environment do
require 'active_record'
require 'activerecord-import'
count=0
count2=0
r=[]
File.open("/home/mlzboy/my/b2c2/doc/zzzzzzz.txt","r") do |file|
file.each_line do |line|
count2+=1
begin
if line.index("@").nil? ==false and line.index("zzzzzzz").nil?
email=line.chomp
#unless Subscription.exists?(:email=>email)
#Subscription.create(:email=>email)
r << Subscription.new(:email=>email)
count+=1
puts count
#end
end
rescue
puts "==================="
puts line
puts count2
end
if r.size > 0 and r.size % 10000 == 0
Subscription.import r
r=[]
end
end
if r.size!=0
Subscription.import r
end
end
put "finished!"
end