ruby on rails - Iterating over Sets of ActiveRecord Query Batch

Question

I have a table that has a few thousand sets of 2-3 nearly identical records, that all share a unique "id" (not database ID, but item id). That is, two to three records share the same item id and there are about 2100 records, or ~700 unique items. Example:

{id: 1, product_id:333, is_special:true}, {id:2, product_id:333, is_special:false}, {id:3, product_id:333, is_special:false}, {id:4, product_id:334, is_special:false}...

I'd like to perform a query that lets me iterate over each set, modify/remove duplicate records, then move on to the next set.

This is what I currently have:

task find_averages: :environment do
    responses = Response.all
    chunked_responses = responses.chunk_while { |a,b|  a.result_id == b.result_id}.to_a
    chunked_responses.each do |chunk|
        if chunk.length < 3
            chunk.each do |chunky_response|
                chunky_response.flagged = true
                chunky_response.save
            end
        else
            chunk.each do |chunky_response|
               **manipulate each item in the chunk here**
            end
        end
    end
end

Edit. I worked this one out after discovering the chunk_while method. I am not positive chunk_while is the most efficient method here, but it works well. I am closing this, but anyone else that needs to group records and then iterate over them, this should help.

question from:https://stackoverflow.com/questions/65911915/iterating-over-sets-of-activerecord-query-batch

Answer 1

The following code should iterate over an array of items that some of them share common values, and group them by common values:

responses = Responses.all
chunked_responses = responses.chunk_while { |a,b|  a.result_id == b.result_id}.to_a
chunked_responses.each do |chunk|
  chunk.each do |chunky_response|
  **manipulate each item in the chunk here**
  end
end