Rectangle 27 0

a very simple solution to your problem would be to split the amount of work, so that workers don't interfere with each other.

if you have 2 workers, you could number them and do a modulo on the id % 2. this way you get only 0 and 1 as a result. worker with number 0 only works on his batch, worker with number 1 on the other. by increasing the modulo you can go up to whatever number of workers you want.

Actual code is much more complicated, and for each photo real set of possible matches can be different. So they are not equal, and I can't just divide this list into sections for each worker. But thank you for an interesting suggestion!

another easy solution would be to persist a worker id with the matches, so that they can be departed that way.

the usual way to do this kind of stuff would be to have a main worker that issues other workers with the matches. so you have just one worker responsible for finding matches, the rest is doing the hard stuff and get assigned just one match.

ruby on rails - How to prevent Sidekiq from parallel modifications to ...

ruby-on-rails multithreading sidekiq
Rectangle 27 0

I would say you have a bad environment configured. As stated on some comments, you could be having one of the following problems:

  • The sidekiq client and server are running under different redis.

Try to debug one by one of the cases above and lemme know if the problem still persists.

Ruby on Rails using Sidekiq to populate MongoDB database - Stack Overf...

ruby-on-rails ruby mongodb mongoid sidekiq