Support non-linear phase dependencies

CHANGELOG.md

@@ -7,7 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-## 2.0.0
+## 2.1.0 : 2025-01-24 [diff](https://github.com/procore-oss/handcuffs/compare/v2.0.0..main)
+
+### Added
+
+- Ability to specify prerequisite phases in a non-linear order
+
+### Changed
+
+- (internal) bumped rspec-rails gem version in development dependencies
+- (internal) bumped minimum gem versions in test Rails app
+- (internal) update github workflow
+
+
+## 2.0.0 : 2024-02-20 [diff](https://github.com/procore-oss/handcuffs/compare/v1.4.1..v2.0.0)
+
+### Removed
+
+- **BREAKING CHANGE**: Removed support for Ruby < 2.7, Rails < 6.1, PostgreSQL < 12.
 
 ### Added
 
@@ -24,7 +41,3 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Updated Bundler to 2.4.22.
 - Added Appraisal for dummy app testing.
 - Moved repo to procore-oss
-
-### Removed
-
-- BREAKING CHANGE: Removed support for Ruby < 2.7, Rails < 6.1, PostgreSQL < 12.

README.md

@@ -2,59 +2,114 @@
 
 [![Test](https://github.com/procore-oss/handcuffs/actions/workflows/test.yaml/badge.svg?branch=main)](https://github.com/procore-oss/handcuffs/actions/workflows/test.yaml)
 [![Gem Version](https://badge.fury.io/rb/handcuffs.svg)](https://badge.fury.io/rb/handcuffs)
-[![Discord](https://img.shields.io/badge/Chat-EDEDED?logo=discord)](https://discord.gg/PbntEMmWws) 
+[![Discord](https://img.shields.io/badge/Chat-EDEDED?logo=discord)](https://discord.gg/PbntEMmWws)
 
-Handcuffs provides an easy way to run migrations in phases in your [Ruby on Rails](https://rubyonrails.org/) application.
+Handcuffs provides an easy way to run [Ruby on Rails](https://rubyonrails.org/) migrations in phases using a simple process:
 
-To configure, first create a handcuff initializer and define a configuration
+1. Define a set of named phases in the order in which they should be run
+2. Tag migrations with one of the defined phase names
+3. Run migrations by phase at start, end or outside of application deployment
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'handcuffs'
+```
+
+And then execute:
+
+```bash
+bundle
+```
+
+Or install it directly on the current system using:
+
+```bash
+gem install handcuffs
+```
+
+
+## Usage
+
+### Configuration
+
+Create a handcuffs initializer and define the migration phases in the order in which they should be run. You should also define a default phase for pre-existing "untagged" migrations, or if you want the option to tag only custom phases.
+
+The most basic configuration is an array of phase names, and using the first one as the default:
 
 ```ruby
 # config/initializers/handcuffs.rb
 
 Handcuffs.configure do |config|
+  # pre_restart migrations will/must run before post_restart migrations
   config.phases = [:pre_restart, :post_restart]
+  config.default_phase = :pre_restart
 end
 ```
 
-Then call `phase` from inside your migrations
+If you have more complex or asynchrous workflows, you can use an alternate hash notation that allows prerequisite stages to be specified explicitly:
 
 ```ruby
-# db/migrate/20160318230933_add_on_sale_column.rb
+# config/initializers/handcuffs.rb
 
-class AddOnSaleColumn < ActiveRecord::Migration
+Handcuffs.configure do |config|
+  config.phases = {
+    # Prevent running post_restart migrations if there are outstanding
+    # pre_restart migrations
+    post_restart: [:pre_restart],
+    # Require pre_restarts before data_migrations, but do not enforce ordering
+    # between data_migrations and post_restarts
+    data_migrations: [:pre_restart],
+    # pre_restarts have no prerequisite phases
+    pre_restart: []
+  }
+end
+```
+
+The default phase order in this case is determined by [Tsort](https://github.com/ruby/tsort) (topilogical sort). In order to validate the configuration and expected phase order it is reccomended that you check the phase configuration after any changes using the rake task:
+
+```ruby
+rake handcuffs:phase_order
+```
+
+This will display the default order in which phases will be run and list the prerequisites of each phase. It will raise an error if there are any circular dependencies or if any prerequisite is not a valid phase name.
+
+### Tagging Migrations
+
+Once configured, you can assign each migration to one of the defined phases using the `phase` setter method:
+
+```ruby
+# db/migrate/20240318230933_add_on_sale_column.rb
+
+class AddOnSaleColumn < ActiveRecord::Migration[7.0]
 
   phase :pre_restart
 
-  def up
+  def change
     add_column :products, :on_sale, :boolean
   end
-
-  def down
-    remove_column :products, :on_sale
-  end
-
 end
 ```
 
 ```ruby
-# db/migrate/20160318230988_add_on_sale_index
+# db/migrate/20240318230988_add_on_sale_index
 
-class AddOnSaleIndex < ActiveRecord::Migration
+class AddOnSaleIndex < ActiveRecord::Migration[7.0]
 
   phase :post_restart
 
-  def up
+  def change
     add_index :products, :on_sale, algorithm: :concurrently
   end
-
-  def down
-    remove_index :products, :on_sale
-  end
-
 end
 ```
 
-You can then run your migrations in phases using
+### Running Migrations In Phases
+
+After Handcuffs is configured and migrations are properly tagged, you can then run migrations in phases using the `handcuffs:migrate` rake task with the specific phase to be run:
 
 ```bash
 rake 'handcuffs:migrate[pre_restart]'
@@ -66,62 +121,44 @@ or
 rake 'handcuffs:migrate[post_restart]'
 ```
 
-You can run all migrations using
+*Note:* If you run phases out of order, or attempt to run a phase before outstanding migrations with a prerequisite phase have been run, a `HandcuffsPhaseOutOfOrderError` will be raised.
 
-```bash
-rake 'handcuffs:migrate[all]'
-```
+### Running All Migrations
 
-This differs from running `rake db:migrate` in that migrations will be run in the _order that the phases are defined in the handcuffs config_.
+In CI and local developement you may want to run all phases at one time.
 
-If you run a handcuffs rake task and any migration does not have a phase defined, an error will be raised before any migrations are run. To prevent this error, you can define a default phase for migrations that don't define one.
+Handcuffs offers a single command that will run all migrations in phases and in the configured order:
 
-```ruby
-# config/initializers/handcuffs.rb
-
-Handcuffs.configure do |config|
-  config.phases = [:pre_restart, :post_restart]
-  config.default_phase = :pre_restart
-end
+```bash
+rake 'handcuffs:migrate[all]'
 ```
 
-## Installation
+This differs from running `rake db:migrate` in that migrations will be run in batches corresponding to the _order that the phases are defined in the handcuffs config_. Again, you can use `rake handcuffs:phase_order` to preview the order ahead of time.
 
-Add this line to your application's Gemfile:
+Of course, you can always run `rake db:migrate` at any time to run all migrations using the Rails default ordering and without regard to Handcuffs phase if you wish.
 
-```ruby
-gem 'handcuffs'
-```
 
-And then execute:
 
-```bash
-bundle
-```
+## Contributing
 
-Or install it yourself as:
+Bug reports and pull requests are welcome on GitHub at <https://github.com/procore-oss/handcuffs>. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
-```bash
-gem install handcuffs
-```
 
-## Running specs
+## Running Tests Locally
 
 The specs for handcuffs are in the dummy application at `/spec/dummy/spec`. The spec suite requires PostgreSQL. To run it you will have to set the environment variables `POSTGRES_DB_USERNAME` and `POSTGRES_DB_PASSWORD`. You can then run the suite using `rake spec`
 
-## Contributing
-
-Bug reports and pull requests are welcome on GitHub at <https://github.com/procore-oss/handcuffs>. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
 ## License
 
 The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
 
+
 ## About Procore
 
 <img
-  src="https://www.procore.com/images/procore_logo.png"
-  alt="Procore Logo"
+  src="https://raw.githubusercontent.com/procore-oss/.github/main/procorelightlogo.png"
+  alt="Procore Open Source"
   width="250px"
 />
 

lib/handcuffs/configuration.rb

@@ -10,15 +10,17 @@ def self.configure
   end
 
   class Configurator
-    attr_accessor :phases
+    attr_reader :phases
     attr_accessor :default_phase
 
     def initialize
       @phases = []
       @default_phase = nil
     end
 
+    def phases=(phases)
+      @phases = Phases.new(phases)
+    end
   end
-
 end
 

lib/handcuffs/extensions.rb

@@ -1,12 +1,12 @@
+# frozen_string_literal: true
+
 module Handcuffs
+  # Extended by ActiveRecord::Migrator in order to track the current phase
   module Extensions
-
-    attr_reader :handcuffs_phase
+    attr_accessor :handcuffs_phase
 
     def phase(phase)
       @handcuffs_phase = phase
     end
-
   end
 end
-

lib/handcuffs/pending_filter_ext.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module Handcuffs
+  # PendingFilter is prepended to ActiveRecord::Migrator in the rake tasks
+  # in order to check the current phase before it is run
+  module PendingFilterExt
+    def runnable
+      attempted_phase = self.class.handcuffs_phase
+      if @direction == :up
+        Handcuffs::PhaseFilter.new(attempted_phase, @direction).filter(super)
+      else
+        phase_migrations = Handcuffs::PhaseFilter.new(attempted_phase, @direction).filter(migrations)
+        runnable = phase_migrations[start..finish]
+        runnable.pop if target
+        runnable.find_all { |m| ran?(m) }
+      end
+    end
+  end
+end

lib/handcuffs/phase_filter.rb

@@ -41,7 +41,7 @@ def runnable_for_phase(by_phase, defined_phases)
   end
 
   def check_order_up!(by_phase, defined_phases)
-    defined_phases.take_while { |defined_phase| defined_phase != attempted_phase }
+    defined_phases.prereqs(attempted_phase)
       .detect { |defined_phase| by_phase.key?(defined_phase) }
       .tap do |defined_phase|
         raise HandcuffsPhaseOutOfOrderError.new(defined_phase, attempted_phase) if defined_phase
@@ -58,7 +58,7 @@ def check_order_down!(by_phase, defined_phases)
   end
 
   def all_phases_by_configuration_order(by_phase, defined_phases)
-    defined_phases.reduce([]) do |acc, phase|
+    defined_phases.in_order.reduce([]) do |acc, phase|
       acc | Array(by_phase[phase])
     end.map { |mh| mh[:proxy] }
   end

lib/handcuffs/phases.rb

@@ -0,0 +1,41 @@
+# frozen_string_literal: true
+
+require 'tsort'
+
+module Handcuffs
+  # Phases encapsulates the list of phases and any interdependencies
+  class Phases
+    def initialize(phases)
+      @phases = case phases
+                when Hash
+                  phases.each_with_object({}) do |phase, acc|
+                    acc[phase[0].to_sym] = Array(phase[1]).map(&:to_sym)
+                  end
+                else
+                  # Assume each entry depends on all entries before it
+                  phases.map(&:to_sym).each_with_object({}) do |phase, acc|
+                    acc[phase] = phases.take_while { |defined_phase| defined_phase != phase }
+                  end
+                end
+    end
+
+    def to_sentence
+      @phases.keys.to_sentence
+    end
+
+    def include?(phase)
+      @phases.include?(phase)
+    end
+
+    def in_order
+      TSort.tsort(
+        @phases.method(:each_key),
+        ->(phase, &block) { @phases.fetch(phase).each(&block) }
+      )
+    end
+
+    def prereqs(attempted_phase)
+      @phases.fetch(attempted_phase, [])
+    end
+  end
+end