Better Rails Performance with JSON Patch

DocSpring is a service that generates PDFs. The backend is written with Ruby on Rails, and the PDF template editor is built using React. We use Postgres, and store the template’s field data in a jsonb column.

The first version of our template editor used a naïve approach when saving field data. Whenever the fields changed, we would just post the entire array to the server. This was working fine for our MVP and initial launch, but then a customer started setting up a PDF template with almost 500 fields. We started seeing alerts for requests that were taking around 5-10 seconds.

The obvious solution was to only send changes to the server. We use Redux in our React app, so I thought about sending the Redux actions. This wasn’t an option for DocSpring, because our Redux actions include some complicated logic. I didn’t want to rewrite that code in Ruby, but if we were using Node.js, then we might have been able to re-use the same JavaScript on the server.

The other option is to diff the plain JSON objects, and send the diff to the server. We can use JSON Patch for that:

JSON Patch is a format for describing changes to a JSON document. It can be used to avoid sending a whole document when only a part has changed.

I think it’s almost always better to use an IETF standard, instead of inventing your own thing¹. I used the following open source libraries:

fast-json-patch — An NPM package to generate a JSON patch on the client
hana — A Ruby gem to apply the JSON patch on the Rails server

Here’s how I am applying patches to json columns in my Rails models:

class Template < ApplicationRecord
  # If there are any errors, we store them in here,
  # and add them during validation
  attr_accessor :json_patch_errors
  validate :add_json_patch_errors
  after_save :clear_json_patches

  attr_reader :fields_patch

  # The patch is applied as soon as this method is called.
  def fields_patch=(patch_data)
    # In case we want to access it later.
    @fields_patch = patch_data
    self.json_patch_errors ||= {}
    json_patch_errors.delete :fields

    unless patch_data.is_a?(Array)
      json_patch_errors[:fields] =
        'JSON patch data was not an array.'
      return
    end

    hana_patch = Hana::Patch.new(patch_data)
    begin
      hana_patch.apply(fields)
    rescue Hana::Patch::Exception => ex
      json_patch_errors[:fields] =
        "Could not apply JSON patch to \"fields\": #{ex.message}"
    end
  end

  # Clear any JSON patches and errors when reloading data
  def reload
    super
    clear_json_patches
  end


  private

  def add_json_patch_errors
    return unless json_patch_errors.present?
    json_patch_errors.each do |attribute, errors|
      errors.add(attribute, errors)
    end
  end

  def clear_json_patches
    @fields_patch = nil
    self.json_patch_errors = nil
  end
end

You could copy these RSpec tests to make sure your implementation is correct. We might also release this as a gem at some point.

I added fields_patch as a permitted parameter in the controller:

params.require(:template).permit(
  fields: {},
).tap do |permitted|
  # Nested arrays and hashes are tricky.
  if params[:template][:fields_patch].is_a?(Array)
    permitted[:fields_patch] = params[:template][:fields_patch].
      map(&:permit!)
  end
end

This means that :fields_patch is treated like a normal attribute, and the patch will be applied during update. If a patch fails to apply, an error will be added during validation.

The front-end implementation was very easy. Our previous code looked like this:

if (!Immutable.is(template.fields, previousTemplate.fields)) {
  data.fields = template.fields.toJS();
}

The new code sends a JSON patch as the fields_patch attribute.

import { compare as jsonPatchCompare } from "fast-json-patch";

if (!Immutable.is(template.fields, previousTemplate.fields)) {
  data.fields_patch = jsonPatchCompare(
    previousTemplate.fields.toJS(),
    template.fields.toJS()
  );
}

Here’s an example AJAX request from the new code:

{
  "template": {
    "fields_patch": [
      {
        "op": "replace",
        "path": "/2/name",
        "value": "image_field_2"
      }
    ]
  }
}

This change was just a few lines of code, but now we are sending much less data.

Another advantage of JSON Patch is that it’s much easier to support concurrent editing with multiple users. It is often possible to apply patches in any order, especially when they only contain replace or insert operations. If there is a conflict, then you can just reload the latest data and let the user try again. You can also keep all the clients in sync by using websockets to send patches from the server to the browser.

Thanks for reading! You can leave a comment on Hacker News.