JSONDB(ish)

By Danielle Kilgore

It's that time of year when I have a little time to piddle about, so I decided to do a little research, prompted by reading this post from Andrea Giammarchi. I've written loads about DataTables over the years, so think about grabbing data from APIs probably far too much for someone to be sane. I'm also old enough to remember that AJAX was initially designed to get XML from data sources rather than the JSON we currently, mainly, acquire and process. There's a section in my book (pp. 143-147) about the costs and benefits of both data formats.

Andrea prompted my examination of JSONH, which - in turn - pointed me to JSONDB, and then I found Devon Govett's implementation. I got to thinking about making JSON even smaller!

In an early draft, I compared the relative sizes of both the XML and JSON used as an example; I thought it'd be interesting to fiddle with the family's data I use to get to grips with the adapted CoffeeScript implementation of JSONDB. I also tweaked it to consider nested arrays of objects because why wouldn't you? Anyway, this is the data I'll be comparing:

const familyJSON = {
  family: [
    {
      title: "Dr",
      forename: "08",
      surname: "08",
      dateOfBirth: "08/08/0808",
      hairColour: "brown",
      eyeColour: "brown",
      handedness: "left",
      something: [
        {
          one: 1,
          two: 2,
          three: [1, 2]
        },
        {
          one: 2,
          two: 4,
          three: [2, 4]
        }
      ]
    },
    {
      title: "Dr",
      forename: "09",
      surname: "09",
      dateOfBirth: "09/09/0909",
      hairColour: "white",
      eyeColour: "silver",
      handedness: "ambidextrous",
      something: [
        {
          one: 3,
          two: 4,
          three: [3, 4]
        },
        {
          one: 6,
          two: 8,
          three: [6, 8]
        }
      ]
    },
    {
      title: "Dr",
      forename: "10",
      surname: "10",
      dateOfBirth: "10/10/1010",
      hairColour: null,
      eyeColour: null,      
      handedness: "ambilevous",
      something: [
        {
          one: 5,
          two: 6,
          three: [5, 6]
        },
        {
          one: 10,
          two: 12,
          three: [10, 12]
        }
      ]
    },
    {
      title: "Dr",
      forename: "11",
      surname: "11",
      dateOfBirth: "11/11/1111",
      hairColour: "brown",
      eyeColour: "brown",
      handedness: "right",
      something: [
        {
          one: 7,
          two: 8,
          three: [7, 8]
        },
        {
          one: 14,
          two: 16,
          three: [14, 16]
        }
      ]
    }
  ]
}

And this is the helper class I created:

class JSONDB {

  static isObject = o => o instanceof Object && o.constructor === Object

  static pack = records => {
    const keys = Object.keys(records[0]);
    const ret = ['JSONDB', keys.length, ...keys]
    for (let i = 0; i < records.length; i++) {
      for (const key in records[i]) {
        const record = records[i][key]
        ret.push(Array.isArray(record) && record.every(el => 
          this.isObject(el)) 
            ? this.pack(record) 
            : record)
      }
    }
    return ret;
  };

  static unpack = array => {
    if (array[0] !== 'JSONDB') { 
      return array; 
    }
    const numKeys = array[1];
    const pos = numKeys + 2;
    const keys = array.slice(2, pos);
    const ret = [];
    let cur = {};
    const iterable = array.slice(pos);
    for (let i = 0; i < iterable.length; i++) {
        let val = iterable[i];
        if(Array.isArray(val)){
          val = this.unpack(val)
        }
        if ((i > 0) && ((i % numKeys) === 0)) {
            ret.push(cur);
            cur = {};
        }
        const key = keys[i % numKeys];
        cur[key] = val;
    }
    ret.push(cur);        
    return ret;
  }

}

This example shows the saving of 339 characters.

This is the compressed data:

[
  "JSONDB",
  8,
  "title",
  "forename",
  "surname",
  "dateOfBirth",
  "hairColour",
  "eyeColour",
  "handedness",
  "something",
  "Dr",
  "08",
  "08",
  "08/08/0808",
  "brown",
  "brown",
  "left",
  [
    "JSONDB",
    3,
    "one",
    "two",
    "three",
    1,
    2,
    [
      1,
      2
    ],
    2,
    4,
    [
      2,
      4
    ]
  ],
  "Dr",
  "09",
  "09",
  "09/09/0909",
  "white",
  "silver",
  "ambidextrous",
  [
    "JSONDB",
    3,
    "one",
    "two",
    "three",
    3,
    4,
    [
      3,
      4
    ],
    6,
    8,
    [
      6,
      8
    ]
  ],
  "Dr",
  "10",
  "10",
  "10/10/1010",
  null,
  null,
  "ambilevous",
  [
    "JSONDB",
    3,
    "one",
    "two",
    "three",
    5,
    6,
    [
      5,
      6
    ],
    10,
    12,
    [
      10,
      12
    ]
  ],
  "Dr",
  "11",
  "11",
  "11/11/1111",
  "brown",
  "brown",
  "right",
  [
    "JSONDB",
    3,
    "one",
    "two",
    "three",
    7,
    8,
    [
      7,
      8
    ],
    14,
    16,
    [
      14,
      16
    ]
  ]
]

I'm wondering where I can implement this approach now; it would seem to offer significant savings in terms of the size of the payload while still returning something that will be relatively quick to convert back into JSON.