Provide schema files for SQL databases

[jenschelkopf]

I'm trying to use remotedev-server with Postgres but I need the report table to be defined in the database before remotedev-server can insert any reports. I could look at the source code to hack it up, but this seems like something remotedev-server should provide if it expects to be usable with SQL databases. Especially if there are any schema changes in the future, which I imagine there could be 😀

Looks like js-data-sql is using Knex under the covers, which supports database agnostic schema files: http://www.dancorman.com/knex-your-sql-best-friend/

[zalmoxisus]

@jrullmann thanks for investigating these! Yes, it's a must and we could probably integrate it with js-data schema.

As you could noticed it's more a prototype here to see how it will go.

[zalmoxisus]

BTW, do you find the unopinionated database engine useful? What if it will take care of everything, will use only one database, which will be installed and managed under the hood?

The advantage of the current approach is that in the feature we could hook in the app's tables, for example, to see details about users which sent the report or about data which failed the request. While using its engine, make it simpler to use. We could offer more sugar out of the box, like GraphQL API. To make it more performant using something like Druid, which is very efficient for logging.

[jenschelkopf]

I like the ability to choose my own database because ultimately I'll have to operate that database. You can't put concerns like backups and high availability under the hood.

On the other hand, I'd rather have fewer choices that are well supported. After hacking together the schema files I ran into a version incompatibility issue between js-data-sql and knex. Since js-data-sql swallows errors it took some deep debugging to find the problem.

I'd definately take fewer choices that are better supported. Assuming they fit my operational experience (Postgres comes to mind 😀)

[zalmoxisus]

@jrullmann, yes, the fact that it is not actively maintained, and that the current implementation is very limited worries me as well.

As js-data cannot get specific fields, we have to mimic that, which will make listing logs rather slow. Adding the ability to specify fields in js-data-sql is pretty trivial, but we should take into account other adapters, and I feel like I wouldn't have time to maintain that part.

Sticking to Postgres would be a good option, moreover there's awesome postgraphql, which author contributed to Redux DevTools Extension as well.

Also I'm considering Elasticsearch. The advantage is that it can be deployed locally and as a service too. Also its Kibana dashboard is pretty awesome. @ccorcos suggested it.

[ccorcos]

@zalmoxisus I'm not actually suggesting it -- its just the solution we currently have at Affirm, although we're planning on moving toward Graphana and InfluxDb...

[zalmoxisus]

OK, so the current plan is to drop js-data and just use knex. I'll make a PR tomorrow, and will add a migration script as well.

[zalmoxisus]

@jrullmann I removed js-data in favour of knex and added migration schemas. You can give it a try with [email protected]. Let's continue the discussion in #30.