You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The spark programming style of session.createDataset(javaObject, javaObject.class) is much more common in Spark 2.0. We need some kind of JS equivalent.
We should be support something like createDataset(jsObject, { fieldName : "String", fieldName2:"Integer"});
This will probably require creating a JSON encoder which is not derived from the RowEncoder.
The text was updated successfully, but these errors were encountered:
Doing a real JSON encoder which is not derived from RowEncoder would be difficult, because encoder implementations generate code under the covers. This is done thru internal classes. Trying to do this for JSON would require good understanding of the Encoder internals.
Possible future versions of spark will make it easier to create custom encoders.
The 'createDatasetFromJSON' api seems to be working for current needs.
The spark programming style of session.createDataset(javaObject, javaObject.class) is much more common in Spark 2.0. We need some kind of JS equivalent.
We should be support something like createDataset(jsObject, { fieldName : "String", fieldName2:"Integer"});
This will probably require creating a JSON encoder which is not derived from the RowEncoder.
The text was updated successfully, but these errors were encountered: