Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eclairjs node treats spark objects as immutable #93

Open
pberkland opened this issue May 25, 2016 · 0 comments
Open

eclairjs node treats spark objects as immutable #93

pberkland opened this issue May 25, 2016 · 0 comments
Labels

Comments

@pberkland
Copy link
Collaborator

pberkland commented May 25, 2016

A side effect of the eclairjs node code generation is a number of spark objects are treated as immutable, thereby making the nodejs spark api not work the same as the normal spark api.
for instance, the following code will not work

      var paramMap2 = new spark.ml.param.ParamMap();
    paramMap2.put(lr.probabilityCol().w("myProbability")); // Change output column name
    var paramMapCombined = paramMap.$plus$plus(paramMap2);

because the $plus$plus will be executed before the paramMap2.put is executed.
This can be worked around by instead doing

    var paramMap3 = paramMap2.put(lr.probabilityCol().w("myProbability")); 
    var paramMapCombined = paramMap.$plus$plus(paramMap3);

but it is not obvious to the eclairjs user when that should be done, because with the normal spark programing (including eclairjs nashorn) it is not necessary.

@pberkland pberkland added the bug label May 25, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant