Create table function in Spark in R not working
Hello, I'm trying to create a simple table in Bluemix Spark (R) but getting a error I can't understand or find useful reference about. Has anyone seen something similar? Any suggestions, please?
one <- rep(1,2)two <- rep(2,2)my_table <- table(one,two)my_tableError: class(objId) == "jobj" is not TRUETraceback:1. table(one, two)2. callJMethod(sqlContext, "table", tableName)3. stopifnot(class(objId) == "jobj")4. stop(sprintf(ngettext(length(r), "%s is not TRUE", "%s are not all TRUE"),. ch), call. = FALSE, domain = NA)
I think this happens due to how in R the libraries are loaded. In this case the last library loaded is "SparkR", which has a function exported with the name "table". The SparkR::table function is used, if you not fully qualify the function. That explains why in the traceback "callJMethod(sqlContext, "table", tableName)" is stated.
On workaround is to qualify the function. This worked for me for R - Spark 1.6
one <- rep(1,2)two <- rep(2,2)my_table <- base::table(one,two)my_table
However, it is interesting that it works for R - Spark 2.0.
Does your code snippet work in other R environments? It looks very much different from the table examples that I could find. Are you sure that "rep" in your example generates a "factor"? Maybe you have to call the "factor" function explicitly, as shown in the first link?
http://www.cyclismo.org/tutorial/R/types.html#tables
Hello, yes, it used to work. I got it working now after changing kernel to "R with Spark 2.0".
Thanks for your help and the links provided.
João