R error 3.10.3.3: Error: water.exceptions.H2OModelBuilderIllegalArgumentException:
Description
When running this code (included in Kaggle Competition):
It generates the error:
Error: water.exceptions.H2OModelBuilderIllegalArgumentException: Illegal argument(s) for GBM model: gbm1. Details: ERRR on field: _ntrees: The tree model will not fit in the driver node's memory (1,0 MB per tree x 500 > 503,2 MB) - try decreasing ntrees and/or max_depth or increasing min_rows!
In version 3.10.3.3
While the same code run in version 3.10.0.10 runs without any error, limitation.
I run the same code (ntrees: 2000) in the same machine with the same dataset and h2o script.
In both cases, I run it with the same initial configuration parameters:
h2o.init(nthreads = -1, min_mem_size = "6G")
Activity
I am still seeing this issue in the latest version of h2o, 3.10.4.8.
On h2o version 3.10.0.8 The following script :
https://s3.amazonaws.com/6si-ds-pojo/tmp/obfuscated.R
with the following dataset:
https://s3.amazonaws.com/6si-ds-pojo/tmp/obfuscated.csv
is able to build all of the models however on the latest version of h2o i get the OOM error:
Illegal argument(s) for DRF model: Grid_DRF_main_model_R_1495513529770_1_model_11_cv_1. Details: ERRR on field: _ntrees: The tree model will not fit in the driver node's memory
I am reopening this issue because this is not a JVM memory setting issue. I was able to perform the grid search without error in 3.10.0.8 using the same exact Rscript and dataset.
Memory would have to be increased from the looks of the exception
