H2O failing start-up on Apache Hadoop 3.0.0-alpha3

Description

Running :
hadoop jar h2o.jar -ice_root /data -flow_dir /flows

On Apache Hadoop 3.0.0-alpha3 (it seems the only version affected)
Following error occurs:
2017-07-02 09:32:35,044 INFO java.NativeLibrary: Loaded library from lib/linux_64/libxgboost4j_gpu.so (/tmp/libxgboost4j_gpu1041391911743955741.so)
2017-07-02 09:32:36,551 INFO util.log: Logging initialized @6202ms
Exception in thread "main" java.lang.NoSuchMethodError: org.eclipse.jetty.server.AbstractConnector: method <init>()V not found
at org.eclipse.jetty.server.bio.SocketConnector.<init>(SocketConnector.java:70)
at water.AbstractHTTPD.startHttp(AbstractHTTPD.java:198)
at water.AbstractHTTPD.start(AbstractHTTPD.java:90)
at water.init.NetworkInit.initializeNetworkSockets(NetworkInit.java:95)
at water.H2O.startLocalNode(H2O.java:1470)
at water.H2O.main(H2O.java:1959)
at water.H2OStarter.start(H2OStarter.java:21)
at water.H2OStarter.start(H2OStarter.java:42)
at water.H2OApp.main(H2OApp.java:9)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)

Activity

Show:
Patrick Rice
July 7, 2017, 9:38 PM
Edited

This is because they launched using h2o instead of h2odriver

I do, however, still get an exception when launching h2odriver
I'm assuming this is because we currently do not have a build for hadoop 3.0.0 and the abi is not identical

This is the exception I get when using an h2odriver

Exception in thread "main" java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$TokenManagementDelegator
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3172)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3217)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3256)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3307)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3275)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:225)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:460)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.addInputPath(FileInputFormat.java:533)
at water.hadoop.h2odriver.run2(h2odriver.java:1539)
at water.hadoop.h2odriver.run(h2odriver.java:1695)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at water.hadoop.h2odriver.main(h2odriver.java:1733)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)

Michal Malohlava
July 6, 2017, 9:57 PM

Hi ,

may you please try to reproduce this one?

Suspicious points:

  • hadoop h2o.jar // Normally we use h2odriver.jar

  • This is orgiinally filled as Sparkling Water issue, but we did not find any SW related code here

Assignee

Patrick Rice

Fix versions

None

Reporter

Fabrizio Torelli

Support ticket URL

None

Labels

None

Affected Spark version

Spark 1.5

Customer Request Type

None

Task progress

None

ReleaseNotesHidden

None

CustomerVisible

Yes

Epic Link

Components

Affects versions

Priority

Blocker