Sparkling water External Backend Support in kerberized cluster

Description

Additionally expose H2O params:
principal, keytab, run_as_user

Activity

Show:
Jakub Hava
August 23, 2018, 8:21 AM

In internal backend, we use spark.yarn.principal and spark.yarn.keytab.

However in external backend, automatic mode, we also need to set up kerberos parameters to h2o driver.

Therefore we need to expose principal and keytab as params. We can be smart and if we see that spark.yarn.keytab and spark.yarn.principal is set, automatically set principal and keytab in External backend

Jakub Hava
August 22, 2018, 10:50 AM

After discussion with , we came to a conclusion that Spark as the main application should be handling kerberised access

Jakub Hava
August 22, 2018, 10:46 AM
Edited

Actually, principal & key tab are available just in case of the H2ODriver.

However it makes more sense to use kerberos principal and & keytab on the sparkl level as H2O is just embedded application

Fixed

Assignee

Jakub Hava

Reporter

Ondrej Bilek

Labels

None

CustomerVisible

No

testcase 1

None

testcase 2

None

testcase 3

None

h2ostream link

None

Affected Spark version

None

AffectedContact

None

AffectedCustomers

None

AffectedPilots

None

AffectedOpenSource

None

Support Assessment

None

Customer Request Type

None

Support ticket URL

None

End date

None

Baseline start date

None

Baseline end date

None

Task progress

None

Task mode

None

ReleaseNotesHidden

None

Fix versions

Priority

Major