Add support for refreshing HDFS delegation tokens

Description

Currently, tickets are managed at the client end (HDFS client) and need a mechanism to renew them.

Similar to spark way that you pass the kerberos ticket such like as:

spark-submit ... --conf spark.yarn.keytab=/opt/software/adkeytabs/<<OWNER>>.keytab --conf spark.yarn.principal=<<OWNER>>@<DOMAIN> …<rest of the arguments>

Activity

Show:
Michal Kurka
March 3, 2021, 4:13 PM

Based on https://hadoop.apache.org/docs/r3.1.2/hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html we should adopt a similar strategy that Spark uses - “AM keytab distributed via YARN; AM regenerates delegation tokens for containers”

Basically we need to re-implement logic of these classes:

The difference from Spark is that we do not implement our own application master, however, in the h2o setting we should be able to do the same on the cluster leader node.

Other useful links:

Fixed

Assignee

Michal Kurka

Fix versions

Reporter

Alan Silva

Support ticket URL

Labels

None

Affected Spark version

None

Customer Request Type

None

Task progress

None

ReleaseNotesHidden

None

CustomerVisible

No

Priority

Major