Add support for refreshing HDFS delegation tokens
Currently, tickets are managed at the client end (HDFS client) and need a mechanism to renew them.
Similar to spark way that you pass the kerberos ticket such like as:
spark-submit ... --conf spark.yarn.keytab=/opt/software/adkeytabs/<<OWNER>>.keytab --conf spark.yarn.principal=<<OWNER>>@<DOMAIN> …<rest of the arguments>
Based on https://hadoop.apache.org/docs/r3.1.2/hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html we should adopt a similar strategy that Spark uses - “AM keytab distributed via YARN; AM regenerates delegation tokens for containers”
Basically we need to re-implement logic of these classes:
The difference from Spark is that we do not implement our own application master, however, in the h2o setting we should be able to do the same on the cluster leader node.
Other useful links:
http://mkuthan.github.io/blog/2016/09/30/spark-streaming-on-yarn/ - says just enabling renewal options might not be enough due to bugs in HDFS (and Spark)