【Ambari开启Kerberos认证】Test Kerberos Client报错：Failed to kinit

温馨提示
本文内容在 Ubuntu 22.04 系统 下完成测试，相关系列系统在配置路径与命令格式上基本一致。
如在部署中遇到问题，可联系作者共同探讨或反馈。
👉 联系入口在此页

一、问题背景与报错原文

1、触发场景

• Ambari Web → Kerberos → Test Kerberos Client / KERBEROS_SERVICE_CHECK。
• Ambari 需要以 KDC 管理员 身份执行一次 kinit。

2、日志摘录（Ambari Server）

2025-11-04 02:41:01,485 INFO  [ambari-client-thread-43] o.a.a.s.c.AmbariManagementControllerImpl:4152 - Received action execution request, clusterName=aaaa, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{HAS_RESOURCE_FILTERS=true}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :aaaa
2025-11-04 02:41:01,485 INFO  [ambari-client-thread-43] o.a.a.s.c.AmbariManagementControllerImpl:4152 - Received action execution request, clusterName=aaaa, request=isCommand :true, action :null, command :KERBEROS_SERVICE_CHECK, inputs :{HAS_RESOURCE_FILTERS=true}, resourceFilters: [RequestResourceFilter{serviceName='KERBEROS', componentName='null', hostNames=[]}], exclusive: false, clusterName :aaaa
2025-11-04 02:41:01,720 WARN  [ambari-client-thread-43] o.a.a.s.s.k.KDCKerberosOperationHandler:329 - Failed to kinit as the KDC administrator user, admin/admin@TTBIGDATA.COM:
        ExitCode: 1
        STDOUT:
        STDERR: kinit.real: Server not found in Kerberos database while getting initial credentials

2025-11-04 02:41:01,720 WARN  [ambari-client-thread-43] o.a.a.s.s.k.KDCKerberosOperationHandler:329 - Failed to kinit as the KDC administrator user, admin/admin@TTBIGDATA.COM:
        ExitCode: 1
        STDOUT:
        STDERR: kinit.real: Server not found in Kerberos database while getting initial credentials

2025-11-04 02:41:01,722 ERROR [ambari-client-thread-43] o.a.a.s.c.KerberosHelperImpl:2501 - Cannot validate credentials: org.apache.ambari.server.serveraction.kerberos.KerberosAdminAuthenticationException: Invalid KDC administrator credentials.
The KDC administrator credentials must be set as a persisted or temporary credential resource.This may be done by issuing a POST (or PUT for updating) to the /api/v1/clusters/:clusterName/credentials/kdc.admin.credential API entry point with the following payload:
{
  "Credential" : {
    "principal" : "(PRINCIPAL)", "key" : "(PASSWORD)", "type" : "(persisted|temporary)"}
  }
}
2025-11-04 02:41:01,722 ERROR [ambari-client-thread-43] o.a.a.s.c.KerberosHelperImpl:2501 - Cannot validate credentials: org.apache.ambari.server.serveraction.kerberos.KerberosAdminAuthenticationException: Invalid KDC administrator credentials.
The KDC administrator credentials must be set as a persisted or temporary credential resource.This may be done by issuing a POST (or PUT for updating) to the /api/v1/clusters/:clusterName/credentials/kdc.admin.credential API entry point with the following payload:
{
  "Credential" : {
    "principal" : "(PRINCIPAL)", "key" : "(PASSWORD)", "type" : "(persisted|temporary)"}
  }
}
2025-11-04 02:41:01,722 ERROR [ambari-client-thread-43] o.a.a.s.api.handlers.CreateHandler:80 - Bad request received: Invalid KDC administrator credentials.
The KDC administrator credentials must be set as a persisted or temporary credential resource.This may be done by issuing a POST (or PUT for updating) to the /api/v1/clusters/:clusterName/credentials/kdc.admin.credential API entry point with the following payload:
{
  "Credential" : {
    "principal" : "(PRINCIPAL)", "key" : "(PASSWORD)", "type" : "(persisted|temporary)"}
  }
}
java.lang.IllegalArgumentException: Invalid KDC administrator credentials.
The KDC administrator credentials must be set as a persisted or temporary credential resource.This may be done by issuing a POST (or PUT for updating) to the /api/v1/clusters/:clusterName/credentials/kdc.admin.credential API entry point with the following payload:
{
  "Credential" : {
    "principal" : "(PRINCIPAL)", "key" : "(PASSWORD)", "type" : "(persisted|temporary)"}
  }
}
        at org.apache.ambari.server.controller.AmbariManagementControllerImpl.createAction(AmbariManagementControllerImpl.java:4198)
        at org.apache.ambari.server.controller.internal.RequestResourceProvider$1.invoke(RequestResourceProvider.java:282)
        at org.apache.ambari.server.controller.internal.RequestResourceProvider$1.invoke(RequestResourceProvider.java:213)
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.invokeWithRetry(AbstractResourceProvider.java:465)
        at org.apache.ambari.server.controller.internal.AbstractResourceProvider.createResources(AbstractResourceProvider.java:288)
        at org.apache.ambari.server.controller.internal.RequestResourceProvider.createResources(RequestResourceProvider.java:213)
        at org.apache.ambari.server.controller.internal.ClusterControllerImpl.createResources(ClusterControllerImpl.java:296)
        at org.apache.ambari.server.api.services.persistence.PersistenceManagerImpl.create(PersistenceManagerImpl.java:97)
        at org.apache.ambari.server.api.handlers.CreateHandler.persist(CreateHandler.java:50)
        at org.apache.ambari.server.api.handlers.BaseManagementHandler.handleRequest(BaseManagementHandler.java:68)
        at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:144)
        at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:164)
        at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:128)
        at org.apache.ambari.server.api.services.RequestService.createRequests(RequestService.java:231)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
        at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
        at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
        at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
        at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:299)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:135)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:95)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90)
        at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:112)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:82)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:221)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:186)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
        at org.apache.ambari.server.api.ContentTypeOverrideFilter.doFilter(ContentTypeOverrideFilter.java:145)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
        at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:73)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
        at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:53)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
        at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:135)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:51)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:690)
        at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:234)
        at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:223)
        at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:153)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.Server.handle(Server.java:503)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: org.apache.ambari.server.serveraction.kerberos.KerberosAdminAuthenticationException: Invalid KDC administrator credentials.

3、Ambari 源码定位（关键片段）

笔记
其实报错日志写了 KDCKerberosOperationHandler:329

ShellCommandUtil.Result result = executeCommand(getKinitCommand(executableKinit, credentials, credentialsCache, kerberosConfiguration),
        environmentMap,
        new InteractivePasswordHandler(String.valueOf(credentials.getKey()), null));
if (!result.isSuccessful()) {
  String message = String.format("Failed to kinit as the KDC administrator user, %s:\n\tExitCode: %s\n\tSTDOUT: %s\n\tSTDERR: %s",
      credentials.getPrincipal(), result.getExitCode(), result.getStdout(), result.getStderr());
  LOG.warn(message);
  throw new KerberosAdminAuthenticationException(message);
}

结论先行
Ambari 通过 Java 调用 shell 执行 kinit，返回码非 0 即判定失败。
核心是：弄清楚 Ambari 实际执行了哪条 kinit 命令，再对症修复。

二、快速判断与常见根因

1、五类高频根因（按命中率排序）

::: details 一句话判断
看到 Server not found，优先怀疑服务主体缺失；
看到 Client not found，优先看 admin/admin 是否存在。
:::

三、定位关键：捕获 Ambari 的 kinit 实参

1、思路说明

不改 Ambari 源码的前提下，用 shell 对系统 kinit 做一次“透明包装”，把调用参数与环境变量落到日志里。

2、包装脚本（支持回滚）

which kinit         # 记下路径，例如 /usr/bin/kinit
sudo mv /usr/bin/kinit /usr/bin/kinit.real
sudo tee /usr/bin/kinit >/dev/null <<'SH'
#!/usr/bin/env bash
echo "$(date) $$ argv: $0 $@" >> /var/log/kinit.args.log
env | grep -E '^KRB5' >> /var/log/kinit.args.log
exec /usr/bin/kinit.real "$@"
SH
sudo chmod +x /usr/bin/kinit

日志文件：/var/log/kinit.args.log

3、查看抓到的“真命令”

/usr/bin/kinit -c FILE:/var/lib/ambari-server/data/tmp/ambari_krb_1690981287658684697cc -S kadmin/dev1 admin/admin@TTBIGDATA.COM

关键信息

• -S kadmin/dev1：表示向 kadmin 服务主体 请求 TGS。
• 一旦 kadmin/dev1@REALM 在 KDC 数据库里不存在，就会命中 Server not found。

四、对症修复：补齐服务主体与 keytab（四）

1、创建缺失的服务主体并写入 keytab

# 查看是否存在 kadmin/* 主体
sudo kadmin.local -q "listprincs" | grep kadmin

# 若不存在则创建（随机密钥）
sudo kadmin.local -q "addprinc -randkey kadmin/dev1@TTBIGDATA.COM"

# 写入 kadm5.keytab（让 kadmind 服务可用）
sudo kadmin.local -q "ktadd -k /etc/krb5kdc/kadm5.keytab kadmin/dev1@TTBIGDATA.COM"

# 再次确认
sudo kadmin.local -q "listprincs" | grep kadmin

RHEL/CentOS/Kylin 路径通常为：/var/kerberos/krb5kdc/kadm5.keytab。
Ubuntu/Debian 路径通常为：/etc/krb5kdc/kadm5.keytab。

2、用抓到的“真命令”复测

/usr/bin/kinit -c FILE:/var/lib/ambari-server/data/tmp/ambari_krb_xxx -S kadmin/dev1 admin/admin@TTBIGDATA.COM

成功后回到 Ambari，再次执行 Test Kerberos Client，应为绿勾：

验证闭环

• kadmin -p admin/admin@REALM -q "get_privs" 能看到权限；
• klist 能看到 krbtgt/REALM@REALM 票据；
• Ambari Service Check 通过。

五、补充校验清单（五）

1、配置与服务

1. krb5.conf 的 default_realm、[realms] 与 Ambari 页面填写一致；
2. kdc.conf 中 acl_file、admin_keytab 路径无误；
3. kadm5.acl 至少包含：*/admin@REALM *；
4. 端口：88/tcp,udp（krb5kdc），749/tcp（kadmind）开放；
5. 服务：krb5kdc、kadmind（或 krb5-kdc、krb5-admin-server）为 active (running)。

2、主体与口令

1. kadmin/host@REALM 等服务主体存在（本问题的关键）；
2. admin/admin@REALM 口令正确，可手动 kinit 成功；
3. Ambari 中已持久化 KDC 管理员凭证（必要时走 REST API）。

3、环境差异

六、另一种定位路径：重编译打点（不推荐生产）

1、做法

• 在 KDCKerberosOperationHandler 的 getKinitCommand(...) 或调用处加详细日志；
• 重新编译替换 Ambari Server 的相应 jar 包，重启后观察调用栈与实参。

2、为什么不推荐

• 上线成本高、回滚复杂；
• shell 包装法零侵入、成本低且可随时回退。